04 / SOURCE REFERENCE

Inside the implementation.

A file-by-file reference for the PyTorch language backbone, training system, and supporting tools. Choose a file to read its implementation walkthrough or inspect the exact source revision.

37 Python files28 in src/9 in scripts/

Repository explorer

Folders follow the original source hierarchy. Files without a completed review remain listed but are not linked to unfinished documentation.

src/28 files
- kimi_k3/28 files
  - __init__.pyPackage root; no public symbols defined here.Source ↗
  - model.pyDecoder backbone, AttnRes, configuration, and language-model forward pass.Source ↗
  - evaluation.pyEvaluation logic and metrics interfaces.Source ↗
  - inference.pyToken generation and inference routines.Source ↗
  - attention/5 files
    __init__.pyAttention package exports.Source ↗
    gated_mla.pyGated multi-head latent attention.Source ↗
    kda/3 files
    recurrent.pyTokenwise KDA reference implementation.Source ↗
    chunkwise.pyChunkwise KDA with triangular transformation and recurrent state.Source ↗
    fused_chunk.pyFLA-backed fused KDA integration.Source ↗
  - data/2 files
    __init__.pyPublic data-loader exports.Source ↗
    dataset.pyPrepared token shards and streaming causal batches.Source ↗
  - ffn/2 files
    __init__.pyFeed-forward module exports.Source ↗
    dense_ffn.pyDense FFN and SiTU-GLU activation.Source ↗
  - moe/2 files
    __init__.pyMixture-of-experts package export.Source ↗
    stable_latent_moe.pyStable LatentMoE routing, expert execution, and balancing.Source ↗
  - training/10 files
    __init__.pyMinimal training package interface.Source ↗
    trainer.pyTraining steps, precision, gradients, and checkpoint handling.Source ↗
    run.pyRun lifecycle and artifact organization.Source ↗
    optimizers.pyMuon and AdamW optimization implementations.Source ↗
    parameter_groups.pyAssignment of parameters to optimizer groups.Source ↗
    qk_clip.pyQK clipping utilities.Source ↗
    metrics.pyMetrics logging and records.Source ↗
    moe_metrics.pyMoE monitoring metrics.Source ↗
    monitoring.pyTraining diagnostics and monitoring.Source ↗
    profiling.pyRuntime and memory profiling support.Source ↗
  - utils/3 files
    __init__.pyUtility package exports.Source ↗
    config.pyConfiguration loading utilities.Source ↗
    rmsn.pyMixed-precision RMSNorm.Source ↗
scripts/9 files
- train.pyTraining command and CLI orchestration.Source ↗
- evaluate.pyEvaluation entry point.Source ↗
- generate.pyText generation entry point.Source ↗
- prepare_data.pyCorpus preparation pipeline.Source ↗
- inspect_data.pyInspect prepared dataset artifacts.Source ↗
- inspect_config.pyInspect resolved model configuration.Source ↗
- summarize_training_run.pySummarize training-run evidence.Source ↗
- bench_muon.pyMuon benchmarking utility.Source ↗
- convert_assets_to_webp.pyWebsite asset conversion and Markdown-link maintenance.Source ↗

Documented source revision: 10c0dcfa8d2b ↗