12 pages tagged

#llm-training

chinchilla-scaling concept Compute-optimal data/params rule; models over-train past it; FLOPs, not params, predict quality. data-engineering concept Data recipe = capability design; "models must get bigger before they can get smaller". distillation concept Teacher-to-student capability transfer; diffusion staircase; release ≠ rightmost checkpoint. eval-grader-reward concept Training-time eval/grader/reward loop; ORM vs PRM; the grader is the critical failure point. grpo concept Group Relative Policy Optimization; drops PPO's value network; default for verifiable-reward RL. llm-training-pipeline concept Six-layer / nine-stage frame; the back half decides perceived capability. mixture-of-experts concept MoE: scale params without scaling per-token compute; routing and infra complexity. post-training concept SFT/RLHF/DPO/RFT routes; DeepSeek-R1 four-stage recipe; SFT teaches style as much as knowledge. pretraining concept Floor not ceiling; tokenizer/context/multimodal commitments lock in here; pretraining = RL priors. reasoning-models concept o1 / R1 paradigm; the second scaling axis (inference compute); effort as the user-facing dial. reward-hacking concept Reward overfitting → hacking → tampering → alignment faking; acute in self-improvement loops. 2026-04-27-llm-training-principles-paths-practices source Tw93's third essay: post-training, eval/reward, harness, distillation — the pipeline's back half.
esc