entity · created Apr 27, 2026 · updated Apr 27, 2026 · cited by 1 source

deepseek

#organization#ai-lab#open-weights

DeepSeek (深度求索) — Chinese AI lab; the most-cited single organization in 2026-04-27-llm-training-principles-paths-practices. The wiki tracks DeepSeek primarily for its public training reports, which are unusually detailed about systems and post-training engineering compared to other frontier labs.

What this wiki currently knows

Models referenced

  • DeepSeek-V3 — the public engineering reference for frontier-scale stability. Pretraining: 14.8T tokens, ~2.788M H800 GPU hours, no irrecoverable loss spikes, no rollbacks. One of the few public verifications of FP8 mixed-precision at frontier scale (pretraining, 2026-04-27-llm-training-principles-paths-practices).
  • DeepSeek-R1 — the reasoning model that defined the public 4-stage post-training recipe (cold-start SFT → reasoning RL → rejection sampling FT → alignment RL). The reference example for reasoning-models alongside OpenAI’s o1.
  • DeepSeek-R1-Zero — the ablation that mattered: pure RL on a base model is technically feasible but produces repetition, language mixing, and unreadable output. Cited in the article as the reason cold-start SFT is not optional in post-training.
  • DeepSeek-R1-Distill series — small (1.5B–70B) dense models trained on R1’s RL trajectories. The public reference for the distillation thesis: capabilities form at scale, then transfer to smaller models. Concrete evidence for “models must get bigger before they can get smaller.”

Algorithms / techniques attributed

  • GRPO (Group Relative Policy Optimization) — introduced in DeepSeekMath (Shao et al. 2024), used as the reasoning-RL algorithm in R1. Eliminates PPO’s separate value network by using group-internal rank as the advantage baseline. (2026-04-27-llm-training-principles-paths-practices)
  • MoE design at frontier scale — DeepSeek-V3 is one of the canonical worked examples of MoE as a cost/effect compromise.

Stance

Of the labs referenced in the wiki so far, DeepSeek is the one whose published technical reports do the most of the work for outside readers — DeepSeek-V3’s report is what the article cites for FP8 viability and stability claims, and the R1 paper is the canonical public account of a four-stage post-training pipeline. If a future ingest contradicts a number from these reports, treat the contradiction as load-bearing — they’re being cited as the empirical backbone of several concept pages.

Referenced by 8

2026-04-27-llm-training-principles-paths-practices data-engineering distillation grpo mixture-of-experts post-training pretraining reasoning-models
esc