DeepSeek R1

DeepSeek R1 - First-Generation Open Reasoning Model

DeepSeek R1 is DeepSeek's first-generation dedicated reasoning model. It is built on DeepSeek V3-Base and trained through a documented four-stage pipeline combining two RL stages and two SFT stages - designed specifically to produce long chain-of-thought reasoning without the readability and consistency problems exhibited by its predecessor, DeepSeek R1-Zero.

Training Pipeline

The four-stage training pipeline is the defining characteristic of DeepSeek R1:

1. Cold-start SFT: Fine-tuning on a small set of carefully curated examples to seed readable, structured reasoning before RL begins - addressing the endless repetition, poor readability, and language mixing exhibited by R1-Zero
2. First RL stage: Large-scale reinforcement learning using GRPO (Group Relative Policy Optimization) to discover improved reasoning patterns; verifiable rewards for math (ground-truth comparison) and code (reward model predicting unit test pass rates)
3. Second SFT stage: Rejection sampling to filter and fine-tune on high-quality outputs, seeding non-reasoning capabilities alongside reasoning
4. Second RL stage: Alignment with human preferences across both reasoning and non-reasoning behaviors.