Phi 4

Phi 4 - Data-quality focused small language model

Phi 4 is a 14-billion-parameter language model from Microsoft positioned as a small but reasoning-focused model developed with a training recipe centered on data quality and extensive synthetic data generation, and released with multimodal and compact variants in the Phi family.

Defining Characteristics

• Data-quality training recipe: Phi 4’s development emphasizes curated high-quality data and synthetic data integrated throughout pretraining rather than relying primarily on organic web or code data.
• Parameter scale: Phi 4 is documented as a 14B-parameter model in official technical reports.
• Post-training and alignment: The model uses supervised fine-tuning and direct preference optimization to improve instruction following and safety.

Variants & Multimodality

• Officially published variants include Phi-4-Mini (3.8B) and Phi-4-Multimodal; the multimodal variant supports text, vision, and speech/audio via modality adapters and routing mechanisms.
• Phi-4-Mini expands vocabulary (200K tokens) and adopts group query attention for more efficient long-sequence generation, per the authors' reports.

Context Strategy & Use Cases

• Documented context length for Phi 4 inference is 16K tokens, making it suitable for longer-context reasoning within the small-model class.
• Microsoft positions Phi 4 as a building block for research and latency/memory-constrained deployments, with emphasis on English-language capabilities.