Nemotron 3 Ultra
Nemotron 3 Ultra by NVIDIA: 550B-total (55B-active) Mixture-of-Experts hybrid with 1M-token context, LatentMoE/MTP layers, and MOPD post-training.
Model Overview
Capabilities, design details, and architectural traits
Nemotron 3 Ultra - NVIDIA's 550B Mixture-of-Experts hybrid
Nemotron 3 Ultra is NVIDIA's flagship Nemotron 3 family model that combines a sparse Mixture-of-Experts design with hybrid Mamba-attention to prioritise long-context reasoning and high inference throughput. It ships with open checkpoints, training blends, and post-training recipes used in a multi-stage MOPD pipeline.
| Trait | Documented detail that identifies Nemotron 3 Ultra |
|---|---|
| Architecture name | Hybrid Mamba-Transformer Mixture-of-Experts using LatentMoE and Mamba-Attention layers, explicit to Nemotron 3 Ultra |
| Parameters & sparsity | 550 billion total parameters with 55 billion active parameters (sparse MoE footprint) |
| Context window | Supports up to 1M token context natively, documented as a core capability |
| Inference features | Includes MTP layers for native speculative decoding and controls for inference-time reasoning budget |
| Post-training recipe | Post-trained with Supervised Fine Tuning, RL (Student RLVR / RLHF) and Multi-Teacher On-Policy Distillation (MOPD) — published training recipes and configs |
| Open release scope | Open release includes pre-trained, post-trained, and quantized checkpoints plus training data and recipes under NVIDIA's release artifacts |
Defining technical ideas
-
LatentMoE: Nemotron 3 Ultra explicitly uses a latent mixture-of-experts mechanism in its hybrid architecture, a central design point that distinguishes it from dense models.
-
MOPD pipeline: The published training and post-training flow centres on Multi-Teacher On-Policy Distillation combined with RLVR and specialised teacher roles, which the documentation presents as the official post-training program for Ultra.
Short usage notes
-
The model is presented and released as open-weight artifacts including NVFP4 and BF16 checkpoints and matching datasets and recipes.
-
Documentation emphasises long-context applications and inference throughput improvements via the hybrid MoE design and MTP speculative decoding.
Benchmark Performance
Independent evaluations · Artificial Analysis
Accuracy & Capability Details
Independent evaluation data provided by Artificial Analysis. To view the latest benchmarks and full details, visit their official site.
Compare Models Side-by-Side
Evaluate specifications, pricing, and independent benchmark indices
| Model Details | NVIDIAHost Nemotron 3 Ultra | ||
|---|---|---|---|
| General Info | |||
| Provider | NVIDIA | Anthropic | Anthropic |
| Release Date | June 4, 2026 | June 9, 2026 | May 28, 2026 |
| Knowledge Cutoff | — | — | — |
| Context & Limits | |||
| Context Window | 1M | 1M | 1M |
| Pricing (per 1M tokens) | |||
| Input Pricing | $0.68 Best Input Pricing | $10 | $5 |
| Output Pricing | $2.67 Best Output Pricing | $50 | $25 |
| Modalities | |||
| Inputs | text | textimagefile | textimagefile |
| Outputs | text | text | text |
| Benchmarks (0-100) | |||
| Intelligence Index | 37.8 | 59.9 Best Intelligence Index | 55.7 |
| Coding Index | 49.3 | 76.5 Best Coding Index | 74.3 |
| Agentic Index | 27.4 | 52.8 Best Agentic Index | 47.2 |
GPQA Benchmark
Graduate-level reasoning and expert Q&A evaluation.
Humanity's Last Exam
Extremely difficult logical reasoning and knowledge.
IFBench Instruction
Evaluation of strict instruction following.
Benchmark scores are independent evaluations sourced from Artificial Analysis. Intelligence, Coding, and Agentic indices reflect composite performance ratings (0–100).