Nemotron 3 Ultra
Nemotron 3 Ultra by NVIDIA: 550B-total (55B-active) Mixture-of-Experts hybrid with 1M-token context, LatentMoE/MTP layers, and MOPD post-training.
Model Overview
Capabilities, design details, and architectural traits
Nemotron 3 Ultra - NVIDIA's 550B Mixture-of-Experts hybrid
Nemotron 3 Ultra is NVIDIA's flagship Nemotron 3 family model that combines a sparse Mixture-of-Experts design with hybrid Mamba-attention to prioritise long-context reasoning and high inference throughput. It ships with open checkpoints, training blends, and post-training recipes used in a multi-stage MOPD pipeline.
| Trait | Documented detail that identifies Nemotron 3 Ultra |
|---|---|
| Architecture name | Hybrid Mamba-Transformer Mixture-of-Experts using LatentMoE and Mamba-Attention layers, explicit to Nemotron 3 Ultra |
| Parameters & sparsity | 550 billion total parameters with 55 billion active parameters (sparse MoE footprint) |
| Context window | Supports up to 1M token context natively, documented as a core capability |
| Inference features | Includes MTP layers for native speculative decoding and controls for inference-time reasoning budget |
| Post-training recipe | Post-trained with Supervised Fine Tuning, RL (Student RLVR / RLHF) and Multi-Teacher On-Policy Distillation (MOPD) - published training recipes and configs |
| Open release scope | Open release includes pre-trained, post-trained, and quantized checkpoints plus training data and recipes under NVIDIA's release artifacts |
Defining technical ideas
-
LatentMoE: Nemotron 3 Ultra explicitly uses a latent mixture-of-experts mechanism in its hybrid architecture, a central design point that distinguishes it from dense models.
-
MOPD pipeline: The published training and post-training flow centres on Multi-Teacher On-Policy Distillation combined with RLVR and specialised teacher roles, which the documentation presents as the official post-training program for Ultra.
Short usage notes
-
The model is presented and released as open-weight artifacts including NVFP4 and BF16 checkpoints and matching datasets and recipes.
-
Documentation emphasises long-context applications and inference throughput improvements via the hybrid MoE design and MTP speculative decoding.
Benchmark Performance
Independent evaluations · Artificial Analysis
Accuracy & Capability Details
Compare Models Side-by-Side
Evaluate specifications, pricing, and independent benchmark indices
| Model Details | |||
|---|---|---|---|
| General Info | |||
| Provider | NVIDIA | Anthropic | Anthropic |
| Release Date | June 4, 2026 | September 22, 2026 | September 28, 2026 |
| Knowledge Cutoff | Sep 2025 | - | Jun 2026 |
| Context & Limits | |||
| Context Window | 1M | 1M | 1M |
| Pricing (per 1M tokens) | |||
| Input Pricing | $0.60 Best Input Pricing | $4 | $2 |
| Output Pricing | $2.50 Best Output Pricing | $20 | $10 |
| Modalities | |||
| Inputs | text | textimagefile | textimagefile |
| Outputs | text | text | text |
| Benchmarks (0-100) | |||
| Intelligence Index | 22.9 | 57.6 Best Intelligence Index | 56.0 |
| Coding Index | 49.3 | - | - |
| Agentic Index | 20.1 | - | - |
Humanity's Last Exam
Extremely difficult logical reasoning and knowledge.
Long Context Reasoning
Logical reasoning over long context windows.
SciCode Benchmark
Scientific coding and mathematical modeling.
Independent evaluation data provided by Artificial Analysis. To view the latest benchmarks and full details, visit their official site.
Explore more from NVIDIA
Other models by NVIDIA
Nemotron 3.5 Lightning
Nemotron 3 Super
Nemotron 3 Nano Omni
Llama 3.3 Nemotron Super 49B V1.5
Top AI Models
Leading alternatives by intelligence score
