Gemma 3 4B by Google DeepMind is an open-weights multimodal model with 128K context, SigLIP pan-and-scan vision, 5:1 local/global attention, and 140+ language support.
Capabilities, design details, and architectural traits
Gemma 3 4B is a multimodal, open-weights model from Google DeepMind, built from the same research used to create Gemini. It is the smallest size in the Gemma 3 family to support image input - the 1B variant is text-only. Despite its size, it shares the full 128K-token context window with the 12B and 27B variants, not the 32K limit of the 1B. It targets deployment on resource-constrained hardware such as laptops and single GPUs.
| Trait | Detail |
|---|---|
| 5:1 local/global attention interleaving | Alternates 5 local sliding window layers (1024-token window) per 1 global attention layer, optimizing KV-cache memory for long contexts |
| SigLIP vision encoder with pan-and-scan | Images normalized to 896x896; pan-and-scan adaptively crops non-square or high-resolution images into tiles, each re-encoded separately to preserve detail |
| 128K context window | Shared with 12B and 27B; not available in the 1B size |
| Open weights in two variants | Released as both a pre-trained base and an instruction-tuned (-it) version |
| Distillation-based post-training | Instruction-tuned variant trained using knowledge distillation and reinforcement learning |
| Multilingual scope | Supports over 140 languages |
The 5:1 local-to-global attention ratio is a documented change from Gemma 2, which used a 1:1 ratio with a 4096-token local window. In Gemma 3, the local window is reduced to 1024 tokens and the ratio shifts to 5:1, reducing KV-cache memory use without degrading language model perplexity. This pattern applies across the 4B, 12B, and 27B sizes.
Independent evaluations · Artificial Analysis
Evaluate specifications, pricing, and independent benchmark indices
| Model Details | |||
|---|---|---|---|
| General Info | |||
| Provider | Anthropic | Anthropic | |
| Release Date | March 12, 2025 | July 24, 2026 | June 9, 2026 |
| Knowledge Cutoff | Aug 2024 | May 2026 | - |
| Context & Limits | |||
| Context Window | 131K | 1M Best Context Window | 1M Best Context Window |
| Pricing (per 1M tokens) | |||
| Input Pricing | Free Best Input Pricing | $5 | $10 |
| Output Pricing | Free Best Output Pricing | $25 | $50 |
| Modalities | |||
| Inputs | textimage | textimage | textimagefile |
| Outputs | text | text | text |
| Benchmarks (0-100) | |||
| Intelligence Index | 1.0 | 63.1 Best Intelligence Index | 62.1 |
| Coding Index | 2.7 | 78.0 Best Coding Index | 76.5 |
| Agentic Index | - | 59.2 Best Agentic Index | 56.6 |
Graduate-level reasoning and expert Q&A evaluation.
Extremely difficult logical reasoning and knowledge.
Logical reasoning over long context windows.
Independent evaluation data provided by Artificial Analysis. To view the latest benchmarks and full details, visit their official site.
