Gemma 3 4B
Gemma 3 4B by Google DeepMind is an open-weights multimodal model with 128K context, SigLIP pan-and-scan vision, and support for over 140 languages.
Model Overview
Capabilities, design details, and architectural traits
Gemma 3 4B - Smallest Open-Weights Gemma 3 Model
Gemma 3 4B is a multimodal, open-weights model from Google DeepMind, built from the same research used to create Gemini. It is the smallest size in the Gemma 3 family to support image input - the 1B variant is text-only. Despite its size, it shares the full 128K-token context window with the 12B and 27B variants, not the 32K limit of the 1B.
Architecture and Vision Handling That Define the 4B
| Trait | Detail |
|---|---|
| 5:1 local/global attention interleaving | Alternates 5 local sliding window layers (1024-token window) per 1 global attention layer, optimizing KV-cache memory for long contexts |
| SigLIP vision encoder with pan-and-scan | Images normalized to 896x896; pan-and-scan adaptively crops non-square or high-resolution images into tiles, each re-encoded separately to preserve detail |
| 128K context window | Shared with 12B and 27B; not available in the 1B size |
| Open weights in two variants | Released as both a pre-trained base and an instruction-tuned (-it) version |
| Distillation-based post-training | Instruction-tuned variant trained using knowledge distillation and reinforcement learning |
| Multilingual scope | Supports over 140 languages |
Attention Design Specific to Gemma 3
The 5:1 local-to-global attention ratio is a documented change from Gemma 2, which used a 1:1 ratio with a 4096-token local window. In Gemma 3, the local window is reduced to 1024 tokens and the ratio shifts to 5:1, reducing KV-cache memory use without degrading language model perplexity. This pattern applies across the 4B, 12B, and 27B sizes.
Benchmark Performance
Independent evaluations · Artificial Analysis
Accuracy & Capability Details
Compare Models Side-by-Side
Evaluate specifications, pricing, and independent benchmark indices
| Model Details | |||
|---|---|---|---|
| General Info | |||
| Provider | Anthropic | Anthropic | |
| Release Date | March 12, 2025 | September 22, 2026 | September 28, 2026 |
| Knowledge Cutoff | Aug 2024 | - | Jun 2026 |
| Context & Limits | |||
| Context Window | 131K | 1M Best Context Window | 1M Best Context Window |
| Pricing (per 1M tokens) | |||
| Input Pricing | Free Best Input Pricing | $4 | $2 |
| Output Pricing | Free Best Output Pricing | $20 | $10 |
| Modalities | |||
| Inputs | textimage | textimagefile | textimagefile |
| Outputs | text | text | text |
| Benchmarks (0-100) | |||
| Intelligence Index | 4.8 | 57.6 Best Intelligence Index | 56.0 |
| Coding Index | 2.7 | - | - |
| Agentic Index | - | - | - |
Humanity's Last Exam
Extremely difficult logical reasoning and knowledge.
Long Context Reasoning
Logical reasoning over long context windows.
SciCode Benchmark
Scientific coding and mathematical modeling.
Independent evaluation data provided by Artificial Analysis. To view the latest benchmarks and full details, visit their official site.
Explore more from Google
Other models by Google
Gemini 4 Argon
Gemini 3.8 Flash
Gemini 3.7 Flash
Gemini 3.6 Flash
Top AI Models
Leading alternatives by intelligence score
