Llama 4 Maverick

Llama 4 Maverick - 128-Expert MoE General-Purpose Multimodal Model

Llama 4 Maverick is Meta's open-weight general-purpose model in the Llama 4 series. It uses a mixture-of-experts (MoE) architecture with 128 routed experts plus one shared expert. Each token is routed to the shared expert and exactly one of the 128 routed experts per MoE layer.

Architecture

Maverick uses alternating dense and MoE layers - not a pure MoE stack - combined with early fusion for native multimodality, integrating text and image inputs at the token level. It is released in both BF16 and FP8 quantized weights.

Context Strategy

Maverick supports a 1 million token context window, positioned for applications requiring extensive document or multi-image reasoning.

Training Data

Pretrained on approximately 22 trillion tokens of multimodal data, including publicly available content, licensed data, and Meta platform data (Instagram, Facebook, Meta AI interactions).