Llama 4 Maverick

Llama 4 Maverick - 128-Expert MoE General-Purpose Multimodal Model

Llama 4 Maverick is Meta's open-weight general-purpose model in the Llama 4 series. It uses a mixture-of-experts (MoE) architecture with 128 routed experts plus one shared expert, totaling 400B parameters while activating only 17B parameters per token. Each token is routed to the shared expert and exactly one of the 128 routed experts per MoE layer.

Architecture

Maverick uses alternating dense and MoE layers — not a pure MoE stack — combined with early fusion for native multimodality, integrating text and image inputs at the token level. It is released in both BF16 and FP8 quantized weights, with the FP8 version fitting on a single H100 DGX host.

Context Strategy

Maverick supports a 1 million token context window, positioned for applications requiring extensive document or multi-image reasoning within a single host deployment.

Training Data

Pretrained on approximately 22 trillion tokens of multimodal data, including publicly available content, licensed data, and Meta platform data (Instagram, Facebook, Meta AI interactions).