LFM2-24B-A2B

LFM2-24B-A2B - Edge-Deployable Hybrid MoE Model

LFM2-24B-A2B is Liquid AI's largest LFM2 model, applying the hybrid LFM2 backbone in a sparse Mixture-of-Experts configuration so a 24B-parameter model runs at roughly the cost of a 2B dense model.

Defining Characteristics

• Hybrid Backbone: 30 double-gated short-convolution blocks paired with 10 grouped-query-attention blocks (roughly a 1:3 attention-to-convolution ratio), with the first two layers kept dense for training stability.
• Sparse MoE Scaling: 64 experts per MoE block with top-4 routing; active parameters grow only ~1.5x (1.5B→2.3B) while total parameters grow 3x (8.3B→24B) versus the smaller LFM2-8B-A1B.
• Predictable Scaling: benchmark quality improves log-linearly from 350M to 24B total parameters, showing the architecture doesn't hit a ceiling at small sizes.

Why It Exists

Built to test whether the LFM2 hybrid backbone keeps scaling past its smaller siblings, while concentrating new capacity in total rather than active parameters so inference stays cheap enough for laptops and desktops with integrated GPUs or NPUs.