Describe what you need.We'll find the right AI.

Agens Volundr 32B Preview: hybrid architecture keeps KV cache on only 18 of 72 layers

Blockway, a small team in Hong Kong, released Agens Volundr 32B Preview, their first model built on a custom hybrid architecture (Apache-2.0). The dense ~32B model has 72 layers, of which only 18 keep a KV cache: 54 KDA (Kimi Delta Attention) layers with fixed-size recurrent state and no KV cache, 17 BCSA compressed-sparse attention layers (exact 4,096-token window plus pooled blocks with a learned indexer), and 1 full-attention layer. It also includes a hashed n-gram memory (Engram) in host RAM and 4 residual streams (mHC), with a 262K context window. Benchmarks show it ahead of Qwen3.8-27B on LiveCodeBench v6, HumanEval, AIME 2025, and MATH-500, roughly level on MMLU-Pro, IFEval, and GPQA Diamond, but behind on agent tasks (tau2-bench, SWE-bench Verified). Known limitations: requires their custom sglang build (stock sglang/vLLM can't load it), no GGUF/llama.cpp support yet, and weak long agentic sessions. BF16 and INT4 weights plus a DFlash2 speculative drafter are on Hugging Face, with Docker images for 48GB Ada and H100/H200 GPUs.