DeepSeek V3.2 Exp

DeepSeek V3.2 Exp – Experimental Sparse Attention Architecture Validation Model

DeepSeek V3.2 Exp is DeepSeek's experimental release positioned explicitly as an intermediate step toward the next-generation architecture. It builds upon DeepSeek V3.1-Terminus by introducing DeepSeek Sparse Attention (DSA) - a fine-grained sparse attention mechanism designed to explore and validate long-context training and inference efficiency optimizations. The production V3.2 and V3.2-Speciale models share the same model structure as V3.2 Exp.

Architecture

V3.2 Exp shares the same underlying MoE structure as DeepSeek V3.2, with DeepSeek Sparse Attention (DSA) as the defining addition:

• DSA achieves fine-grained sparse attention - described officially as the first time fine-grained sparse attention of this kind was achieved - delivering substantial improvements in long-context training and inference efficiency while maintaining virtually identical output quality
• Uses a non-standard sparse attention variant requiring custom code for inference, with custom kernels released across three official repositories: TileLang (research-purpose kernels), DeepGEMM (high-performance CUDA indexer logit kernels including paged versions), and FlashMLA (sparse attention kernels)
• A documented implementation discrepancy in RoPE within the indexer module was identified and patched post-release: the indexer module requires a non-interleaved layout, while the MLA module expects an interleaved layout.