Alibaba previews Qwen4 architecture with open-weight Qwen3.8-Flash-Next

Alibaba's Qwen team released Qwen3.8-Flash-Next, an open-weight experimental preview of the architecture that will underpin Qwen4. It is a multimodal Mixture-of-Experts model with 125B total parameters but only 6B active per token, targeting near-frontier performance on hardware a serious individual can actually own.

Bottom line — Qwen3.8-Flash-Next activates only 6B of its 125B parameters per token, making it the most accessible frontier-class open-weight model to date.

Go deeper (9)

• Qwen3.8-Flash-Next uses a sparse Mixture-of-Experts architecture with a novel hybrid attention mechanism combining Gated Delta Network (GDN) layers and Qwen Sparse Attention (QSA), according to the Qwen team’s Hugging Face card.
• The model includes a 51B-parameter n-gram embedding table, described by the Qwen team as a fast local token lookup for cheap token prediction, potentially acting as an integrated draft model for speculative decoding.
• On Qwen’s own launch table, Flash Next outperforms Claude Opus 4.6 Max on eight of nine shared language benchmarks and leads DeepSeek V4 Flash on eight of nine shared rows, per the Qwen team’s measurements.
• The checkpoint is 354 GB in BF16; 4-bit quantized versions from Atomic Chat reduce memory footprint to around 55 GB of fast memory with the n-gram table paged from SSD, enabling it to run on a 64 GB MacBook Pro M5 Max at 36 tokens per second.
• Alibaba claims training cost for Flash Next was about one-ninth that of Qwen3.7-Plus, per SaaSCity citing community analysis of the briefly visible ModelScope card.
• The model ships under the Qwen Community License 1.0, which allows commercial use but requires a separate license for API-as-a-service products or for products exceeding 100 million MAU or €18.5 million monthly revenue (USD20 million at time of writing).
• Unsloth, vLLM, SGLang, and llama.cpp are all working on kernel support for the new attention mechanisms, according to multiple community reports.
• Ollama and LM Studio already publish quantized tags; hardware minimum is 64 GB unified memory for the smallest 4-bit build.
• Qwen3.8-Flash-Next supports native 256K context and can be extended to 1M tokens using YaRN RoPE scaling, per the Hugging Face documentation.

Read the reporting