Qwen3.8-Flash-Next Brings 262K-Context MoE to Consumer GPUs

Alibaba's Qwen team released an experimental preview of Qwen3.8-Flash-Next, a 125B-parameter sparse MoE model that activates only 6B per token and natively handles 262,144 tokens. Optimised for four RTX 3090s via FP8 KV caching and speculative decoding, it targets research and agentic workloads on hardware widely available in European labs.

Bottom line — 109 GB at Q4_K_M and 163–174 tok/s on four consumer cards make long-context AI affordable outside the data centre.

Go deeper (7)

• The model alternates three Gated DeltaNet blocks (recurrent compression) with one Qwen Sparse Attention block (precise long-range retrieval), per the vLLM recipe published on vllm.ai.
• A maintainer-reported gate compared an Intel-authored quantised checkpoint against a VnimanieAI baseline; all three point estimates favoured Intel, though none reached statistical significance.
• On H100s (80 GB VRAM) the 51B n-gram embedding table requires CPU offload; the recipe automatically enables it and sustains about 1,430 output tok/s at concurrency 64 on random workloads, according to vllm.ai.
• MTP speculative decoding degraded throughput by 8–36% on H100s due to a low acceptance rate; vllm.ai recommends disabling it by default on that hardware.
• The DFlash2 drafter quantisation pipeline, described on DeepWiki, compresses the 1.92B-parameter external drafter from BF16 (3.85 GB) to W4A16 (1.19 GB) for single-user speculative decoding on RTX 3090.
• The checkpoint is licensed under Qwen Community License 1.0, which permits commercial use subject to acceptable-use terms, per llmconfigurator.com.
• A maintainer stress test on four RTX 3090s reached 209,683 tokens before running short of the 262,144 limit; the script flagged 753 MB of free VRAM against a 1,024 MB safety threshold.

Read the reporting