$ the-wire · showcase
LLAMA.CPP UNLEASHES QWEN3.8-FLASH-NEXT SUPPORT
By RepoJournal · Filed · About Local LLMs
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4)
16 people shipped this
ngxson
@ngxson
2 cited
hoyyeva
@hoyyeva
1 cited
github-actions[bot]
@github-actions[bot]
1 cited
danielhanchen
@danielhanchen
1 cited
Xarbirus
@Xarbirus
1 cited
Misha Goin
@mgoin
1 cited
zijiecode
@zijiecode
1 cited
billishyahao
@billishyahao
1 cited
dhiltgen
@dhiltgen
shenron0101
@shenron0101
MatthewBonanni
@MatthewBonanni
Ronald1995
@Ronald1995
jiahanc
@jiahanc
BBuf
@BBuf
aurickq
@aurickq
hnyls2002
@hnyls2002
The biggest local-model news in weeks: llama.cpp just landed draft support for Qwen3.8-Flash-Next, and it ships with three quantizer fixes the model needs.
Llama.cpp's draft PR adds the full Qwen3.8-Flash-Next stack: converter, text graph, sparse attention, vision, and the quantizer fixes, but it is held back because "the weights are not public yet" [1]. While you wait, two supporting patches are worth grabbing now: quantize caps working memory to 8GB by default to stop OOM on big tensors [2], and the new TENSOR_READ_LAZY flag lets PLE and engrams embeddings stream from disk instead of loading into RAM [3]. On the AMD side, SGLang shipped a gfx950 ASM FMHA kernel that rescues Qwen3.5 prefill, which was running at "roughly 15% of the GPU's FP8 peak" on MI355X [4], and the same repo enables the aiter MLA asm path on Kimi K3 by padding attention heads for TP=8 [5]. vLLM squashed a parser bug where a stale </think> from history broke reasoning detection on multi-turn prompts [6], and fixed a race condition in fused groupwise RMSNorm quantization [7]. Ollama released v0.33.2 overnight, restoring system dark mode and syncing macOS app handoff [8], but the real headline there is the Claude Desktop picker now includes signed-in cloud models beyond recommendations [9].
Action items
- → Watch llama.cpp PR 27742 for Qwen3.8-Flash-Next weights, but start testing --max-buffer-size and --tensor-read-lazy against your current models ggml-org/llama.cpp [monitor]
- → Update SGLang to get the MI355X ASM FMHA kernel and Kimi K3 MLA path if you run AMD GPUs sgl-project/sglang [plan]
- → Upgrade vLLM to pick up the reasoning parser scoping fix and the RMSNorm race fix vllm-project/vllm [plan]
- → Upgrade Ollama to v0.33.2 to get dark mode restore and cloud model selection in Claude Desktop ollama/ollama [immediate]
References
- [1] model: add Qwen3.8-Flash-Next (qwen4exp) ↗ ggml-org/llama.cpp
- [2] quantize: cap working memory size to avoid loading big tensors onto RAM ↗ ggml-org/llama.cpp
- [3] llama: model_loader: add TENSOR_READ_LAZY ↗ ggml-org/llama.cpp
- [4] [AMD] Qwen3.5 ASM FMHA chunked-prefill context attention ↗ sgl-project/sglang
- [5] [AMD] Enable aiter mla asm path through padding attn heads for Kimi K3 ↗ sgl-project/sglang
- [6] [Bugfix][Parser] Scope reasoning-end detection to the current turn via turn-boundary tokens ↗ vllm-project/vllm
- [7] [Bugfix] Remove race in fused groupwise RMSNorm quantization (#54111) vllm-project/vllm
- [8] v0.33.2 ↗ ollama/ollama
- [9] app: list account cloud models for Claude ↗ ollama/ollama
