$ the-wire · showcase
SGLang drops its 0.5.x flags, llama.cpp fixes FP8 qwen4exp conversion
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
SGLang 0.5.x removals now hard-fail at launch with a message naming the replacement, while llama.cpp unblocks conversion of the Qwen3.8-Flash-Next FP8 checkpoints that stopped on master.
[Refactor] server_args: Remove deprecated flags, choice values and environment variables sgl-project/sglang
Every spelling deprecated in 0.5.x is gone from server_args.py: a launch that passes one of the flags in REMOVED_FLAGS, on the command line or inside a --config YAML, now stops with a message naming what to use instead, rather than being remapped with a warning or silently ignored. That turns old launch scripts into hard failures, and the mapping is not always one-to-one: --disable-cuda-graph b...
convert : fix FP8 qwen4exp checkpoints (n-gram table scale, block-scaled FP8) ggml-org/llama.cpp
FP8 qwen4exp checkpoints (Qwen/Qwen3.8-Flash-Next-FP8, nvidia/Qwen3.8-Flash-Next-NVFP4) store the n-gram PLE table as 128 F8_E4M3 shards behind a single shared BF16 scale and ship no ngram_embedding.weight, so conversion on master aborted with a ValueError naming the tensor it could not map. The fix handles both that layout and block-scaled FP8, which unblocks anyone converting these checkpoint...
[Frontend] Support output logprobs for gpt-oss in the Responses API vllm-project/vllm
Setting include=["message.output_text.logprobs"] on /v1/responses used to return a 400 for gpt-oss; output-text logprobs now work in both streaming and non-streaming paths, kept in parity. The plumbing is a token_index on each HarmonyParser Segment plus HarmonyContext picking logprobs for output-text tokens, closing the logprobs-plus-harmony TODO left from earlier work.
[Fix] diffusion/security: Use safe_pickle_loads for DiffusionServer inbound requests (#49290) (#49290) sgl-project/sglang
DiffusionServer inbound requests now go through safe_pickle_loads instead of the default unpickler, which removes arbitrary pickle deserialization from the request path for anyone serving diffusion models through SGLang.
[Feat] [CPU] quantization: Add CPU FP8/INT8 quant ops and compressed-tensors support sgl-project/sglang
The long tail: SGLang added CPU FP8/INT8 quantization ops for Xeon/AMX so compressed-tensors linear schemes run through in-tree sgl_kernel instead of falling into CUDA/Triton-only paths, and llama.cpp landed CUDA few-row MMQ for MUL_MAT_ID at 2 to 24 tokens plus Minimax-Music3 in mtmd, while MXFP8 GQA KV caches on NPU now round-trip through L2 HiCache.
Action items
- → Rewrite SGLang launch commands and YAML configs that use removed 0.5.x flags: --disable-cuda-graph becomes --disable-... sgl-project/sglang [immediate]
- → Update SGLang to a build containing safe_pickle_loads for DiffusionServer inbound requests before exposing a diffusio... sgl-project/sglang [plan]