$ the-wire · showcase
Ollama stops overriding model sampler defaults on /v1
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
17 people shipped this
dhiltgen
@dhiltgen
1 cited
Mark Todorovich
@marktodorovich
1 cited
HangruiCao2000
@HangruiCao2000
1 cited
merrymercy
@merrymercy
1 cited
ronaldmannak
@ronaldmannak
1 cited
cabelo
@cabelo
shaofeiqi
@shaofeiqi
bosh
@boshjerns
Isotr0py
@Isotr0py
AMD-yanfeiwang
@AMD-yanfeiwang
TimRen01
@TimRen01
yzong-rh
@yzong-rh
libertyeagle
@libertyeagle
Codoge
@Codoge
metamergebot
@metamergebot
axiom-of-choice
@axiom-of-choice
Srijan Keshri
@arcusbuilds
Ollama's OpenAI-compatible endpoints no longer clobber the temperature, top_p and penalty values each model ships with, a silent behavior change that will alter output for anyone calling /v1 rather than /api/chat.
openai: don't override model sampler defaults ollama/ollama
When a client omitted temperature or top_p, /v1/chat/completions, /v1/completions and /v1/responses filled in 1.0, and /v1/completions always sent frequency_penalty and presence_penalty as 0. Because request options are applied last, those values replaced the model's packaged defaults: lfm3 ships temperature 0.1 but ran at 1.0 on /v1, and qwen3.8, qwen3.6, gemma4 and nemotron-3.5 all ship top_p...
hexagon : fix DMA transfers with strides > 24 bits (#31442) ggml-org/llama.cpp
On v75+ the 2D DMA descriptor stores row_size and stride in 24 bits, and the queue path only split nrows, so strides of 16 MB or more were truncated into wrong addresses on Hexagon. The fix falls back to one transfer per row in that case, and two new CONT tests exercise source strides over 16 MB.
[Bugfix][Model] Make compile_mm_encoder work for RADIO/NaViT encoders vllm-project/vllm
Serving a RADIO/NaViT encoder model such as nvidia/Nemotron-3.5-Parse with --compilation-config.compile_mm_encoder true raised 'NoneType' object has no attribute 'cu_seqlens' during aot_compile on the first image request, killing the engine with EngineDeadError rather than returning a 500, so the server refused connections afterward. The bug died at the same point every time, which means the fl...
[Fix] rust-server: Validate gathered listener ranks and address families sgl-project/sglang
The Rust listener table could omit or repeat an attention-DP rank, leaving control fanout incomplete, and a listener bound to 0.0.0.0 could advertise an IPv6 address it cannot accept connections on. Both are now rejected at startup after the collective completes, so listener and non-listener ranks report the same failure instead of one side silently proceeding.
Fix sharded-to-all bias and integer split positions on quantized layers ml-explore/mlx
ShardedToAllLinear ranks started with different random biases and added the bias after the sum, returning inconsistent values; bias now starts at zeros. Integer split positions on QuantizedShardedToAllLinear could also split in the wrong places and are now converted to exact fractions of the input size, with ValueError raised when a segment cannot be split. Elsewhere on the long tail: mlx fixed...
Action items
