$ the-wire · showcase
Ollama fixes OpenCode v2 launch, llama.cpp rewrites CPU concat as one kernel
By RepoJournal · Filed · About Local LLMs · Composed from the cited sources · methodology
16 people shipped this
hoyyeva
@hoyyeva
1 cited
dhiltgen
@dhiltgen
1 cited
ServeurpersoCom
@ServeurpersoCom
1 cited
mkhazraee
@mkhazraee
1 cited
zcbenz
@zcbenz
1 cited
jberg5
@jberg5
ggerganov
@ggerganov
virajwad
@virajwad
Titaniumtown
@Titaniumtown
soodoshll
@soodoshll
njhill
@njhill
chaunceyjiang
@chaunceyjiang
Etelis
@Etelis
sherlockwu
@sherlockwu
maocheng23
@maocheng23
Shunkangz
@Shunkangz
A day of plumbing and kernel work: Ollama repaired how it hands its runtime to OpenCode v2, llama.cpp collapsed four concat kernels into one, and mlx added a runtime config module modeled on jax.config.
cmd/launch: fix OpenCode v2 launch and config parsing ollama/ollama
OpenCode v2 rejects --model on the bare command and its shared service could keep the global default instead of the model picked by ollama launch, so detected v2 clients now get --standalone and the backend inherits the per-launch configuration. Version detection gets five seconds, and on failure or an unknown version neither version-specific flag is passed; v2's debug config array is now parse...
ggml-cpu : rewrite concat as one row kernel for all types ggml-org/llama.cpp
The i8/f16/f32/any CPU concat variants are replaced by a single kernel where a dense source row is one memcpy, quantized types included. Work now splits across all destination rows rather than i2 alone, which the author notes is often 1 and left one thread copying the whole tensor.
mlx: add DFlash2 support ollama/ollama
DFlash2 extends the DFlash block draft with block-local dynamic convolutions and a candidate selector that walks a chain through each row's top-k proposals, conditioning each pick on its predecessor. As in the reference implementation, the draft samples with the request's temperature only, while top_k, top_p, min_p and history penalties apply to the target during verification.
Add "config" module ml-explore/mlx
The new mx.config module overrides configuration options at runtime as an alternative to environment variables, so you can toggle something like MLX_ENABLE_TF32 for a single op: mx.config.update("MLX_ENABLE_TF32", 0), then flip it back. A mx.config.scoped_update context manager covers the scoped case.
[KV Offload] Expose the processed prefix at request finalization vllm-project/vllm
Otherwise: vllm exposes num_computed_tokens, num_in_flight_tokens, num_prompt_tokens and num_tokens as read-only ReqContext properties so KV offload can tell a processed prefix from an untouched suffix when a request terminates early, sglang lets a plain target server capture aux hidden states without loading a speculative draft, and mlx's strided_scan no longer reads or writes out of bounds wh...
