Extracts: repo, release, stars delta · metadata or bounded excerpt
Retention: D1 event history with canonical source link and deduplication metadata.
Access basis: Public API; token recommended for production rate limits.
Last ingested: 2026-11-17 · run status: success with data
<details open> opencl: Q6_K GEMM/GEMV fix for ne01 of weights that are not multiples of 128. (#25464) * opencl: fix garbled output for Q6_K weights with ne01 % 128 != 0 on Adreno Observed with granite-3.1-3b-a800m-instruct, whose vocab is an odd number. Route Q6_K dense mul
<details open> vulkan: disable FA mask_opt on GCN to improve performance (#24362) * vulkan: disable FA mask_opt on GCN to improve performance * reenable mask opt over attention head size 256 </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-or
<details open> opencl: ragged-tile MoE prefill FP16 GEMM optimization (skip padded expert tiles) (#25433) * opencl: ragged-tile MoE prefill GEMM (skip padded expert tiles) The MoE prefill GEMM groups tokens into TILESIZE_N=32 per-expert tiles; at low tokens-per-expert most til
<details open> llama-batch: fix allowed decreasing pos in a seq (#25449) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9930/llama-b9930-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled)
<details open> vulkan: for small AMD GPUs, reduce submission threshold based on CU count (#25240) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9929/llama-b9929-bin-macos-arm64.tar.gz) - macOS Apple Silicon (
<details open> hexagon: new vtcm layouts and improved pipelines for MUL_MAT, MUL_MAT_ID and FLASH_ATTN_EXT (#25425) * hex-fa: refactor kernel param compute to use common layout builder * hmx: add explicit compiler barriers to make hmx funcs more robust * hex-vtcm: more generi
# PyTorch 2.13.0 Release Notes - [Highlights](#highlights) - [Backwards Incompatible Changes](#backwards-incompatible-changes) - [Deprecations](#deprecations) - [New Features](#new-features) - [Improvements](#improvements) - [Bug fixes](#bug-fixes) - [Performance](#perfo
<details open> spec : fix naming, spacing (#25410) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9912/llama-b9912-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://git
<details open> CUDA: Fuse MMVQ post-scale for NVFP4 (#24481) * CUDA: Fuse MMVQ for NVFP4 and BS 1 TODO: 1. Add tests to test-backend-ops (did verify correctness manually for one model) 2. Reorder bias/scale once PRs for NVFP4 are merged/landed * Add dense MMVQ fusion as we
<details open> server : fix draft model fit vs load inconsistency (#25056) * fix: draft model fit vs load inconsistency * refactor(server): unify draft/mtp parameter initialization, model, and context load - moves speculative init to speculative.cpp - changes server_context_im
### CuTe DSL * Bug fixing and improvements - Fixed a compilation time regression issue in 4.5.0. Compilation times now match those in the 4.4 and 4.6 branches.
<details open> server : add timings and progress to /responses API stream (#25348) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9909/llama-b9909-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI
<details open> server: enforce prompt cache RAM limit (#25070) Before this commit, --cache-ram was not a hard limit: - The cache always kept at least one entry, even if that entry exceeded the RAM/token limits. - Old entries were only evicted for the RAM/token limits after s
<details open> common : add missing <fstream> include in common.h (#25220) Signed-off-by: zhangrunda <zhangrunda1234@outlook.com> </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9907/llama-b9907-bin-macos-arm6
<details open> ggml-hip : add -fno-finite-math-only alongside -ffast-math (#25373) -ffast-math implies -ffinite-math-only under ROCm/clang 22, which disables INFINITY/NaN and triggers -Wnan-infinity-disabled (errors under -Werror in CI). Re-enable infinity handling without drop
<details open> llama: fix quantized kv-cache for dsv4 (#25202) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9905/llama-b9905-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED]
<details open> [SYCL] fix unsupported UT cases of CONT & CPY (#25231) * fix unsupported UT cases of CONT & CPY * update ops.md * rm unused head file </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9904/llama
<details open> [SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236) * support OP cross_entropy_loss, cross_entropy_loss_back * correct format issue </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/down
<details open> vulkan : check src0 type in GGML_OP_SET_ROWS to avoid failures due to unimplemented f16 support (#25351) * vulkan : check src0 type in GGML_OP_SET_ROWS to avoid failures due to unimplemented f16 support * chore : get rid of else --------- Co-authored-by: Stani
<details open> opencl: general flash attention decode performance optimizations (#25366) * opencl: vec flash-attention decode kernels for f16/q8_0/q4_0 KV * opencl: improve non FA KQ mv kernels * opencl: tweaks for multiquery FA * opencl: some tweaks for FA q1 kernels * ope
<details open> common: Set optimal default thread count for ppc ( linux as well as AIX) (#25237) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9892/llama-b9892-bin-macos-arm64.tar.gz) - macOS Apple Silicon (a
<details open> metal: add col2im_1d op (f32/f16/bf16) (#25176) * metal: add col2im_1d op (f32/f16/bf16) Gather kernel mirroring the CPU/CUDA path: each output (t_out, oc) reads its ceil(K/s0) source columns with an F32 accumulator, a single write and no atomics. One thread per
<details open> CUDA: remove -sm row, refactor cuBLAS (#24216) * CUDA: remove -sm row, refactor cuBLAS * fix CDNA + BF16 logic * fix bad return * fix src0 strides, contiguous requirements * fix GGML_CUDA_FORCE_CUBLAS * fix casts to BF16 </details> **macOS/iOS:** - [macOS
<details open> CUDA: extend K-type validation to V-types for flash attention (#24403) * CUDA: extend K-type validation to V-types for flash attention * reorder </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b
<details open> ggml-cpu: use UE4M3 LUT in ARM NVFP4 dot product (#25331) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9886/llama-b9886-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled)
<details open> ggml-cpu: Enable tiled matmul on AIX (#25199) The matmul_tiled path uses large local stack buffers for A_pack and B_pack. On AIX this can trigger a segmentation fault, so reduce the buffer footprint there to keep the tiled path usable. Performance Impact: ~
<details open> vulkan: fix 32-bit integer overflow in CEIL_DIV (#25245) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9884/llama-b9884-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [
<details open> scripts : use HF_TOKEN when downloading UI assets (#25280) Signed-off-by: Adrien Gallouët <angt@huggingface.co> </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9882/llama-b9882-bin-macos-arm64.t
<details open> ggml-hip: enable -ffast-math for HIP builds (#23862) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9881/llama-b9881-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISA
<details open> Fix stale tensor-split params for draft models (#24814) * meta: fix tensor split metadata for GQA attention * Tidied the code a bit to match existing style * Revert "Tidied the code a bit to match existing style" This reverts commit b90c6c6300091fe09e2350a3d4e
<details open> abort if we see a multi buffer (#25276) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9877/llama-b9877-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https:/
<details open> ggml : fix tensor-parallel + -ncmoe crash on MoE models (#25028) Tensor parallelism (-sm tensor) combined with -ncmoe (CPU-offloaded MoE experts) aborts during warm-up on MoE models with GGML_ASSERT(ggml_is_contiguous(tensor)) in ggml-backend-meta.cpp. The faili
<details open> cuda : concat implementation for quantized types (#25303) * cuda : concat implementation for quantized types * chore : apply am17an clever suggestion to shorten the code --------- Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> </details> **macOS/iOS
<details open> llama : add guard for K/V rotation input when buffer is unallocated (#25215) llm_graph_input_attn_kv::set_input and llm_graph_input_attn_kv_iswa::set_input call set_input_k_rot / set_input_v_rot whenever the rotation tensor pointer is non-null, but the tensor's b
<details open> ggml : fix broken CPU concat implementation for quantized types (#25247) * ggml : fix broken CPU concat implementation for quantized types * tests : concat tests for quantized types --------- Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> </details>
<details open> chat: trim messages sent to StepFun parser (fixes long reasoning loops) (#25238) * chat: trim messages sent to StepFun parser (fixes long reasoning loops) * add regression test; remove duplicate template * chat: trim StepFun content parts before rendering The
# Release v5.13.0 ## New Model additions ### KimiK 2.5, 2.6, and 2.7 <img width="1097" height="400" alt="image" src="https://github.com/user-attachments/assets/c24d2232-a9b4-413b-a2c8-58d013b6dfbd" /> This release includes the architecture for Kimi 2.5 which is used
<details open> spec: support spec-draft-p-min in DFlash (#25246) * spec: support spec-draft-p-min in DFlash * dflash: add n_min guard * dflash: guard both n_min and n_max </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/release
<details open> cuda: enable topk-moe fusion for 288 experts (#25267) * cuda: enable topk-moe fusion for 288 experts The topk-moe fusion only accepted power-of-2 expert counts (or the special-cased 576), so models with 288 experts (e.g. Step-3.7-Flash) fell back to the unfused
<details open> server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections (#25241) * server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections * server + ui: ss
<details open> Remove redundant CUDA copies after gated_delta_net. (#23940) * Remove redundant CUDA copies after gated_delta_net. Currently, GDN writes recurrent state snapshots into its output tail, then the graph immediately copies those snapshots into ssm_states_all. With M
<details open> vendor : update cpp-httplib to 0.49.0 (#25218) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9861/llama-b9861-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](
<details open> llama : add llama_model_ftype_name() (#25134) * llama : add llama_model_ftype_name() Expose the model file type (quantization) name, e.g. "Q8_0" or "Q4_K - Medium", through a new public C API. The returned pointer is valid for the lifetime of the model and nullp
<details open> opencl: allow loading precompiled binary kernels from library (#23042) * opencl: allow loading binary kernel * opencl: add libdl.h * ggml-backend-dl is in ggml, which depends backend libs, thus ggml-opencl cannot depend on ggml-backend-dl * add libdl.h to bre
<details open> common : use hf primary split as model path (#25194) Fixes #25181 </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9858/llama-b9858-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI
<details open> hexagon: flash attention rework (optimizations, accuracy improvements, etc) (#25085) * hex-mm: fold mm quant tasks into the main matmul threads * hex-mm: minor formatting fixes * hex-mm: cleanup is_quant checks in dma dispatch * hex-mm: fix dst-spad alignment
<details open> CUDA: consistent use of __restrict__ + PDL for FA (#25185) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9856/llama-b9856-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled)
<details open> ggml-cpu: add AVX2 optimization for nvfp4 dot product and use UE4M3 LUT (#23961) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9855/llama-b9855-bin-macos-arm64.tar.gz) - macOS Apple Silicon (ar
<details open> opencl: initial q1_0 support (#25160) * opencl: general q1_0 support * opencl: add Adreno GEMM/GEMV for q1_0 </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b9852/llama-b9852-bin-macos-arm64.tar
<details open> cuda : prevent integer truncation and overflow errors when using KQ mask strides in flash_attn_mask_to_KV_max kernel (#24945) Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/gg