github

<details open> webgpu : add CONV_2D_DW (depthwise conv2d) kernel (#25847) * webgpu : add CONV_2D_DW (depthwise conv2d) kernel Implement GGML_OP_CONV_2D_DW for the WebGPU backend, ported from the Vulkan backend's conv2d_dw.comp. Assisted-by: Claude Opus-4.8 * Remove unnecessa

<details open> cuda: GET_ROWS quants (#25962) * cuda: add k-quant support to GET_ROWS Device-side embedding lookups require GET_ROWS to handle the k-quants used by common GGUF recipes (Q4_K_M stores token_embd as q6_K). Without it the backend rejects the op and the scheduler f

<details open> llama-arch: fix DeepSeek4 APE tensor op (#25945) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10088/llama-b10088-bin-macos-arm64.tar.gz) - macOS Apple Silic

<details open> Add support for Laguna XS.2 & M.1 (#25165) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10087/llama-b10087-bin-macos-arm64.tar.gz) - macOS Apple Silicon (ar

<details open> mtmd : use align_corners for qwen3vl vision position embedding interpolation (#25781) The Qwen3-VL learned position embedding is interpolated to the runtime patch grid with the default bilinear+antialias (align_corners=False) sampling, while the transformers refe

<details open> hexagon: check tensor type when reusing descriptors (#25968) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10084/llama-b10084-bin-macos-arm64.tar.gz) - macOS

<details open> cuda: add sqrt_softplus in topk-moe for dsv4 (#25896) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10083/llama-b10083-bin-macos-arm64.tar.gz) - macOS Apple

<details open> kleidiai : warn once when a weight type has no KleidiAI kernel (#25701) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10082/llama-b10082-bin-macos-arm64.tar.

<details open> common: resolve draft repo to its requested sidecar (#25955) With -hfd pointing to a repo shipping speculative sidecars, the draft resolved to the main model of that repo, since find_best_model() excludes sidecar files, and the explicit draft plan suppressed the

<details open> server: return 400 instead of 500 on validation error with X-Conversation-Id (#25760) * server: return 400 instead of 500 on validation error with X-Conversation-Id set_req() attaches the spipe as soon as the header is present, before the request body is parsed.

<details open> server : properly handle null llama_context (#25868) Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b1

<details open> vulkan: Refactor vk_queue to use per-instance mutexes and unique handles (#23570) * Refactor vk_queue to use per-instance mutexes and unique handles * integrates VK_KHR_internally_synchronized_queues, abstracting the queue submission into a polymorphic interface

<details open> ggml-openvino: Add GGML_BACKEND_DL_IMPL invocation for OpenVINO backend (#25795) This adds the missing `GGML_BACKEND_DL_IMPL()` macro invocation, that other backends have. Fixes #25586 for me </details> **Website:** - <https://llama.app> **macOS/iOS:** - [m

<details open> CUDA: vectorize same-type get_rows with int4 copy (#25929) k_get_rows_float did a scalar one-element-per-thread copy and recomputed the row-invariant work (index load, fast_div_modulo, src/dst row pointers) for every element. Hoist that out of the per-element loo

<details><summary>Changelog Details</summary> - beep boop 🤖: Bumping Megatron Core to v0.18.1 by @nvidia-megatron-lm-release-bot[bot] :: PR: #5439 - build: add fast-hadamard-transform and point flash_mla at nv_dev by @ko3n1g :: PR: #5447 - cp: `build: install flash_mla from

<details open> hexagon: add CLAMP op (#25934) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10075/llama-b10075-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiA

<details open> opencl: Support broadcast for Adreno MUL_MAT and honor `view_offs` for Adreno Q8_0 MUL_MAT for llama-server multi-stream (#25910) * opencl: handle broadcast for adreno gemm/gemv_noshuffle * opencl: honor view_offs for adreno noshuffle gemm/gemv * opencl: genera

<details open> model: rotate injected K/V cache for DFlash (#25823) * dflash: rotate injected K/V cache when using K/V quantization * Update src/models/dflash.cpp Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * clearer format * remove trailing whitespace ---------

<details open> llama-quant : exclude i32 ffn_gate_tid2eid routing table from quantization (#25787) DeepSeek-V4's ffn_gate_tid2eid tensor is an i32 token-id -> expert-id index table, not weights. It was never added to the name-based exclusion list alongside ffn_gate_inp.weight,

<details open> opencl: load and use `kernel_gemm_moe_q6_k_f32_ns` from bin kernel lib (#25797) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10066/llama-b10066-bin-macos-ar

<details open> opencl: transpose q4_K noshuffle scales for coalesced reads (#25805) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10064/llama-b10064-bin-macos-arm64.tar.gz)

<details open> sync : ggml </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10063/llama-b10063-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLE

<details open> tests : initialize all tensors in test_dsv4_hc to avoid NaNs in sentinel tensors (#25822) Co-authored-by: Stanisław Szymczyk <sszymczy@gmail.com> </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/gg

<details open> ggml-blas: default hadamard mul_mat to cpu routine (#25710) Signed-off-by: Aaron Teo <aaron.teo1@ibm.com> </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b1005

<details open> vulkan: Support Q2_0 (#25430) * vulkan: Support Q2_0 The backend perf tests for mat-vec-mul weren't very good at first (worse than q2_k), doubling the rows per workgroup made a big difference. * reorder * resolve merge conflict, adjust err threshold for f16->q

<details open> sycl: fix row calculation when K_QUANTS_PER_ITERATION is 1 (#25690) * sycl: fix incorrect row calculation when K_QUANTS_PER_ITERATION=1 Signed-off-by: Todd Malsbary <todd.malsbary@intel.com> * sycl: use K_QUANTS_PER_ITERATION for non-reordered Q5_K kernel This

<details open> opencl: add ABS op (#25115) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10056/llama-b10056-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI e

<details open> docs: added a note about using OpenCl with Adreno 810 (#25786) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10054/llama-b10054-bin-macos-arm64.tar.gz) - mac

<details open> hexagon: L2 cache handling rework (dirty bit tracking with lazy flushing) and more MUL_MAT updates (#25762) * hex-mm: fix artificial limit in the solver that restricted number of act-prep threads * hex-mm: fix warning * hex-prof: do not apply --top to the timel

<details open> kleidiai: Add SME vs SME2 distinction in kernel dispatch (#25478) The current integration treats SME as a single capability (CPU_FEATURE_SME) with no distinction between SME(v1) and SME2. The kernels dispatched under CPU_FEATURE_SME use SME2-specific instructions

<details open> vulkan: when using transfer queue for async copies, sync on event_wait to avoid race (#25229) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10050/llama-b1005

<details open> TP: fix Phi3, Bert, Plamo2/3, ChatGLM (#25536) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10048/llama-b10048-bin-macos-arm64.tar.gz) - macOS Apple Silicon

<details open> vendor: update BoringSSL to 0.20260713.0 (#25624) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10047/llama-b10047-bin-macos-arm64.tar.gz) - macOS Apple Sili

<details open> tests: actually exercise `test-recurrent-state-rollback` (#25758) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10046/llama-b10046-bin-macos-arm64.tar.gz) -

<details open> server : allow text-only slot save/restore with mtmd (#25076) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10045/llama-b10045-bin-macos-arm64.tar.gz) - macO

<details open> CUDA: Support CUDA Virtual Devices (#25228) * support cuda virtual devices * disable NCCL path when virtual devices are used * label virtual devices in description; add GPUx2 server CI jobs * code refactor </details> **Website:** - <https://llama.app> **mac

<details open> Enable CUDA graphs on volta+turing (#25749) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10042/llama-b10042-bin-macos-arm64.tar.gz) - macOS Apple Silicon (a

# Patch release v5.14.1 This patch solves a few issues which appeared when integrating Inkling model, most notably an issue affecting models using EncoderDecoderCache during assisted generation. It also fixes an issue that could appear during prefill with StaticCache and sdpa

<details open> ci : add official website link to release notes (#25728) Assisted-by: pi:llama.cpp/Qwen3.6-27B </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10038/llama-b10

<details open> quant : allow using manual tensor types with --pure (#25716) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10037/llama-b10037-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enab

<details open> opencl: disable FA and MoE weights repack to work around compiler issues for Adreno 850 GPU (#25745) * opencl: workaround for A850 compiler compat * opencl: fix DX compiler version parsing and cleanup --------- Co-authored-by: Li He <lih@qti.qualcomm.com> </d

<details open> cuda: extract Q1_0 elements via __byte_perm (#25628) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10035/llama-b10035-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DI

# ROCm Core SDK 7.14.0 release notes ROCm Core SDK 7.14.0 transitions ROCm to [TheRock](https://github.com/ROCm/TheRock), a build and release system that introduces a modular architecture to improve flexibility, maintainability, and alignment with community use cases: * **L

<details open> opencl: exclude some moe kernels on Adreno a7x (#25698) * opencl: exclude Adreno A7x from using Adreno MoE kernels Some compilers for A7x devices miscompile the repack kernels, corrupting the weights and causing MoE models to generate garbage output * opencl: e

<details open> cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) (#25545) * cuda : CUDA GGML_OP_LIGHTNING_INDEXER implementation (generic vector kernel + wmma kernel) * chore : remove indentation of #pragma unroll * cuda : remove unnec

<details open> tokenize : drop --stdin mutual-exclusion check (#25672) match cli and completion, which don't enforce it </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10031/llama-b10031-bin-macos-arm64.tar.gz

# Release v5.14.0 ## New Model additions ### Inkling (fresh from Thinking Machines): 975B total, 41B active * Add Inkling model #47347 by @molbap @Cyrilvallez @eustlb and @zucchini-nlp <img width="3840" height="2160" alt="image" src="https://github.com/user-attachmen

<details open> cuda : relax tensor contiguity requirements for quantized concat (#25678) * cuda : relax tensor contiguity requirements for quantized concat * tests : add test cases for non-contiguous quantized concat * ggml : relax contiguity requirements for quantized concat

<details open> DeepseekV4: reduce graph splits (#25702) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10021/llama-b10021-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](http

<details open> sycl : fix get_rows Q2_K, Q4_K, Q5_K (#25656) </details> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10020/llama-b10020-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED]