github

## Overview

## Overview New version has been released. **Nightly build:** [b10566](https://github.com/ggml-org/llama.cpp/releases/tag/b10566) **Web UI:** the `nightly-tag.txt` asset contains the tag of the corresponding nightly release **More info:** [dist : releases and versioning of gg

<details open> TP: enable tensor split for LFM2/LFM2MOE (#26993) Assisted-by: deepseek-v4-flash </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/42096995> **macOS/iOS:** - [macOS Apple Silicon (arm64)](htt

<details open> metal : clamp K extent in tensor API mat-mat kernel for K not a multiple of 32 (#27450) The Tensor API mat-mat path of kernel_mul_mm (GGML_METAL_HAS_TENSOR) fed a static K=32 tile to the matmul2d op on every iteration. On the last, partial K tile (ne00 % 32 != 0)

<details open> mtmd: add --mmproj-device argument (#23255) * feat: add --mmproj-device arg & backwards compatible MTMD_BACKEND_DEVICE env var * feat: load mmproj device backend immediately, add -mmdev shortflag * fix: its a pointer now get the name * clean up * gen docs *

<details open> vulkan: FA MMQ should use fp32 for Q quantization calculations (#27413) Codex found that qd could be a denorm and 1/qd would overflow. </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/4203484

<details open> metal : dequant kv cache only for large batches (#27438) </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/42031082> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/

<details open> CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (#26678) * CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows Currently, we ship the non-redist debug version of microsoft's libomp. This PR changes this to official LLVM's release, also packag

<details open> server: (router) lazy-load startup_models after main setup (#27424) * server: (router) lazy-load startup_models after main setup * only allow is_first_load to populate it * nits * nits 2 </details> **Website:** - <https://llama.app> **Attestations:** - <htt

<details open> CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover (#26079) * CUDA: runtime GGML_CUDA_MMVQ_MAX to tune the mvq->MMQ decode crossover Add a runtime override of the mul_mat_vec_q -> MMQ batch crossover (default MMVQ_MAX_BATCH_SI

<details open> common : gracefully fallback on unsupported regex patterns in JSON schema (#26939) </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/42018860> **macOS/iOS:** - [macOS Apple Silicon (arm64)](ht

<details open> metal : dequantize quantized KV to F16 before flash attention (#27390) * metal: dequantize q8_0 KV to f16 before flash attention Add a preprocessing pass for GGML_OP_FLASH_ATTN_EXT on the Metal backend: when the KV cache is quantized (Q8_0 for now), dequantize K

<details open> Revert "tensor-split meta backend fixes (#26502)" (#27433) This reverts commit d59d455fd8ea09e5a2e87ce2a9d668267ffb5ccd. </details> **Website:** - <https://llama.app> **Attestations:** - <https://github.com/ggml-org/llama.cpp/attestations/42012577> **macOS/iO

#### MoE * **Quantile Balancing router:** adds auxiliary-loss-free load balancing using per-expert routing biases derived from quantile estimates ([\#5349](https://github.com/NVIDIA/Megatron-LM/pull/5349)). * **Fused shared-expert MLP:** enables shared experts through group

# Patch release v5.15.1 This patch most notably solves a few issues with DFlash and MTP candidate generators, as well as an issue where images could sometimes not be processed on accelerator if using Lanczos filter. It contains the following commits: - Fix DFlash candida

<details open> ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test (#27292) * update to ov-2026.3, update device drivers * ci: skip nemotron-h rollback test on OpenVINO The OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph

<details open> mtmd: fix LFM2 image tiling threshold (#27057) * mtmd: fix LFM2 image tiling threshold * refactor testing * fix * fix on windows --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co> </details> **Website:** - <https://llama.app> **macOS/iOS:** -

<details open> sync : ggml </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10485/llama-b10485-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLE

<details open> build : fix xcframework + cmake clean-up (#27304) * xcframework : fix build * mtmd : remove unused include path * vendor : use vendor::hash alias target in cmake CMake reserves "::" in target names for imported/alias targets, so the real target keeps the name

<details open> cuda : skip UMA override for HIP builds (#27083) AMD APUs report accurate memory via hipMemGetInfo. Using MemAvailable over-promises on small-carveout systems. fixes #18159 </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (a

<details open> ci : push release tag explicitly in release.yml (#27261) Add a "Create and push git tag" step to the release job, right before the "Create release" step. The tag is created with git tag and pushed with the deploy key already configured by the Clone step, instead

<details open> sycl: fix thread/block count in quantized cpy kernel launches (#27160) Adjusts the thread/block count to be proportional to the size of the quant, reducing under/over subscription. Largest perf improvement is the q4_0 -> f32 path, with, on a Arc 70, throughput g

<details open> [SYCL] support OP OPT_STEP_ADAMW, OPT_STEP_SGD (#25268) * fix conflict * fix conflict of ops.md * fix conflict of ops.md * update the ops.md --------- Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com> </details> **Website:** - <https://llama.app>

<details open> model : remove some ggml_concat (#27176) Co-authored-by: Xuan Son Nguyen <son@huggingface.co> </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10453/llama-b104

<details open> chat: refactor handling supports_string_content / supports_typed_content (#27130) * better supports_string_content cap detect * test: add "skip" * messages_inp_normalizer </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (ar

<details open> llama : check LoRA tensor data is within file bounds (#27056) * llama : check LoRA tensor data is within file bounds * Update src/llama-adapter.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret

<details open> model: add Kimi-K3 text model (#26185) * model: add Kimi-K3 text model Hybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five things that architecture does not have: 1. cross-layer residual attention (attn_res_block_size) 2. latent MoE

<details open> server: re-design yield_to_queue thread model (#27133) * run common_speculative_process in worker * swap worker <--> main thread design </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/ll

<details open> vendor: update BoringSSL to 0.20260813.0 (#27099) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10446/llama-b10446-bin-macos-arm64.tar.gz) - macOS Apple Sili

<details open> common: support --models-dir loading MTP assistant models (#24431) * common: support --models-dir loading MTP assistant models * common: preset: check for MTP models with strict prefix * common: preset: Take advantage of PR #27005 * handle other draft types *

<details open> fix: check gguf array type before reading (#27075) * fix: check gguf array type before reading * update skill </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/

<details open> vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat1 on Intel Xe (#25380) * vulkan: add SHMEM_STRIDE_PAD/APPLY_SLM_A_RESHAPE for coopmat mul_mm on Intel Xe * vulkan: fix shmem estimate for Intel SHMEM_STRIDE_PAD=0 in matmul_shmem_support * cacheline al

<details open> common: migrate the deprecated --mmap/--no-mmap to --load-mode (#26934) Replace the deprecated --mmap, --no-mmap, --mlock, and --direct-io flags with the unified --load-mode argument across scripts, examples, and documentation. Internal warning message and env va

<details open> model : add support for MiniMaxText01ForCausalLM and MiniMaxM1ForCausalLM (#27018) * llama : support for MiniMax-Text-01 model * chore : renames to match the other MiniMax models * model : add logits mask as MiniMax-Text-01 embeddings tensor has zero-valued emb

<details open> mtmd, common: various fixes (#27071) * apply fixes * cont * revert gguf fix </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10436/llama-b10436-bin-macos-arm

<details open> jinja : fix quadratic cost in gather_string_parts (#27034) * jinja : fix quadratic cost in gather_string_parts * fix some comments * remove test </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/g

<details open> chat : pass reasoning_effort to template * chat: add reasoning_effort to common_chat_templates_inputs Store OpenAI Chat Completions reasoning_effort and make it available to jinja templates (with model specific translations where required). Assisted-by: llama.c

<details open> sync : ggml </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10433/llama-b10433-bin-macos-arm64.tar.gz) - macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLE

<details open> ggml : recurrent state rollback for ggml_ssm_scan (#26623) * Initial changes for Recurrent state rollback for nemotron for cpu and cuda * Removing CPU RS rollback. Will enable it in subsequent PRs * addition of test case * Removing assert and calling runtime A

<details open> llama : allow virtual igpu devices (#26953) * llama : allow virtual igpu devices * cont : better comment </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b1043

<details open> server: allow accessing /metrics and /slots during llama_decode() (#27041) * server_queue::worker * call llama_decode inside yield_to_queue * also handle process_mtmd_chunk * clean up * nits * rm test </details> **Website:** - <https://llama.app> **macOS/

<details open> tests : replace personal home directory paths with generic placeholders (#27043) Scrub developer-specific /home/<user>/ paths from example docs and test fixtures so they don't leak into the tree. - examples/test-cmake/README.md: /home/danbev/... -> /path/to/llam

<details open> sycl: fuse mul_mat(gate) + mul_mat(up) + GLU for q4_K dense FFN (#26779) Measured on Arc Pro B70 (Battlemage, Level Zero), llama-bench -r 20, two interleaved rounds, tg128: qwen2.5-3B-Instruct Q4_K_M 154.18 -> 158.53 t/s +2.8% gemma-2-2b-it Q4_K_M

<details open> sycl: fuse the gated-delta-net state writeback cpy (#26643) Port of https://github.com/ggml-org/llama.cpp/pull/23940. Arc Pro B70, Qwen 3.6 27B Q4_K - Medium (48 of its 64 blocks run gated_delta_net), -ngl 99 -fa 1 -ctk f16 -ctv f16 -b 2048 -ub 2048, interleaved

<details open> dflash : clarify output logging of target_layer_ids (#27013) This commit tries to make the logging of target_layer_ids a bit clearer and easier to read. Currently the output generated looks like this: ```console 0.00.468.624 D load_arch_hparams: DFlash extract_l

<details open> common: apply CPU parameters across tools (#27026) </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10423/llama-b10423-bin-macos-arm64.tar.gz) - macOS Apple Sil

<details open> OpenVINO: Qwen3.5, memory optimization, and test-recurrent-state-rollback (#26952) * OpenVINO backend: 1) enable gpt-oss moe on OV bk; 2) enable mxfp4 support * OpenVINO backend: disable TOPK_MOE op test * OpenVINO Backend: Add op FILL support * OpenVINO backe

<details open> [SYCL] Support host pinned mem to improve SYCL Host-to-Device Memory Access (#26789) * support host pinned mem, ggml_backend_sycl_host_buffer_type_get_max_size, * fix the thread-safe issue </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS

<details open> chat : fix LFM2 tool call arg name prefix ambiguity (#26960) Assisted-by: Claude Opus 5 </details> **Website:** - <https://llama.app> **macOS/iOS:** - [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10417/llama-b10417-bin

<details open> server : serve index.html with no-cache (#27006) index.html was served with `max-age=31536000, immutable` like the hashed assets, but its name is stable while its contents change every build, so a cached copy pins the UI to an old build. It now revalidates via it