# ggerganov/llama.cpp v0.8.0: v0.8.0 — retained record

> Retained github record from github:ggerganov/llama.cpp.

Canonical: https://highsignal.app/data/records/6a9e32c0da338372

[← sources](https://highsignal.app/data)


source: [github](https://highsignal.app/data/github) (github:ggerganov/llama.cpp)  published 2026-10-28 12:47 ingested 2026-10-29 02:33

[original source ↗](https://github.com/ggml-org/llama.cpp/releases/tag/v0.8.0)

## extract

## Overview

This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in `mtmd`, and wires realtime audio in and out of the server through the new `/v1/realtime` and `/tts` endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models from LiquidAI, Baidu and NVIDIA. The core got sparse-attention (QSA/DSA) reworked onto a fused masking op, a 50/50 SLRU cache for host-resident MoE experts, and cleaner `no_alloc` memory handling; the server got router memory-management and tool-calling improvements; the UI can now browse and download models straight from HF Hub, search conversations and compact them. Finally, `ggml` was bumped to v0.28.0, bringing `GGML_OP_TOP_K_MASK`, `-1` expert-index semantics and large backend performance work (CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP rework, KleidiAI SVE kernels).

### Highlights

- LFM3 family: LFM3-1.7B (text) and LFM3-VL-2B (vision), verified against the reference HF implementation [llama/31252](https://github.com/ggml-org/llama.cpp/pull/31252)
- LFM3-Audio-2B and LFM3-Omni-2B: ASR, TTS and voice chat from a single model, with streaming audio input and output [llama/31257](https://github.com/ggml-org/llama.cpp/pull/31257)

### API changes

- DRY sampler: new `LLAMA_DRY_MAX_SEQ_BREAKERS` (64) caps distinct sequence breakers, each costing `O(n_vocab)` at init [llama/31035](https://github.com/ggml-org/llama.cpp/pull/31035)
- `mtmd`: new experimental `mtmd_audio_stream` API turning PCM fed as it arrives into decoder-ready embedding rows [llama/30629](https://github.com/ggml-org/llama.cpp/pull/30629)
- `mtmd`: audio generation is now stateful - `mtmd_gen_state` replaces the raw `state_data` buffers of the audio gen API [llama/30902](https://github.com/ggml-org/llama.cpp/pull/30902)
- `mtmd`: `mtmd_caps` gained `gen_audio`; new TTS types `SOPRANO` and `LFM3A`; repetition penalty for sampled codes [llama/28607](https://github.com/ggml-org/llama.cpp/pull/28607) [llama/31257](https://github.com/ggml-org/llama.cpp/pull/31257)
- `mtmd_helper_get_capabilities()` to query the capabilities of an mmproj file from helper code [llama/31183](https://github.com/ggml-org/llama.cpp/pull/31183)
- Server: new `/v1/realtime` endpoint implementing the OpenAI realtime protocol for conversation and transcription [llama/31269](https://github.com/ggml-org/llama.cpp/pull/31269) [llama/30636](https://github.com/ggml-org/llama.cpp/pull/30636)
- Server: new `POST /tts` endpoint for text-to-speech models [llama/26603](https://github.com/ggml-org/llama.cpp/pull/26603)
- Server: reasoning effort levels of the chat template are exposed and selectable via the API [llama/31271](https://github.com/ggml-org/llama.cpp/pull/31271)
- Chat API: support for deferred tools and `tool_reference` content parts [llama/31256](https://github.com/ggml-org/llama.cpp/pull/31256)
- CLI: `--model` or `-hf` is now required [llama/31242](https://github.com/ggml-org/llama.cpp/pull/31242)

### New models

- LiquidAI LFM3-1.7B, LFM3-VL-2B, LFM3-Audio-2B, LFM3-Omni-2B [llama/31252](https://github.com/ggml-org/llama.cpp/pull/31252) [llama/31257](https://github.com/ggml-org/llama.cpp/pull/31257)
- LiquidAI d1-2B and d1-omni-2B decision models [llama/31275](https://github.com/ggml-org/llama.cpp/pull/31275)
- DeepSeek V4.1 Flash [llama/31086](https://github.com/ggml-org/llama.cpp/pull/31086)
- SenseNova U1 [llama/28919](https://github.com/ggml-org/llama.cpp/pull/28919)
- Beam (Reflection) [llama/30670](https://github.com/ggml-org/llama.cpp/pull/30670)
- Kolibri-1 (Aleph-Alpha) [llama/30673](https://github.com/ggml-org/llama.cpp/pull/30673)
- Flint [llama/30928](https://github.com/ggml-org/llama.cpp/pull/30928)
- Muse Spark Open (text, vision and audio) [llama/30616](https://github.com/ggml-org/llama.cpp/pull/30616)
- Granite Switch Decision, including the 256-way decision head [llama/30676](https://github.com/ggml-org/llama.cpp/pull/30676) [llama/30693](https://github.com/ggml-org/llama.cpp/pull/30693)
- Nemotron-3(.5)-Embed, with non-standard GQA and BPE tokenizer support [llama/30877](https://github.com/ggml-org/llama.cpp/pull/30877) [llama/23343](https://github.com/ggml-org/llama.cpp/pull/23343)
- H2O-Lightning-4B decision model [llama/30361](https://github.com/ggml-org/llama.cpp/pull/30361)
- MiniCPM-OCR 1.0 and MiniCPM-OCR-Jev [llama/30979](https://github.com/ggml-org/llama.cpp/pull/30979) [llama/30980](https://github.com/ggml-org/llama.cpp/pull/30980)
- Soprano TTS [llama/28607](https://github.com/ggml-org/llama.cpp/pull/28607)
- LFM2 short-conv and FFN blocks moved into a shared LFM base reused by newer LFM models [llama/30874](https://github.com/ggml-org/llama.cpp/pull/30874)

### Core changes

- MoE experts kept in host memory now use a 50/50 SLRU cache instead of a plain LRU cache [llama/30637](https://github.com/ggml-org/llama.cpp/pull/30637)
- Sparse attention: fused QSA mask op, QSA/DSA selection built without `kq_mask`, k-pool views shared between layers [llama/30640](https://github.com/ggml-org/llama.cpp/pull/30640) [llama/31009](https://github.com/ggml-org/llama.cpp/pull/31009) [llama/30965](https://github.com/ggml-org/llama.cpp/pull/30965) [llama/31243](https://github.com/ggml-org/llama.cpp/pull/31243) [llama/30613](https://github.com/ggml-org/llama.cpp/pull/30613)
- MoE: amax rescaling enabled for `laguna`, and `-sm tensor` denied on arches that use it [llama/31147](https://github.com/ggml-org/llama.cpp/pull/31147) [llama/31081](https://github.com/ggml-org/llama.cpp/pull/31081)
- Memory: `no_alloc` respected in all memory caches, moved out of `llama_hparams`, obsolete `n_pad` dropped [llama/31087](https://github.com/ggml-org/llama.cpp/pull/31087) [llama/31141](https://github.com/ggml-org/llama.cpp/pull/31141) [llama/31200](https://github.com/ggml-org/llama.cpp/pull/31200)
- Vocabulary loading optimized; `d2t` placed next to its `set_rows` use so it can stay on the GPU [llama/30679](https://github.com/ggml-org/llama.cpp/pull/30679) [llama/30912](https://github.com/ggml-org/llama.cpp/pull/30912)
- Tokenizer: new `clean_spaces` and metaspace prepend-scheme GGUF keys, hybrid DNA tokenizer fixes [llama/30940](https://github.com/ggml-org/llama.cpp/pull/30940) [llama/30825](https://github.com/ggml-org/llama.cpp/pull/30825)
- Tokenizer: Tekken pre-tokenizer for Pixtral, `\p{M}` in Tekken/GPT-4o lookaheads, Nemotron-3.5-Embed tokenization fix [llama/30960](https://github.com/ggml-org/llama.cpp/pull/30960) [llama/30845](https://github.com/ggml-org/llama.cpp/pull/30845) [llama/30974](https://github.com/ggml-org/llama.cpp/pull/30974)
- Grammar: recursion depth limited (and enforced) to prevent stack overflow; deep JSON rejected before walking it [llama/29755](https://github.com/ggml-org/llama.cpp/pull/29755) [llama/31153](https://github.com/ggml-org/llama.cpp/pull/31153) [llama/30242](https://github.com/ggml-org/llama.cpp/pull/30242)
- Qwen: merge tensors on convert, 2D output reshape, Qwen3 bidirectional attention and `dense_2` output projection [llama/31028](https://github.com/ggml-org/llama.cpp/pull/31028) [llama/30823](https://github.com/ggml-org/llama.cpp/pull/30823) [llama/30607](https://github.com/ggml-org/llama.cpp/pull/30607)
- Speculative decoding: persist the dynamic ngram cache, probabilistic DFlash2 drafts, optional DFlash FC input scale [llama/22055](https://github.com/ggml-org/llama.cpp/pull/22055) [llama/30663](https://github.com/ggml-org/llama.cpp/pull/30663) [llama/30945](https://github.com/ggml-org/llama.cpp/pull/30945) [llama/30927](https://github.com/ggml-org/llama.cpp/pull/30927)
- LoRA: support for `prism.hadamard` folded weights [llama/30910](https://github.com/ggml-org/llama.cpp/pull/30910)
- Tensor parallel: `stepfun` and `laguna` now support 4-10 GPUs [llama/24554](https://github.com/ggml-org/llama.cpp/pull/24554)
- GGUF tools reject truncated tensor data in `split`, `export-lora` and `merge` [llama/30497](https://github.com/ggml-org/llama.cpp/pull/30497) [llama/31044](https://github.com/ggml-org/llama.cpp/pull/31044)
- `llama-quant`: fix 4D tensor quantization with `ggml_nrows()` and name-based expert detection [llama/30820](https://github.com/ggml-org/llama.cpp/pull/30820)
- `convert-hf`: faster endian conversion with more datatypes, text-only checkpoints in pplx-decider [llama/30647](https://github.com/ggml-org/llama.cpp/pull/30647) [llama/30575](https://github.com/ggml-org/llama.cpp/pull/30575)
- `convert-hf`: fixes for Granite speech tensors, Granite Switch MultiSwitch block count, Mistral-format DeepseekV2 [llama/30628](https://github.com/ggml-org/llama.cpp/pull/30628) [llama/30621](https://github.com/ggml-org/llama.cpp/pull/30621) [llama/30669](https://github.com/ggml-org/llama.cpp/pull/30669)
- Common: honor `HTTP_PROXY`/`HTTPS_PROXY`, allow concurrent downloads, pin model resolution on real HF repo layouts [llama/21752](https://github.com/ggml-org/llama.cpp/pull/21752) [llama/29312](https://github.com/ggml-org/llama.cpp/pull/29312) [llama/30548](https://github.com/ggml-org/llama.cpp/pull/30548)

### Multi-modality changes

- Streaming audio input via `mtmd_audio_stream`, used to support `voxtral-realtime` [llama/30629](https://github.com/ggml-org/llama.cpp/pull/30629)
- Soprano TTS support and MiniCPM-OCR 1.0 support [llama/28607](https://github.com/ggml-org/llama.cpp/pull/28607) [llama/30979](https://github.com/ggml-org/llama.cpp/pull/30979)
- Image preprocessing aligned with transformers/torchvision: LFM2 resize, tiles and layout, Qwen-VL resize [llama/30943](https://github.com/ggml-org/llama.cpp/pull/30943) [llama/30261](https://github.com/ggml-org/llama.cpp/pull/30261) [llama/30736](https://github.com/ggml-org/llama.cpp/pull/30736)
- LFM2 tiling parameters are read from the GGUF metadata [llama/30840](https://github.com/ggml-org/llama.cpp/pull/30840)
- `lightonocr` defaults to native resolution instead of forcing a 1540 longest edge [llama/30600](https://github.com/ggml-org/llama.cpp/pull/30600)
- `build_inp_temporal_merge` moved to the `mtmd` base class; fix Gemma-4 FFN activation [llama/31065](https://github.com/ggml-org/llama.cpp/pull/31065) [llama/30882](https://github.com/ggml-org/llama.cpp/pull/30882)

### Server changes

- Realtime conversation and transcription on `/v1/realtime`, with `response.done` sent after the committed audio drains [llama/31269](https://github.com/ggml-org/llama.cpp/pull/31269) [llama/30636](https://github.com/ggml-org/llama.cpp/pull/30636) [llama/30994](https://github.com/ggml-org/llama.cpp/pull/30994)
- `POST /tts` endpoint for TTS models [llama/26603](https://github.com/ggml-org/llama.cpp/pull/26603)
- Expose the reasoning effort levels of the chat template [llama/31271](https://github.com/ggml-org/llama.cpp/pull/31271)
- Router: configurable device memory margin for dynamic unloading of models [llama/21231](https://github.com/ggml-org/llama.cpp/pull/21231)
- Router: propagate client disconnects to the child task, stop re-adding the mmproj margin on sleep/resume [llama/29707](https://github.com/ggml-org/llama.cpp/pull/29707) [llama/28233](https://github.com/ggml-org/llama.cpp/pull/28233)
- Rerank endpoint accepts typed content input and substitutes the placeholder from the original template [llama/29625](https://github.com/ggml-org/llama.cpp/pull/29625) [llama/30864](https://github.com/ggml-org/llama.cpp/pull/30864)
- Anthropic `/v1/messages`: forward llama.cpp sampling params and pass `id_slot` through the conversion [llama/30856](https://github.com/ggml-org/llama.cpp/pull/30856) [llama/28554](https://github.com/ggml-org/llama.cpp/pull/28554)
- Support `logprobs` together with speculative decoding [llama/27196](https://github.com/ggml-org/llama.cpp/pull/27196)
- Fix `usage.completion_tokens` undercounting with `n > 1` and missing `output_index` in Responses API SSE events [llama/29486](https://github.com/ggml-org/llama.cpp/pull/29486) [llama/30863](https://github.com/ggml-org/llama.cpp/pull/30863)
- Fix SWA checkpoint invalidation logic [llama/30666](https://github.com/ggml-org/llama.cpp/pull/30666)
- New `--connect` flag to allow p2p access to the server outside the LAN [llama/28277](https://github.com/ggml-org/llama.cpp/pull/28277)

### UI changes

- Discover and download models directly from HF Hub, with org-less models listed first [llama/29584](https://github.com/ggml-org/llama.cpp/pull/29584) [llama/31215](https://github.com/ggml-org/llama.cpp/pull/31215)
- New search dialog for conversations and messages [llama/31173](https://github.com/ggml-org/llama.cpp/pull/31173)
- Conversation compaction to keep long chats within the context [llama/27401](https://github.com/ggml-org/llama.cpp/pull/27401)
- Models manager moved to its own route, model details open in a drawer without leaving the page [llama/30988](https://github.com/ggml-org/llama.cpp/pull/30988) [llama/31176](https://github.com/ggml-org/llama.cpp/pull/31176)
- Reasoning menu built from the chat template's effort levels, with a clearer highlighted toggle [llama/31273](https://github.com/ggml-org/llama.cpp/pull/31273) [llama/31212](https://github.com/ggml-org/llama.cpp/pull/31212)
- Support OGG audio attachments by converting them to WAV [llama/31237](https://github.com/ggml-org/llama.cpp/pull/31237)
- New "Show tool call in progress" setting [llama/30681](https://github.com/ggml-org/llama.cpp/pull/30681)
- Save and reset settings buttons shown only when there are unsaved changes [llama/31248](https://github.com/ggml-org/llama.cpp/pull/31248)
- Sidebar resizing and general layout/UX improvements [llama/31119](https://github.com/ggml-org/llama.cpp/pull/31119) [llama/31030](https://github.com/ggml-org/llama.cpp/pull/31030)
- Fixes: draft lost after "Add System Message", system row stretching in WebKit [llama/31214](https://github.com/ggml-org/llama.cpp/pull/31214) [llama/31246](https://github.com/ggml-org/llama.cpp/pull/31246)
- Fixes: clipped attachment text preview, markdown tables and lists cut off on phones [llama/30806](https://github.com/ggml-org/llama.cpp/pull/30806) [llama/30687](https://github.com/ggml-org/llama.cpp/pull/30687)

### ggml changes

- Bumped from v0.27.0 to [ggml v0.28.0](https://github.com/ggml-org/ggml/releases/tag/v0.28.0)
- New `GGML_OP_TOP_K_MASK` fused sparse-attention mask, plus accessors for the precision configured on a compute node
- `-1` is now a valid expert index in `MUL_MAT_ID`/`ADD_ID`: the slot is skipped and the output row zeroed
- `SCALE` accepts non-contiguous row views; `SUM` accepts non-contiguous inputs on CPU, CUDA, Vulkan and WebGPU
- Backends: fused MoE top-k and gate/up+GLU with chunked graph launches on CUDA, graph record/replay on SYCL
- Backends: Q2_K/Q3_K weights and a lightning indexer on OpenCL, subgroup reductions and `src1` precision on Vulkan
- More: restructured Hexagon HTP backend, flash attention on ZDNN, Q2_0 on WebGPU, KleidiAI SVE GEMV/GEMM kernels

## Assets

**Nightly build:** [b11960](https://github.com/ggml-org/llama.cpp/releases/tag/b11960)

## More info

- [Releases and versioning of `ggml-org` projects](https://github.com/ggml-org/ggml/discussions/1579)

## Changelog since [v0.7.0](https://github.com/ggml-org/llama.cpp/releases/tag/v0.7.0)

8642541b2 llama.cpp : bump version to 0.8.0 (#31309)
d4f1b18b3 mtmd: enable video input support for LFM3 model family (#31283)
d1eb0deba server : support tool calls in /v1/realtime (#31287)
5d7793d6f sync : ggml
ec941ee85 ggml : bump version to v.28.0 (ggml/1676)
e486a4b55 ui: build the reasoning menu from the template effort levels (#31273)
f1a7dea99 tests : adjust constants to reduce irrelevant warnings (#31254)
a7af149a2 cuda : add top-k MoE fusion for 384 experts (#31288)
2864937a7 vulkan: clamp A prefetch to end_k in coopmat1 int8 matmul (#30283)
a71c00f0b chat : stop qwen3-coder reasoning and content before template separators (#30557)
f4d8af608 ui : show save and reset settings buttons only when relevant (#31248)
02d502f31 ui: Support OGG audio attachments by converting them to WAV (#31237)
b1756b23f ui: Fix draft for new message disappearing after using "Add System Message" action (#31214)
6886628cb ui : Search dialog (#31173)
8ca4060b7 server : add realtime conversation and transcription on /v1/realtime (#31269)
d05706b4f model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B (#31275)
1b4554eff server: expose the reasoning effort levels of the chat template (#31271)
a91917994 tests: fix CI riscv timeout in test-llama-context (#31265)
338fce651 HIP/CUDA: remove pragma unroll from loops in conv2d where itteration count cant be infered at compile time (#31255)
19ff817fb model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B (#31257)
eb7818c0b common : support deferred tools and tool_reference content parts (#31256)
6c4459104 snapdragon: switch QDC artifact upload and log download to v2 API (#31240)
19cce6919 model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B (#31252)
e86f5fd1e cmake: WASI doesn't support spawning (#30751)
27b7a8a00 llama : build the k-pool sparse selection independently of the kq_mask shape (#31243)
7ecb57f7a rpc : catch std::length_error in recv_msg (#31204)
8542e9435 tests : speed up tensor data generation and model saving (#31145)
922f53ed3 vulkan: fix CI llvmpipe by allowing dim 3 broadcast in MUL_MAT (#31239)
0c8c59ef3 mtmd-cli: Reduced scope of batch pointer (#31155) (#31165)
3ab37aedf ui: fix system message row stretching to full prompt height in WebKit (#31246)
85cc432e2 server: add router device memory margin parameter for dynamic unloading (#21231)
74a1fd3d0 common : --model or -hf is required (#31242)
7eed6b768 model: Add MiniCPM-OCR-Jev (#30980)
276378515 metal : add GGML_OP_TOP_K_MASK (#31230)
d78f40d9d glm5-next : QSA without kq_mask (+ DSA) (#31009)
fae9ac561 ci : use ccache depend mode for ROCm release builds (#31231)
a8276b70c ci : force python 3.12 on ubuntu 26.04 (#31222)
de5122d87 ggml : relax scale to non-contiguous views (#31146)
6666a7330 tests : fix multi-seq split replay and partial-state cases for trained models (#31216)
7a2e5aba9 llama : remove obsolete n_pad from memory instances (#31200)
0d5ca6009 ui: list org-less models first in the model selector (#31215)
d930fe3c2 opencl: refine mul mat id -1 index (#31170)
a2b3e121c opencl: initial support for q3_k (#31172)
823df7d0f ui: Highlight reasoning toggle on hover and align it with model selector (#31212)
1a97e387d ui: add conversation compaction (#27401)
bc262f33c ui: Open model details in a drawer without leaving the page (#31176)
f5892ca5a ui: use shared confirmation dialog for conversation deletion (#31187)
7aca0d2ea hexagon: add GGML_HEXAGON_VTCM_SIZE (#31221)
961c376a2 llama : enable moe amax rescaling for laguna (#31147)
4244ad7ac ci : add windows behavior check (#31191)
63b4b303b opencl: write KV-cache views in set_tensor so a state restore is not dropped (#30471)
caa8191df hexagon: MUL_MAT_ID row skipping optimizations and improvements (#31169)
a4699400e llama : move no_alloc out of llama_hparams (#31141)
4b851fbee models : replace ggml_tensor** out-params with output structs (#31142)
4d71b42fd ggml-cpu : rewrite concat as one row kernel for all types (#31179)
70c5f2632 vulkan: rms norm optimization using subgroup reductions (#29882)
5cd62ffee sycl: template the plain MMVQ launch triples (#31013)
38645c2ad sycl: template the per-quant concat impls (#31008)
02269b4c0 sycl: add set_tensor_2d for faster tensor-split loading (#30358)
5320c87ff sycl : bound reorder temp memory to 32 MiB (#31062)
74b104c9b common: pin model resolution on real HF repo layouts (#30548)
c6fb3b5be mtmd: add mtmd_helper_get_capabilities (#31183)
8b28c2394 ggml-cpu : fix unused variables in arm arch (#31180)
fd8f8451a tes

## retained text (truncated)



```
## Overview

This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in `mtmd`, and wires realtime audio in and out of the server through the new `/v1/realtime` and `/tts` endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models from LiquidAI, Baidu and NVIDIA. The core got sparse-attention (QSA/DSA) reworked onto a fused masking op, a 50/50 SLRU cache for host-resident MoE experts, and cleaner `no_alloc` memory handling; the server got router memory-management and tool-calling improvements; the UI can now browse and download models straight from HF Hub, search conversations and compact them. Finally, `ggml` was bumped to v0.28.0, bringing `GGML_OP_TOP_K_MASK`, `-1` expert-index semantics and large backend performance work (CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP rework, KleidiAI SVE kernels).

### Highlights

- LFM3 family: LFM3-1.7B (text) and LFM3-VL-2B (vision), verified against the reference HF implementation [llama/31252](https://github.com/ggml-org/llama.cpp/pull/31252)
- LFM3-Audio-2B and LFM3-Omni-2B: ASR, TTS and voice chat from a single model, with streaming audio input and output [llama/31257](https://github.com/ggml-org/llama.cpp/pull/31257)

### API changes

- DRY sampler: new `LLAMA_DRY_MAX_SEQ_BREAKERS` (64) caps distinct sequence breakers, each costing `O(n_vocab)` at init [llama/31035](https://github.com/ggml-org/llama.cpp/pull/31035)
- `mtmd`: new experimental `mtmd_audio_stream` API turning PCM fed as it arrives into decoder-ready embedding rows [llama/30629](https://github.com/ggml-org/llama.cpp/pull/30629)
- `mtmd`: audio generation is now stateful - `mtmd_gen_state` replaces the raw `state_data` buffers of the audio gen API [llama/30902](https://github.com/ggml-org/llama.cpp/pull/30902)
- `mtmd`: `mtmd_caps` gained `gen_audio`; new TTS types `SOPRANO` and `LFM3A`; repetition penalty for sampled codes [llama/28607](https://github.com/ggml-org/llama.cpp/pull/28607) [llama/31257](https://github.com/ggml-org/llama.cpp/pull/31257)
- `mtmd_helper_get_capabilities()` to query the capabilities of an mmproj file from helper code [llama/31183](https://github.com/ggml-org/llama.cpp/pull/31183)
- Server: new `/v1/realtime` endpoint implementing the OpenAI realtime protocol for conversation and transcription [llama/31269](https://github.com/ggml-org/llama.cpp/pull/31269) [llama/30636](https://github.com/ggml-org/llama.cpp/pull/30636)
- Server: new `POST /tts` endpoint for text-to-speech models [llama/26603](https://github.com/ggml-org/llama.cpp/pull/26603)
- Server: reasoning effort levels of the chat template are exposed and selectable via the API [llama/31271](https://github.com/ggml-org/llama.cpp/pull/31271)
- Chat API: support for deferred tools and `tool_reference` content parts [llama/31256](https://github.com/ggml-org/llama.cpp/pull/31256)
- CLI: `--model` or `-hf` is now required [llama/31242](https://github.com/ggml-org/llama.cpp/pull/31242)

### New models

- LiquidAI LFM3-1.7B, LFM3-VL-2B, LFM3-Audio-2B, LFM3-Omni-2B [llama/31252](https://github.com/ggml-org/llama.cpp/pull/31252) [llama/31257](https://github.com/ggml-org/llama.cpp/pull/31257)
- LiquidAI d1-2B and d1-omni-2B decision models [llama/31275](https://github.com/ggml-org/llama.cpp/pull/31275)
- DeepSeek V4.1 Flash [llama/31086](https://github.com/ggml-org/llama.cpp/pull/31086)
- SenseNova U1 [llama/28919](https://github.com/ggml-org/llama.cpp/pull/28919)
- Beam (Reflection) [llama/30670](https://github.com/ggml-org/llama.cpp/pull/30670)
- Kolibri-1 (Aleph-Alpha) [llama/30673](https://github.com/ggml-org/llama.cpp/pull/30673)
- Flint [llama/30928](https://github.com/ggml-org/llama.cpp/pull/30928)
- Muse Spark Open (text, vision and audio) [llama/30616](https://github.com/ggml-org/llama.cpp/pull/30616)
- Granite Switch Decision, including the 256-way decision head [llama/30676](https://github.com/ggml-org/llama.cpp/pull/30676) [llama/30693](https://github.com/ggml-org/llama.cpp/pull/30693)
- Nemotron-3(.5)-Embed, with non-standard GQA and BPE tokenizer support [llama/30877](https://github.com/ggml-org/llama.cpp/pull/30877) [llama/23343](https://github.com/ggml-org/llama.cpp/pull/23343)
- H2O-Lightning-4B decision model [llama/30361](https://github.com/ggml-org/llama.cpp/pull/30361)
- MiniCPM-OCR 1.0 and MiniCPM-OCR-Jev [llama/30979](https://github.com/ggml-org/llama.cpp/pull/30979) [llama/30980](https://github.com/ggml-org/llama.cpp/pull/30980)
- Soprano TTS [llama/28607](https://github.com/ggml-org/llama.cpp/pull/28607)
- LFM2 short-conv and FFN blocks moved into a shared LFM base reused by newer LFM models [llama/30874](https://github.com/ggml-org/llama.cpp/pull/30874)

### Core changes

- MoE experts kept in host memory now use a 50/50 SLRU cache instead of a plain LRU cache [llama/30637](https://github.com/ggml-org/llama.cpp/pull/30637)
- Sparse attention: fused QSA mask op, QSA/DSA selection built without `kq_mask`, k-pool views shared between layers [llama/30640](https://github.com/ggml-org/llama.cpp/pull/30640) [llama/31009](https://github.com/ggml-org/llama.cpp/pull/31009) [llama/30965](https://github.com/ggml-org/llama.cpp/pull/30965) [llama/31243](https://github.com/ggml-org/llama.cpp/pull/31243) [llama/30613](https://github.com/ggml-org/llama.cpp/pull/30613)
- MoE: amax rescaling enabled for `laguna`, and `-sm tensor` denied on arches that use it [llama/31147](https://github.com/ggml-org/llama.cpp/pull/31147) [llama/31081](https://github.com/ggml-org/llama.cpp/pull/31081)
- Memory: `no_alloc` respected in all memory caches, moved out of `llama_hparams`, obsolete `n_pad` dropped [llama/31087](https://github.com/ggml-org/llama.cpp/pull/31087) [llama/31141](https://github.com/ggml-org/llama.cpp/pull/31141) [llama/31200](https://github.com/ggml-org/llama.cpp/pull/31200)
- Vocabulary loading optimized; `d2t` placed next to its `set_rows` use so it can stay on the GPU [llama/30679](https://github.com/ggml-org/llama.cpp/pull/30679) [llama/30912](https://github.com/ggml-org/llama.cpp/pull/30912)
- Tokenizer: new `clean_spaces` and metaspace prepend-scheme GGUF keys, hybrid DNA tokenizer fixes [llama/30940](https://github.com/ggml-org/llama.cpp/pull/30940) [llama/30825](https://github.com/ggml-org/llama.cpp/pull/30825)
- Tokenizer: Tekken pre-tokenizer for Pixtral, `\p{M}` in Tekken/GPT-4o lookaheads, Nemotron-3.5-Embed tokenization fix [llama/30960](https://github.com/ggml-org/llama.cpp/pull/30960) [llama/30845](https://github.com/ggml-org/llama.cpp/pull/30845) [llama/30974](https://github.com/ggml-org/llama.cpp/pull/30974)
- Grammar: recursion depth limited (and enforced) to prevent stack overflow; deep JSON rejected before walking it [llama/29755](https://github.com/ggml-org/llama.cpp/pull/29755) [llama/31153](https://github.com/ggml-org/llama.cpp/pull/31153) [llama/30242](https://github.com/ggml-org/llama.cpp/pull/30242)
- Qwen: merge tensors on convert, 2D output reshape, Qwen3 bidirectional attention and `dense_2` output projection [llama/31028](https://github.com/ggml-org/llama.cpp/pull/31028) [llama/30823](https://github.com/ggml-org/llama.cpp/pull/30823) [llama/30607](https://github.com/ggml-org/llama.cpp/pull/30607)
- Speculative decoding: persist the dynamic ngram cache, probabilistic DFlash2 drafts, optional DFlash FC input scale [llama/22055](https://github.com/ggml-org/llama.cpp/pull/22055) [llama/30663](https://github.com/ggml-org/llama.cpp/pull/30663) [llama/30945](https://github.com/ggml-org/llama.cpp/pull/30945) [llama/30927](https://github.com/ggml-org/llama.cpp/pull/30927)
- LoRA: support for `prism.hadamard` folded weights [llama/30910](https://github.com/ggml-org/llama.cpp/pull/30910)
- Tensor parallel: `stepfun` and `laguna` now support 4-10 GPUs [llama/24554](https://github.com/ggml-org/llama.cpp/pull/24554)
- GGUF tools reject truncated tensor data in `split`, `export-lora` and `merge` [llama/30497](https://github.com/ggml-org/llama.cpp/pull/30497) [llama/31044](https://github.com/ggml-org/llama.cpp/pull/31044)
- `llama-quant`: fix 4D tensor quantization with `ggml_nrows()` and name-based expert detection [llama/30820](https://github.com/ggml-org/llama.cpp/pull/30820)
- `convert-hf`: faster endian conversion with more datatypes, text-only checkpoints in pplx-decider [llama/30647](https://github.com/ggml-org/llama.cpp/pull/30647) [llama/30575](https://github.com/ggml-org/llama.cpp/pull/30575)
- `convert-hf`: fixes for Granite speech tensors, Granite Switch MultiSwitch block count, Mistral-format DeepseekV2 [llama/30628](https://github.com/ggml-org/llama.cpp/pull/30628) [llama/30621](https://github.com/ggml-org/llama.cpp/pull/30621) [llama/30669](https://github.com/ggml-org/llama.cpp/pull/30669)
- Common: honor `HTTP_PROXY`/`HTTPS_PROXY`, allow concurrent downloads, pin model resolution on real HF repo layouts [llama/21752](https://github.com/ggml-org/llama.cpp/pull/21752) [llama/29312](https://github.com/ggml-org/llama.cpp/pull/29312) [llama/30548](https://github.com/ggml-org/llama.cpp/pull/30548)

### Multi-modality changes

- Streaming audio input via `mtmd_audio_stream`, used to support `voxtral-realtime` [llama/30629](https://github.com/ggml-org/llama.cpp/pull/30629)
- Soprano TTS support and MiniCPM-OCR 1.0 support [llama/28607](https://github.com/ggml-org/llama.cpp/pull/28607) [llama/30979](https://github.com/ggml-org/llama.cpp/pull/30979)
- Image preprocessing aligned with transformers/torchvision: LFM2 resize, tiles and layout, Qwen-VL resize [llama/30943](https://github.com/ggml-org/llama.cpp/pull/30943) [llama/30261](https://github.com/ggml-org/llama.cpp/pull/30261) [llama/30736](https://github.com/ggml-org/llama.cpp/pull/30736)
- LFM2 tiling parameters are read from the GGUF metadata [llama/30840](https://github.com/ggml-org/llama.cpp/pull/30840)
- `lightonocr` defaults to native resolution instead of forcing a 1540 longest edge [llama/30600](https://github.com/ggml-org/llama.cpp/pull/30600)
- `build_inp_temporal_merge` moved to the `mtmd` base class; fix Gemma-4 FFN activation [llama/31065](https://github.com/ggml-org/llama.cpp/pull/31065) [llama/30882](https://github.com/ggml-org/llama.cpp/pull/30882)

### Server changes

- Realtime conversation and transcription on `/v1/realtime`, with `response.done` sent after the committed audio drains [llama/31269](https://github.com/ggml-org/llama.cpp/pull/31269) [llama/30636](https://github.com/ggml-org/llama.cpp/pull/30636) [llama/30994](https://github.com/ggml-org/llama.cpp/pull/30994)
- `POST /tts` endpoint for TTS models [llama/26603](https://github.com/ggml-org/llama.cpp/pull/26603)
- Expose the reasoning effort levels of the chat template [llama/31271](https://github.com/ggml-org/llama.cpp/pull/31271)
- Router: configurable device memory margin for dynamic unloading of models [llama/21231](https://github.com/ggml-org/llama.cpp/pull/21231)
- Router: propagate client disconnects to the child task, stop re-adding the mmproj margin on sleep/resume [llama/29707](https://github.com/ggml-org/llama.cpp/pull/29707) [llama/28233](https://github.com/ggml-org/llama.cpp/pull/28233)
- Rerank endpoint accepts typed content input and substitutes the placeholder from the original template [llama/29625](https://github.com/ggml-org/llama.cpp/pull/29625) [llama/30864](https://github.com/ggml-org/llama.cpp/pull/30864)
- Anthropic `/v1/messages`: forward llama.cpp sampling params and pass `id_slot` through the conversion [llama/30856](https://github.com/ggml-org/llama.cpp/pull/30856) [llama/28554](https://github.com/ggml-org/llama.cpp/pull/28554)
- Support `logprobs` together with speculative decoding [llama/27196](https://github.com/ggml-org/llama.cpp/pull/27196)
- Fix `usage.completion_tokens` undercounting with `n > 1` and mis
```



document fetched 2026-10-29 02:33
