← sources

Is the memory a max-context KV reservation holds actually costing anything besides concurrency?

source: reddit (reddit:LocalLLaMA)published 2026-09-17 17:49ingested 2026-09-18 02:42

retained text

chose c. not confident

my first instinct was that it slows decode down, like you're dragging the dead region around every step. i don't think that's right though. nothing reads it, it's just unavailable, so the cost is that concurrency gets capped by reservations instead of by live tokens and you hit the wall way below what the tokens would fit in. which i assume is the whole argument for paged kv. what i can't work out is whether it matters if your workload is uniform, if requests really do run near max context, is paging buying you anything except bookkeeping?

document fetched 2026-09-18 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1wj10mk/is_the_memory_a_maxcontext_kv_reservation_holds