Final revision of zzpanic vllm-radiance for 1 x R9700 GPU, qwen3.8-27b
retained text
[Summary of project status metrics](https://preview.redd.it/mgvxs008wsuh1.png?width=935&format=png&auto=webp&s=30b9b75ef5030830f8aaf059e3960bfe65f41aa0) Since StillDeadcode has released his new [radiance inference engine](https://codeberg.org/StillDeadcode/radiance), the writing is on the wall for the old vllm-radiance builds, people have moved on to newer and better. I can see a few more single digit percent speedups available but probably won't chase them. I thought I'd spend the last 5% of effort and finish off my personal build. This borrows from quite a few people who have done work in the [Launch80 Discord](https://discord.gg/W76npbfCe) and is by no means my sole work. I've personally worked through a few things: \- startup caching for vllm to prevent recompiling various things on each restart \- a working kv-offload feature for vram-sysram-disk, with memory thinning to save storage. This makes multi-agent work possible on a single card with long context. \- the occasional bugfix [zzpanic/qwen3.6-vllm-gfx1201-launchers](https://github.com/zzpanic/qwen3.6-vllm-gfx1201-launchers)
document fetched 2026-10-12 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1x3368a/final_revision_of_zzpanic_vllmradiance_for_1_x