← sources

Qwen3.5-9B running on a PS4’s GPU, offline, using its own MTP head for speculative decoding

source: reddit (reddit:LocalLLaMA)published 2026-11-13 16:00ingested 2026-11-14 02:42

retained text

I made this, so a quick note: it’s my open-source project. Qwen3.5-9B runs on a stock PS4 at 31–39 tok/s. The MTP head that ships with Qwen3.5 made the biggest difference: it drafts up to two tokens and the full model checks them in one pass, which took it from 22 to 31–39 tok/s on this old GPU. Weights are re-packed into custom 4- and 3-bit formats to fit about 4.5 GiB.

Your old PS4 can now run a 9B AI, offline, on its own GPU. It talks, listens, and writes code. No cloud, no subscription, no Linux. Free and open source.

Code: [https://github.com/leonardo-alegria/garlic-llm](https://github.com/leonardo-alegria/garlic-llm)   
Direct download link: [https://huggingface.co/garlic-llm/garlic-llm-ps4](https://huggingface.co/garlic-llm/garlic-llm-ps4)

document fetched 2026-11-14 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xvbn2m/qwen359b_running_on_a_ps4s_gpu_offline_using_its