← sources

An LLM inside an LLM - Same guy who put a RISK-V CPU inside Qwen has now made Karpathy's stories260k run on an NPU compiled inside Qwen 3.5-4B's weights

source: reddit (reddit:LocalLLaMA)published 2026-10-31 19:18ingested 2026-11-01 02:40

retained text

  
Code: [https://github.com/divagr18/QwenPU](https://github.com/divagr18/QwenPU)  
Weights: [https://huggingface.co/divagr1925/QwenPU](https://huggingface.co/divagr1925/QwenPU)

From the original [tweet](https://x.com/divagr1925/status/2116544960023687432):

1/3    
  
A CPU inside an LLM is one thing, but running another LLM inside an LLM would be taking things a bit too far, wouldn't it?     
  
Anyways, here's an LLM running inside an LLM : QwenPU, an LLM (@karpathy's stories260k) running on an NPU compiled inside Qwen 3.5-4B's weights.

2/3  
  
stories260K is just a C program. To run it inside Qwen, I compiled an NPU into Qwen's weights that does matmuls, softmax and attention bit-exact. The LLM weights are in Qwen's own memory, with no external tools.  
  
A lot of work went into making it fast, more on that soon.

3/3  
  
I'm super sick right now, so the full writeup is still a work in progress, and I'm working on optimisations and size reductions too. Stay tuned, I'll finish the blog & post it soon.   
  


document fetched 2026-11-01 00:38 · canonical https://reddit.com/r/LocalLLaMA/comments/1xkiuut/an_llm_inside_an_llm_same_guy_who_put_a_riskv_cpu