← sources

I spent a week testing whether LLMs can really "program weights." Reading weights: yes. Self-improvement: no. 20 steps of fine-tuning erase the edit

source: reddit (reddit:singularity)published 2026-11-06 17:28ingested 2026-11-07 02:39

retained text

The Sparks post split this sub into "recursive self-improvement is here" and "nothing burger." I tested both claims on 4 real bugs in open models.

* **Models can read weights.** Blind Claude Opus sessions, given only raw weights and tools, fixed Whisper's hallucinations and a language-mixing bug 3 out of 3 times each.
* **Edits work as patches.** One rewired neuron cut fake biographies from 98% to 16% and left 98% of other answers word-for-word identical.
* **But they don't stick.** 20 steps of ordinary fine-tuning erased the fix completely. A model editing itself mid-training is fighting its own optimizer.

So it's a sticky note on a frozen model for now. If someone finds a way to make edits survive training, that's when the self-improvement conversation gets real.

Full write-up: [https://kevinpl07.github.io/weight-patching/](https://kevinpl07.github.io/weight-patching/)

document fetched 2026-11-07 00:28 · canonical https://reddit.com/r/singularity/comments/1xpip7q/i_spent_a_week_testing_whether_llms_can_really