# mini-AGI: Continual-learning dynamically looped transformer with evolutionary grown (on a laptop) — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/d6c2a4a20d73f3d3

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-09-22 17:46 ingested 2026-09-23 02:43

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1wnggz5/miniagi_continuallearning_dynamically_looped/)

## retained text



```
Saw this today and found it very intriguing. Lots of interesting design choices here, and it's cool to see someone doing something different. Here's a few highlights:

* Looped transformer: dynamic recurrent depth on a per-token basis, up to 24 cycles
* Self-supervised learning: trains itself on new material constantly
* Weights stored on SSD and paged in on-demand
* Mixture of Experts: 8 active, 32 routed held in VRAM, smart caching of 96 more
* Dynamic size: builds new experts and increases parameter counds as-needed
* Evolutionary growth: trials newly generated experts, unused ones are pruned back
* No tokenizer: it reads raw bytes directly
* Catastrophic forgetting prevented by slow trunk/fast experts learning rate split

Weights will be released in "a couple weeks" once training progress reaches \~GPT-2 levels. The trend line has held 15-fold so far, but it may bend at some point, so that is definitely a rough estimate of the trajectory.

What do you guys think?
```



document fetched 2026-09-23 00:44 · canonical https://reddit.com/r/LocalLLaMA/comments/1wnggz5/miniagi_continuallearning_dynamically_looped
