← sources

What exciting architectural moves are we looking to see next in AI models?

source: reddit (reddit:LocalLLaMA)published 2026-10-31 20:31ingested 2026-11-01 02:40

retained text

We know from all the major chinese labs the current cutting edge architectures such as: conditional engram memory, hyper connections, attention residuals, sparse attention, etc...

In the near horizon, we can likely expect looped transformers, Gated Delta Net 2 variants, maybe revival of diffusion transformers somehow but without the intelligence loss, and likely many others that I myself don't know of yet.

What are they cooking?

document fetched 2026-11-01 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xkklyp/what_exciting_architectural_moves_are_we_looking