What exciting architectural moves are we looking to see next in AI models?
retained text
We know from all the major chinese labs the current cutting edge architectures such as: conditional engram memory, hyper connections, attention residuals, sparse attention, etc... In the near horizon, we can likely expect looped transformers, Gated Delta Net 2 variants, maybe revival of diffusion transformers somehow but without the intelligence loss, and likely many others that I myself don't know of yet. What are they cooking?
document fetched 2026-11-01 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xkklyp/what_exciting_architectural_moves_are_we_looking