Age of the bespoke engines?
retained text
When I started, and for some time, llama.cpp was the answer. Now, it seems like llama.cpp is the new ollama. It's slow and out of date and just behind. There was Ninfer for those for whom that applied. Then strata came out... And that was an unimaginable upgrade. How is llama so far behind? Now every day someone has their own forked engine that claims even better numbers. I already maxed out my token budget on Claude for the week and have my GPUs all running benchmarks so I'm waiting a few days, but I'll be trying Avalon to see if that lives up to the hype for my machine. Then there's basalt. There's beatleguese? It's kind of a mess. Innovation is great but it's better when the effort all goes in one direction and isn't spread into a thousand fractures of one-off projects. And yes, my critique of llama.cpp is a bit overstated. I still use it for e.g. 27b runs. But... I've now verified that flash next is both better and faster than 27b so I'm struggling to find a reason to use it. :/ Why are all these solo devs pulling something off for weeks that llama can't figure out?
document fetched 2026-11-02 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1xky7p0/age_of_the_bespoke_engines