← sources

What are benchmarks where even the most recent frontier models barely make a dent or outperform previous ones only minimally?

source: reddit (reddit:singularity)published 2026-11-10 17:40ingested 2026-11-11 02:39

retained text

Many benchmarks now see scores in at least the mid-twenties (the ones most recently reported by Anthropic for Fable 5.2 are all above 40%) if we set aside ELO-scored ones,

What are challenging benchmarks with low top percentages and/or where the needle has been barely moving over the past months?

I'll start with [**SWE-sweep**](https://swesweep.com/) (mascot pictured), which deals with bug discovery and remediation without any hints, with a current top score of 9,5% (Astra). Fable 5.2 hasn't been tested yet, but Fable 5.1 scored 2.9%.

document fetched 2026-11-11 00:30 · canonical https://reddit.com/r/singularity/comments/1xstb50/what_are_benchmarks_where_even_the_most_recent