What are benchmarks where even the most recent frontier models barely make a dent or outperform previous ones only minimally?
retained text
Many benchmarks now see scores in at least the mid-twenties (the ones most recently reported by Anthropic for Fable 5.2 are all above 40%) if we set aside ELO-scored ones, What are challenging benchmarks with low top percentages and/or where the needle has been barely moving over the past months? I'll start with [**SWE-sweep**](https://swesweep.com/) (mascot pictured), which deals with bug discovery and remediation without any hints, with a current top score of 9,5% (Astra). Fable 5.2 hasn't been tested yet, but Fable 5.1 scored 2.9%.
document fetched 2026-11-11 00:30 · canonical https://reddit.com/r/singularity/comments/1xstb50/what_are_benchmarks_where_even_the_most_recent