We seriously need benchmark for research, wide web search and fact retrieval.
retained text
https://preview.redd.it/kzf66pdcvurh1.png?width=1080&format=png&auto=webp&s=1ec903085f115f01510ee56886b30e9e685219ea Almost all the benchmarks are either saturated or tested under strict conditions. For example - AA Omniscience have restricted tool access. ***Many people use Chatbots for information retrieval, deep research and broad information gathering.*** There are few benchmarks like - https://preview.redd.it/b0ky46sdvurh1.png?width=257&format=png&auto=webp&s=fe6375b098eaab1c4f8341341f2229a1541e07f7 But the problem is - they don't have any official leaderboard and were last updated years ago. ***We really need a benchmark which is tested in actual environment (tools access and allowed internet search) along with used harness systems without any restrictions (like Claude, ChatGPT Work, etc).*** If there exist a benchmark like this, can someone please tell?
document fetched 2026-09-27 00:31 · canonical https://reddit.com/r/singularity/comments/1wqoxlg/we_seriously_need_benchmark_for_research_wide_web