We seriously need benchmark for research, wide web search and fact retrieval.
retained text
https://preview.redd.it/mflm6o9krurh1.png?width=1218&format=png&auto=webp&s=8c62560d67d002f734a18dbc660c1828ff4b67a7 Almost all the benchmarks are either saturated or tested under strict conditions. For example - AA Omniscience have restricted tool access. ***Many people use Chatbots for information retrieval, deep research and broad information gathering.*** There are few benchmarks like - https://preview.redd.it/zfehx31nturh1.png?width=257&format=png&auto=webp&s=588f6e77ab93d0228e9c90f5b8ea39c8b931948f But the problem is - they don't have any official leaderboard and were last updated years ago. ***We really need a benchmark which is tested in actual environment (tools access and allowed internet search) along with used harness systems without any restrictions (like Claude, ChatGPT Work, etc).*** If there exist a benchmark like this, can someone please tell?
document fetched 2026-09-27 00:29 · canonical https://reddit.com/r/OpenAI/comments/1wqouwy/we_seriously_need_benchmark_for_research_wide_web