← sources

Unpopular take: If anything, it's the US Closed Models that are benchmaxxed

source: reddit (reddit:LocalLLaMA)published 2026-11-17 18:23ingested 2026-11-18 02:42

retained text

Open models: whatever the test group download and benchmark is the same as what you get, you work with it day-to-day, same results. What you see is what you get.

  
Closed models: we all know that they have already implemented token watermarking for a while and all latest models have overzealous guard rails that refuse tasks that it deems as bad. So the models being benchmarked and the models you are actually using day-by-day basis are likely to be very different, as in the sense of you getting degraded model output and performance or flat-out task rejections.

Also closed models providers implement lots of cost-optimization techniques such as reducing reasoning efforts during peak hours or could be re-direct your request to a lesser model during high traffic without you knowing it. I don't believe that Sonnet 5.5 is just a single model TBH, there could be Sonnet High or Sonnet Lite depending on your subscription level and traffic. That could explain why sometimes you get dumb responses with Claude during peak hours.

  
If anything, it's the US Closed Models that are benchmaxxed.

document fetched 2026-11-18 00:45 · canonical https://reddit.com/r/LocalLLaMA/comments/1xys12t/unpopular_take_if_anything_its_the_us_closed