← sources

Claude Code's spring regression was a harness bug, not a model bug. I compared Claude Code's harness with 11 other production agents.

source: reddit (reddit:AI_Agents)published 2026-10-21 08:48ingested 2026-10-22 02:36

retained text

    Quick recap of why I started: in March and April, Claude Code got measurably worse for about a month. Per Fortune's account of Anthropic's postmortem, one change made the model keep discarding its own reasoning history mid-session, and another capped responses at 25 words between tool calls. The API was unaffected. Same model, different harness.
    
    
    So I read what twelve teams have published about the code around their models (Codex, Claude Code, Cursor, Devin, Stripe's Minions, Manus, Agentforce, Shopify Sidekick, Sierra, Intercom Fin, Harvey, Hebbia) and compared them.
    
    
    Where Claude Code sits:
    
    
    - 
    **It's a thin layer**
     around the model: tools, context management, and file snapshots so you can undo. Compared with Stripe or Salesforce, very little of the path is fixed in code.
    - 
    **That works because coding mistakes are cheap to catch.**
     Compilers, tests and git catch most errors after the fact. Stripe makes the same point: coding agents work inside decades of fast, verifiable guardrails, and knowledge work has very little of that.
    - 
    **The risk is the one the regression showed.**
     When the harness is thin, a small harness change moves quality a lot, and nobody outside sees it until users complain. The fix is boring: rerun a fixed task set after every harness release and track reliability across repeated runs, not single runs.
    
    
    Two things that surprised me across all twelve: none publishes cost per successful task, and almost everything the teams say about their harnesses is self-reported (66 of the 83 claims I collected).
    
    
    One Claude detail from my own setup: the agent that drafts these posts runs through headless Claude Code (`claude -p`) with every tool switched off. It's a plain model call on my subscription, and Claude Code reports each run's cost at the API list price, which is how I measure cost per accepted draft. I set the pass mark before the first run and publish the result either way.
    
    
    If you've seen Claude Code's harness behave differently from what I describe, tell me. That's the correction I most want.
    
    
    The comparison, every claim linked: harness[dot]psyprasad[dot]tech[/]compared

document fetched 2026-10-22 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xbhvh8/claude_codes_spring_regression_was_a_harness_bug