# I analyzed 246 repos and 57 papers on agent harnesses. Here's what actually works — retained record

> Retained reddit record from reddit:ClaudeAI.

Canonical: https://highsignal.app/data/records/e946cd451c81a0a7

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:ClaudeAI)  published 2026-09-25 15:53 ingested 2026-09-26 02:35

[original source ↗](https://www.reddit.com/r/ClaudeAI/comments/1wpzynz/i_analyzed_246_repos_and_57_papers_on_agent/)

## retained text



```
It seems like more and more people are building agent harnesses (me included lol), but very few people are measuring whether they actually work. 

Quick context for anyone new to agent harnesses: a harness constrains, guides and elevates an AI model.

Since harness engineering is still a relatively new field, I wanted to do my own research to figure out which practices are emerging as best practices vs. which ones are just hype. 

So I looked at 246 open-source repositories and 57 publications to try to get a clearer picture. (FYI I’ve limited the research to harnesses used for software development and coding agents as it’s what I know the best.) 

Thought I’d share some of my main takeaways here in case it saves anyone else going down that rabbit hole.

**TLDR**: The pattern is pretty clear: the best harnesses are small, evidence-based and constantly refined.

What seems to work:

* **Write the harness yourself (or have another human do it):** An ETH Zurich study found that machine-generated context actually *reduces* task success compared to giving the agent no context at all, while increasing inference cost by 20%. Human-written context improved success by about 4%. So yeah, don't outsource writing the harness to an agent.
* **Build the harness in response to observed failures:** Modern agents are already pretty capable, the harness should compensate for actual weaknesses you've seen.
* In the same vein: **Limit the tools and skills exposed to the agent:** Only add skills or specialized subagents when there's a real need, not just because they are popular. Vercel removed 80% of their tools and success jumped from 80% to 100%, with half the tokens. Microsoft cut 100 tools down to 2. Less really is more here.
* **Change one component at a time and measure the impact:** Basically the scientific method. If you change 5 things and things get better, you have no idea which one helped (or which one is silently making things worse).
* **Have both tests and evals:** Just like code, a harness may contain bugs or overly verbose or ambiguous instructions that don’t work all the time. Don't assume your [`AGENTS.md`](http://AGENTS.md), hooks, or scripts are doing anything just because they exist as instructions can simply be ignored.
* **Test both "should happen" and "should NOT happen" cases.** This one bit me personally. If your tests only contain things that must be blocked, tightening the guard is always safe and loosening it is invisible. So the harness slowly drifts toward blocking everything and basically strangling the agent. One-sided evals create one-sided optimization.

Curious to hear from others building agents: how are you measuring whether your harness is actually helping? 
```



document fetched 2026-09-26 00:22 · canonical https://reddit.com/r/ClaudeAI/comments/1wpzynz/i_analyzed_246_repos_and_57_papers_on_agent
