# How much code do AI models and harnesses add? I made an open-source benchmark to measure code complexity. — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/ee972a946085086f

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-11-08 22:19 ingested 2026-11-09 02:37

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xrathc/how_much_code_do_ai_models_and_harnesses_add_i/)

## retained text



```
  
Existing benchmarks measure the solve rate and token output/cost, but don't punish excess code or overengineering, which increases technical debt in the long term.  
  
Parsimony counts syntax units (statements, calls, names) in the patches other benchmarks publish. It doesn't count lines, comments, tests. Each model or harness is compared with the human fix for the same issues it solved.  
  
  
\- Smarter models surprisingly write slightly more code.

\- Models: the median model adds 1.9× the human fix on SWE-bench Verified (older models), but about 1× on DeepSWE.

\- Harnesses: with the same two open-weight models, the harness alone moves it from 0.80× (Codex) to 0.99× (mini-SWE-agent, OpenClaw). The harness data comes from Joel Niklaus's Harness Leaderboard.

[https://parsimony.lasu.dev](https://parsimony.lasu.dev/)  
  
[https://github.com/La5u/Parsimony](https://github.com/La5u/Parsimony)  
  
Looking for: public patches for newer models, any ideas or contributors for self run models.  
  
\* some newer models are self run in codex and claude code, shown as faded (unverified) in DeepSWE
```



document fetched 2026-11-09 00:36 · canonical https://reddit.com/r/LocalLLaMA/comments/1xrathc/how_much_code_do_ai_models_and_harnesses_add_i
