← sources

What building an agent harness around GPT-3.5 Turbo taught me: every fix was code, not a prompt

source: reddit (reddit:AI_Agents)published 2026-09-14 11:28ingested 2026-09-15 02:36

retained text

I gave a talk at AI Engineer Europe on agent harnesses, and the live demo taught me more than the slides did, so here is the short version.

**The task:** an agent drives a real browser with Playwright and upvotes the top story on Hacker News. I used GPT-3.5 Turbo on purpose, and I set one rule: the prompt never changes.

1. **A bare agent loop.** It clicked upvote, hit the login wall, and reported success anyway. The loop ended when the model said it was done, so it was "done".
2. **Guardrails.** A maximum number of iterations and a maximum number of messages, plus very naive context trimming (keep the system prompt and the task, drop the middle). This fixes nothing, but it bounds the damage.
3. **A deterministic verify step, plus retries.** After each attempt, plain code reads the tool call trace and checks whether the upvote actually happened, with up to 3 attempts. It still failed, but it stopped lying about it.
4. **A login handler in the harness.** Before each step, the harness checks the browser URL. On a login page it fills in credentials from environment variables the model never sees, then tells the model it is logged in. It succeeded in 6 iterations.

What I took away:

* A model saying it finished is a claim, not evidence. Verify it in code.
* Put secrets and fragile, deterministic steps in the harness, not in the prompt.
* The harness is not the agent loop. It is everything around the loop, and sometimes a loop around the loop.

How do you all handle verification? Do you check completion in code, or hand it to a second model?

Disclosure: I wrote this up in full with the code for each step. Link in the comments, per the sub rules.

document fetched 2026-09-15 00:20 · canonical https://reddit.com/r/AI_Agents/comments/1wg0rqd/what_building_an_agent_harness_around_gpt35_turbo