What building an agent harness around GPT-3.5 Turbo taught me: every fix was code, not a prompt
retained text
I gave a talk at AI Engineer Europe on agent harnesses, and the live demo taught me more than the slides did, so here is the short version. **The task:** an agent drives a real browser with Playwright and upvotes the top story on Hacker News. I used GPT-3.5 Turbo on purpose, and I set one rule: the prompt never changes. 1. **A bare agent loop.** It clicked upvote, hit the login wall, and reported success anyway. The loop ended when the model said it was done, so it was "done". 2. **Guardrails.** A maximum number of iterations and a maximum number of messages, plus very naive context trimming (keep the system prompt and the task, drop the middle). This fixes nothing, but it bounds the damage. 3. **A deterministic verify step, plus retries.** After each attempt, plain code reads the tool call trace and checks whether the upvote actually happened, with up to 3 attempts. It still failed, but it stopped lying about it. 4. **A login handler in the harness.** Before each step, the harness checks the browser URL. On a login page it fills in credentials from environment variables the model never sees, then tells the model it is logged in. It succeeded in 6 iterations. What I took away: * A model saying it finished is a claim, not evidence. Verify it in code. * Put secrets and fragile, deterministic steps in the harness, not in the prompt. * The harness is not the agent loop. It is everything around the loop, and sometimes a loop around the loop. How do you all handle verification? Do you check completion in code, or hand it to a second model? Disclosure: I wrote this up in full with the code for each step. Link in the comments, per the sub rules.
document fetched 2026-09-15 00:20 · canonical https://reddit.com/r/AI_Agents/comments/1wg0rqd/what_building_an_agent_harness_around_gpt35_turbo