← sources

Where agent spend actually leaks — it's rarely the model you picked

source: reddit (reddit:AI_Agents)published 2026-10-24 06:13ingested 2026-10-24 10:56

retained text

Something I keep coming back to when thinking about agent design: per-run cost is mostly structural, not model-tier. The model choice is the line item everyone argues about; the actual leaks are in the architecture. Four patterns I'd design against:

1. Unbounded retries replaying full context. A stuck step that retries 3 times doesn't cost 3x one call — each retry replays the whole trajectory, so one flaky step can burn through tens of thousands of tokens to produce nothing. I'd cap retries, and more importantly shrink what's replayed per attempt: the last error plus the current state, not the entire conversation.

2. Context bloat from tool results. Full API responses, directory listings, and stack traces dumped into the loop mean every subsequent call is priced on accumulated junk. I'd truncate at the tool-result layer — keep the fields the agent needs, summarize or drop the rest before it ever enters context.

3. Fan-out multiplication. Five parallel sub-agents for "quick research" each get a full copy of the context. Parallelism buys speed, not savings — the bill is context × N. I'd keep shared state in one place and hand sub-agents the minimal brief, not the world.

4. No per-task cost accounting. Teams can quote per-million-token prices but can't say what a single task costs end to end. If it's not measured per run, changes can't be compared, so nothing gets cheaper. I'd log tokens per task the same way I'd log latency.

The short version: cost scales with trajectory length and context width, not output quality — so design for short trajectories and narrow context.

What's the line item that surprised you most when you first measured per-run cost — retries, tool bloat, fan-out, or something dumber?

document fetched 2026-10-25 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xe124q/where_agent_spend_actually_leaks_its_rarely_the