# What broke when I split a voice agent into a talker and background workers — retained record

> Retained reddit record from reddit:AI_Agents.

Canonical: https://highsignal.app/data/records/6a1bbe947f0cf941

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:AI_Agents)  published 2026-09-30 02:11 ingested 2026-10-01 02:37

[original source ↗](https://www.reddit.com/r/AI_Agents/comments/1wtsyln/what_broke_when_i_split_a_voice_agent_into_a/)

## retained text



```
I've been building a voice assistant where a live voice model only talks and anything slow goes to background agents with a shell and a real browser. Some things I didn't expect:



The talker needs a list of what the workers can do, or it stops handing things off. When I took the capability list out of the voice prompt, "let's hang up" went from reaching the backend 5 times out of 7 to 1 out of 21.



It says "on it" and does nothing. In one 20 minute session the voice said it would take care of something four times with no hand-off after the words. The prompt now says "delegate before giving an answer that depends on backend work", and I count hand-offs per user turn.



Workers need a way back into the conversation. A worker's result or question gets relayed into the live call, and only the worker that asked waits; the rest keep going.



Jobs should outlive the call. People hang up mid-task. The work keeps running and the result is there next time, or the screen rings when it's done.



Silence costs money. The voice bills per minute including silence, so an idle call hangs up on its own. I started at 40 seconds and it was too long to sit through; it's 25 now, with the last ten counted down on screen.



How do others deal with the "said it but didn't do it" problem with realtime voice models? Repo and a short film are in the comments.



(English isn't my first language, I had an LLM help me write this.)


```



document fetched 2026-10-01 00:20 · canonical https://reddit.com/r/AI_Agents/comments/1wtsyln/what_broke_when_i_split_a_voice_agent_into_a
