Voice booking agent on ElevenLabs + AWS: tool turns take 5 s. How would you fix it?
retained text
\*\*TL;DR:\*\* Hosted LLM on ElevenLabs, deterministic tools on AWS, an LLM judge after every call. Prices and availability are now right. The problems are latency on tool turns and keeping the booking steps in order. Diagram attached. \*\*Setup\*\* \- ElevenLabs: STT, turn-taking, TTS (\`eleven\_v4\_turbo\`), hosted LLM GLM 5.2 with reasoning off, one prompt per language (EN/ES) \- AWS Lambda tools: \`quote\` (same pricing function the checkout charges), \`availability\` (live), \`prepare\_booking\` (issues a one-time code; checkout won't accept a booking without it) \- Per-call state in DynamoDB, so the agent doesn't re-ask \- After each call, Claude Haiku 4.5 on Bedrock judges the transcript against 4 checks \*\*Results\*\* (128 calls, 108 simulated) Price 101/103 · availability 76/76 · permission + checkout 54/70 · no re-asking 120/126 \*\*Problems\*\* \- First audio on tool turns: P95 5.2 s (target 2.5 s). Tools take 105–454 ms; the rest is the model \- The booking order lives in the prompt, so the agent sometimes skips steps \- The judge sometimes misses failures \*\*Questions\*\* 1. Booking order: ElevenLabs Workflow nodes, a server-side state machine, or both? 2. How do you get tool turns under 2.5 s on ElevenLabs? 3. How do you make a post-call LLM judge trustworthy? More detail in the comments.
document fetched 2026-10-20 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1x9lnw1/voice_booking_agent_on_elevenlabs_aws_tool_turns