Anyone running AI agents in production? What are you paying for to keep them reliable?
retained text
I’ve been looking into how companies are handling reliability when AI agents are actually talking to customers or making decisions based on company data. Not talking about demos or personal chatbots. I mean actual production systems like customer support agents, internal knowledge assistants, RAG applications, etc. One thing I’m curious about is what happens when an agent gives a convincing answer but uses outdated information, cites the wrong source, or just makes something up. I’ve been building something in this space, but before going further I want to understand how people are dealing with this in the real world. A few things I’m curious about: What tools or services are you currently paying for to monitor, evaluate, or verify your AI agents? (LangSmith, Langfuse, Arize, custom solutions, etc.) What’s the most annoying reliability problem you keep running into? How do you currently catch incorrect answers before they reach customers? Or do you mostly find out after something goes wrong? Are you happy with your current tools, or is there something you wish they actually did? Roughly how much are you spending on reliability, monitoring, or evaluation? Even a ballpark would be helpful. I’m especially interested in whether teams have built their own verification systems because existing tools weren’t solving their problems. Not trying to pitch anything. I’m genuinely trying to understand what the real problems are versus what sounds like a problem in theory. Would appreciate hearing from anyone actually running these systems, including people who tried tools and decided they weren’t worth paying for.
document fetched 2026-11-19 00:18 · canonical https://reddit.com/r/AI_Agents/comments/1xzs28w/anyone_running_ai_agents_in_production_what_are