My AI can follow its instructions—but often fails to retrieve them when needed. How do you solve this?
retained text
I’m a non-engineer in Japan, building a long-term AI assistant system through real-world testing with ChatGPT and Google Drive. I’ve previously posted here about failure modes in an iPhone-based AI operations system. Your feedback helped me improve its design. After months of testing, I’ve narrowed down one particularly stubborn problem. The instructions already exist. The AI can follow them when explicitly asked to read them. But it doesn’t consistently recognize when it needs to retrieve them. The system already has: * Canonical documents and authority rules * A context-resolution design for mapping conversations to relevant projects and sources * Freshness and conflict checks * Human approval and execution verification * Recovery procedures for uncertain results The failure happens earlier. During an ordinary conversation, the AI may start reasoning or suggesting a solution without first consulting an existing instruction that already addresses the situation. If I simply say, “Check the relevant project documents first,” the quality of its response often improves significantly. So the problem isn’t primarily storing knowledge or following retrieved instructions. It’s reliably triggering the right retrieval at the right moment. I don’t want to hard-code hundreds of keywords or turn natural conversation into a rigid workflow. I also don’t want to load every instruction into every conversation. I’m looking for approaches that preserve flexible conversational reasoning while making relevant instruction retrieval more reliable. My questions: 1. Has anyone solved this problem in a production agent? 2. How do you detect when retrieval is necessary without depending entirely on the LLM to remember to do it? 3. Are there practical approaches to intent-based routing or dynamic instruction loading that generalize across different projects? 4. What failure modes remain even with a well-designed retrieval system? 5. Am I framing the problem incorrectly? I’m especially interested in implementations, measured results, and counterexamples—not generic advice to “just use RAG.” I don’t expect perfect autonomy. A short user reminder is an acceptable fallback. I’m trying to reduce how often that reminder is necessary. What would you test or change first?
document fetched 2026-10-29 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xho4ik/my_ai_can_follow_its_instructionsbut_often_fails