RAGFlow 1.0 preview: moving from RAG engine to agent-native database
retained text
**Context as Infrastructure: Knowledge Compilation for Retrieval in Agentic Systems with Harnesses** 📄 [https://arxiv.org/abs/2610.41172](https://arxiv.org/abs/2610.41172) Sharing the paper behind RAGFlow 1.0. RAG’s center of gravity has moved. Retrieval is now a loop: the agent queries, reads, reformulates, and decides when it has enough. Once that loop exists, better chunking/embeddings/Recall@k stop being the main bottleneck. The agent can find things. The problem is the cost of reading. Two conclusions from the paper: 1. Retrieval is a harness problem before it’s an index problem. The index decides what can be found; the harness decides whether finding it becomes an answer. 2. Compilation is not a better retriever, and doesn’t need to be. We tested summary trees, outlines, wikis, and graphs under a competent harness. None gave a detectable accuracy gain. Structure isn’t extra evidence. What compilation can do is supply better context: smaller, provenance-bound, and sufficient. So we stopped treating compilation as retrieval and started treating it as a supply layer. At ingestion time, docs are turned into typed artifacts—claims with verbatim evidence, relations, timelines, etc. The harness then decides which artifacts enter the context window and when to stop. Context becomes infrastructure when it’s compiled once, typed, versioned, and governed. The tradeoff is explicit: we save on reading cost, but pay for it in extra retrieval rounds. For claims, graph, and wiki, we published an empty results ledger—metrics and decision rules are listed, but every row is marked “no measured effect” because none has met its pre-registered target yet. An artifact the harness never uses is just cost, not context. I’d rather say that than fill the column with claims from demo collections. Happy to discuss.
document fetched 2026-11-03 00:43 · canonical https://reddit.com/r/LocalLLaMA/comments/1xlubz9/ragflow_10_preview_moving_from_rag_engine_to