loading
loading
Sarvam's Knowledge Fabric Beats GPT-5 on Legal Benchmarks at Half the Cost Sarvam's knowledge fabric restructures enterprise documents into a navigable tree and typed ontology, lifting agent accuracy to 84.2% while slashing costs. - Sarvam's knowledge fabric compiles enterprise docs into a navigable tree plus a typed domain ontology. - Harvey LAB: 84.2% rubric met vs 55.1% for agentic RAG, same GLM 5.3 model. - Cost per LAB task fell from $0.64 to $0.28, with 14.2 tool calls versus 36.5. - On Sierra's tau-knowledge, matched baseline accuracy with 64% fewer tokens. - Tree alone lifted BrowseComp-Plus from 75.5% to 91.3% but used more tokens than RAG. - Served over MCP, parses documents with Sarvam Vision 2.1, deployable air-gapped. Sarvam has introduced a knowledge fabric that compiles enterprise documents into two interfaces for AI agents: a navigable knowledge tree and a typed domain ontology. The company reports that this structure helped the same underlying models answer more legal questions correctly and match a banking baseline with fewer tokens. For developers, the design moves recurring work from inference time to ingestion. An agent can browse source hierarchies, filter entities, join records and cite passages without reconstructing the corpus structure for every task. Same model, different data layer Sarvam kept the model constant within each benchmark comparison, reducing model choice as a confounding variable. Prompts, tools and agent behavior can still affect results, and all figures below are company-reported. Sarvam also compares its 84.2% result with scores from Harvey’s own harness: 79.1% for GPT-5.6 Sol and 71.7% for Opus 4.8. Different models and harness configurations limit that cross-report comparison. On Sierra’s tau-knowledge banking benchmark, GLM 5.2 solved 39 of 97 tasks with the fabric and 38 with Sierra’s default retrieval setup. Accuracy was effectively tied, while the fabric used 64% fewer tokens. Where RAG drops context Chunk-based retrieval often separates a passage from the headings, footnotes, tables and customer segments that qualify it. A retrieved excerpt can appear complete while omitting a condition that changes the answer. Vector search also ranks semantic similarity without guaranteeing coverage. Tasks that require evidence from several files can fail when one relevant document never reaches the model, even if every retrieved passage looks plausible. A tree for browsing, an ontology for joins The knowledge fabric exposes two compiled layers with distinct jobs: - Knowledge tree: Organizes the corpus like a filesystem, with folders, document summaries, page-range reads and tables preserved as structured data. - Domain ontology: Defines entity types, properties and relationships, allowing agents to run database-style filters and joins across documents. Parsing begins with Sarvam Vision 2.1, which converts scans, complex layouts, tables and Indic scripts into Markdown. The tree is exposed through the Model Context Protocol, or MCP, a standard interface for connecting agents to external tools. Agents can list folders, inspect indexes, open page ranges and read tables; Sarvam says citations resolve to the original passages. The ontology produced the efficiency gain The Legal Agent Benchmark ablation shows that the tree improved answer quality while increasing exploration. The knowledge fabric achieved 84.2% accuracy on Harvey's LAB benchmark while reducing average task cost from $0.64 to $0.28. The ontology raised the score to 84.2% while cutting tool calls, tokens and cost below the RAG baseline. Database queries handled filtering and joins, reducing the repeated reading and synthesis performed by the model. One benchmark task required every firm-commitment capital-markets offering to be joined with its executed underwriting agreement. Flat retrieval found 7 of 14 qualifying offerings and added 3 false positives. The ontology query returned all 14, produced no false positives and included source citations. Compilation changes the cost model The reported benchmark costs cover each task after the fabric has been built. Initial parsing, ontology construction, incremental rebuilds, expert review and storage remain unquantified in the published figures described here. Those expenses will determine the break-even point for smaller corpora or frequently changing data. A production evaluation also needs answers to several operational questions: - Freshness: How quickly do document changes propagate into the tree and ontology? - Permissions: Do queries and citations enforce the source system’s access controls? - Schema evolution: How are new entity types and changed relationships migrated? - Regression testing: Can teams replay representative tasks before publishing a new fabric version? - Observability: Can developers trace each answer through tool calls, ontology queries and source passages? Sarvam says agents can draft domain models for expert review, while proposed corrections are versioned and re-evaluated before release. The fabric can also run inside an organization’s environment, including air-gapped deployments disconnected from the public internet and installations using smaller models. Best suited to cross-document work The approach fits domains where answers must aggregate evidence and survive audit, including banking product catalogs, legal matters, insurance policies and regulatory filings. Each domain requires its own ontology, making stable, repeated workflows stronger candidates than one-off collections. The tree also improved four broader retrieval benchmarks without a domain-specific ontology. Sarvam ran GLM 5.2 over the same parsed documents in each comparison. Shared memory comes next Sarvam’s broader claim is that agents repeatedly perform work that can be compiled into reusable structure. Typed entities, explicit relationships and navigable document hierarchies can reduce repeated search and synthesis when a corpus receives enough queries to justify the ingestion cost. The company has identified shared agent memory as the next layer. That system would use the same primitives to retain corrections and user preferences across agents, subject to the same requirements for versioning, permissions and evaluation.
document fetched 2026-10-27 02:55