What stack would you use in 2026 for a fully local AI document assistant that has to understand messy real-world family documents?
retained text
I’m building a fully local document assistant for personal/family use, and I’d like to sanity-check the architecture before I keep adding more code. The end goal is \- throw in hundreds or eventually thousands of messy PDFs/scans/images \- documents can be German, Ukrainian, English, Polish, and occasionally other languages \- some PDFs may contain only one page of a larger document, and the other pages may arrive later or in another batch \- automatically OCR everything \- detect which pages belong to the same logical document \- understand what each document actually is \- extract sender, recipient, organization, document type, topic/category, dates, deadlines, amounts, contract/customer/case/invoice numbers, important actions, etc. \- preserve evidence/provenance for extracted facts \- organize documents dynamically without me manually creating every category beforehand \- search across all documents semantically \- let me ask questions in Russian even though most source documents are German \- return the original document, but explain the result to me in Russian \- later add reminders/tasks, form filling, voice interaction, and a normal UI \- everything must remain local/private with no cloud APIs The machine is reasonably powerful: \- Core Ultra 9 285K \- RTX 4080 SUPER 16 GB \- 64 GB RAM Current general approach: \- document archive / original file storage \- local OCR/document parsing \- local LLM through Ollama \- PostgreSQL for structured metadata/facts \- a validation layer between the LLM and the database \- document cards with provenance \- batch processing \- an agent with tools for document search / facts / tasks \- separate stages for: 1. document classification, 2. people/organizations, 3. dates/amounts/identifiers/facts, 4. relations between documents. One thing I’m running into is that “valid JSON” and high model confidence do NOT necessarily mean the document was understood correctly. For example, a model can correctly understand the general meaning of an invoice, but: \- treat a product name as a person; \- treat the date of a payment as the document date; \- treat a contract date as an appointment; \- miss important amounts; \- analyze page 2/4 as if it were a complete standalone document. So I’m increasingly thinking the hard part is not the chatbot or even the LLM it’s ingestion, document completeness, entity extraction, validation, grouping and retrieval. If you were building this today, what tools / architecture would you use for: \- OCR and layout understanding? \- multi-page / cross-file document grouping? \- multilingual embeddings and semantic search? \- hybrid search (BM25/FTS + vector)? \- entity extraction and normalization? \- evidence/provenance validation? \- deciding whether a document is actually “ready” without manually reviewing everything? \- dynamically building document categories/topics without ending up with hundreds of duplicate labels? \- local RAG / agent orchestration? \- scaling this to 5,000–50,000 documents on one machine? I’m especially interested in lessons from people who have actually built a local document pipeline, not just “put PDFs into a vector DB and chat with them.” Would you still use something like Paperless-ngx + custom processing around it, or would you build the document layer differently? What would you absolutely NOT build yourself today?
document fetched 2026-09-29 00:30 · canonical https://reddit.com/r/selfhosted/comments/1wsfrd3/what_stack_would_you_use_in_2026_for_a_fully