# What's the actual self-hosted RAG stack when docs can't leave your cloud? — retained record

> Retained reddit record from reddit:selfhosted.

Canonical: https://highsignal.app/data/records/ede11bda54242377

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:selfhosted)  published 2026-10-27 04:11 ingested 2026-10-28 02:37

[original source ↗](https://www.reddit.com/r/selfhosted/comments/1xgirpf/whats_the_actual_selfhosted_rag_stack_when_docs/)

## retained text



```
We ended up pulling our managed APIs out entirely, because our documents aren't allowed to leave our own infrastructure for compliance reasons. And our problem has nothing to do with cost, because the per-page pricing was perfectly reasonable.

And that 'docs not able leave our cloud' constraint rules out Textract, OpenAI embeddings and Cohere in one go, since every one of them ships your data off to someone else. That forced us into self-hosting the whole pipeline, OCR, embeddings and reranking included, and after a couple of weeks of testing this is the shortlist I've built up:  
  
docling holds up well on clean digital PDFs, but it gets noticeably shaky once you hand it scans or handwriting

PaddleOCR-VL deals well with messy multilingual layouts and stays small enough to run on a fairly modest GPU

TEI is dependable for embeddings, though it only serves one model per server  
Infinity handles embeddings and reranking together from a single process

SIE runs OCR, embeddings and reranking behind one endpoint with model-swapping by ID, and it runs inside your own cloud.  
  
What you give up by self-hosting is all the turnkey convenience you used to get for free, so there are no parsed tables handed to you, no confidence scores, and the entire pipeline becomes yours to own and maintain.  
  
We've made peace with that because the residency requirement simply isn't negotiable for us, but I'll admit that if cost were the only thing driving this decision it would be a far harder one to justify.
```



document fetched 2026-10-28 00:30 · canonical https://reddit.com/r/selfhosted/comments/1xgirpf/whats_the_actual_selfhosted_rag_stack_when_docs
