← sources

Open weight LLM run locally?

source: reddit (reddit:Accounting)published 2026-09-18 14:04ingested 2026-09-19 02:37

retained text

I'm a CPA running a small 2-CPA/no-employee tax practice. We're architecting a completely local AI environment: open-weight model, local compute, RAG over tax authorities/research materials, and eventually secure retrieval over client workpapers and prior returns.

Have any small firms or sole practitioners deployed local LLM/RAG yet?

• What hardware are you running?  
• Which model/model size?  
• What are you using for RAG/vector storage?  
• Are you indexing IRC/Regs/Rev. Rulings/cases or commercial tax materials?  
• Genuinely useful for tax research/review? Or still mostly a science project?

I'm trying to determine whether there are 5 firms doing this, 500 firms doing it, or 5,000.  I'm guessing closer to 50>?

document fetched 2026-09-19 00:18 · canonical https://reddit.com/r/Accounting/comments/1wjr532/open_weight_llm_run_locally