Open weight LLM run locally?
retained text
I'm a CPA running a small 2-CPA/no-employee tax practice. We're architecting a completely local AI environment: open-weight model, local compute, RAG over tax authorities/research materials, and eventually secure retrieval over client workpapers and prior returns. Have any small firms or sole practitioners deployed local LLM/RAG yet? • What hardware are you running? • Which model/model size? • What are you using for RAG/vector storage? • Are you indexing IRC/Regs/Rev. Rulings/cases or commercial tax materials? • Genuinely useful for tax research/review? Or still mostly a science project? I'm trying to determine whether there are 5 firms doing this, 500 firms doing it, or 5,000. I'm guessing closer to 50>?
document fetched 2026-09-19 00:18 · canonical https://reddit.com/r/Accounting/comments/1wjr532/open_weight_llm_run_locally