How are you affording your tokens?
retained text
So I have been using local QFN these days as daily driver, and honestly, its fine. It needs steering for coding but very good. Gets lot of things done. But I check my statistics log, and I'm burning about 1B to 2B Input token a day, and not even fully 100% GPU saturation. Not even going to consider output token since that is small compared to input. Most of the time, the LLM is just waiting for shells/bash to finish. At max capacity, it is capable of 950million uncached input token (11000 prefill/s\*3600\*24) This is light for me, and it is already burning this much. Light considering it is only spend time building one project with little oversight. Granted QFN is at xhigh thinking for intelligent, but if this was like Haiku or Luna, that's like 10$ cost a day (90% cache estimate). That's 300$ a month for basically one project stream AI suppose to work multiple projects at once for max productivity let alone future integration with pretty much everything. My wife already starting to use my harness with QFN for all her search and queries. How are you guys make this affordable in the long term from purely API. Feeling paying like $1000+ a month is crazy. Local one still cost insane upfront as I'm sing a single RTX6000 with basalt, and it is the only GPU that can serve the whole household growing AI needs. PS. I might get like 4x in near future for the amount of load I will be doing. Always on agent is very useful. Edit: The number is measure of input only, as output is insignificant. I'm measuring QFN only at home.
document fetched 2026-11-14 00:46 · canonical https://reddit.com/r/LocalLLaMA/comments/1xv9kgh/how_are_you_affording_your_tokens