Looking for "load testing" tools
retained text
Basically, think of HTTP load-testers, but for inference. I want to throw generations (parallel) at my inference server and see how it behaves under sustained load and meassure token generation and prefill and figure out the actual storage needs in-use. For HTTP stuff, I was using siege and k6; but what do I use to run such a load-test on an LLM server..? Any good known tools for that? Thanks!
document fetched 2026-10-12 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1x2ucg7/looking_for_load_testing_tools