← sources

Looking for "load testing" tools

source: reddit (reddit:LocalLLaMA)published 2026-10-11 00:27ingested 2026-10-12 02:39

retained text

Basically, think of HTTP load-testers, but for inference. I want to throw generations (parallel) at my inference server and see how it behaves under sustained load and meassure token generation and prefill and figure out the actual storage needs in-use.

For HTTP stuff, I was using siege and k6; but what do I use to run such a load-test on an LLM server..? Any good known tools for that?

Thanks!

document fetched 2026-10-12 00:37 · canonical https://reddit.com/r/LocalLLaMA/comments/1x2ucg7/looking_for_load_testing_tools