# Just "rediscovered" batching and swarming while running models locally. — retained record

> Retained reddit record from reddit:LocalLLaMA.

Canonical: https://highsignal.app/data/records/b9fe63b8f7f318b5

[← sources](https://highsignal.app/data)


source: [reddit](https://highsignal.app/data/reddit) (reddit:LocalLLaMA)  published 2026-10-31 01:16 ingested 2026-11-01 02:40

[original source ↗](https://www.reddit.com/r/LocalLLaMA/comments/1xjx63n/just_rediscovered_batching_and_swarming_while/)

## retained text



```
Running one harness/agent gives overall less tokens-per-second (tps) than running multiple agents at the same time. Who'd have thought?

Though each agent has slower tps and a smaller cache, on average they are much more efficient overall in the token generation.

I'm Running a 3090 with 24gb of VRAM.
Qwen3.8 27B Q4_K_M    
Context: 197k

Of course the more agents you have, the less context each gets. Thus one needs to auto-compact / loop efficiently. This way one can run multiple agents on one piece of hardware and maximise the burn-rate without increasing the energy used.

From there swarms can be allowed, were each agent has the ability to spawn as many sub-agents with small tasks upto the optimum amount of agents per machine (12 on mine).

Agents generating | Total tok/s | Per agent
---|---|----
1 | 30 | 30
2 | 46 | 23
4 | 64 | 16
6 | 74 | 12
8 | 78 | 10
12 | 110 | 9
16 | 111 | 7
```



document fetched 2026-11-01 00:38 · canonical https://reddit.com/r/LocalLLaMA/comments/1xjx63n/just_rediscovered_batching_and_swarming_while
