← sources

Ran a six-model swarm like a 1930s design bureau. The swarm added +0.05, splitting the work did everything else

source: reddit (reddit:AI_Agents)published 2026-10-04 16:11ingested 2026-10-05 02:36

retained text

Everyone's stacking agents right now, so here's a boring result. Six small local models as a swarm, organized like an old Soviet design institute: disciplines, checks inside, checks across. The swarm part added +0.05 over the best single model. Cutting the input into blocks did the rest.

On my extraction task (18 records in the context, return the ones matching a condition) recall goes from 0.609 at position 0 to 0.214 at position 17. Monotonic, no U. ~~The model just says "no" more and more often the further it reads.~~ Wrong, see the edit at the bottom.

So I did what a design institute does with any job that's too big for one guy. Cut it into disciplines.

Why that was my first idea. Albert Kahn's Detroit firm more or less bootstrapped Soviet industrial design around 1930, five hundred-something factories and a few thousand trained engineers. The org chart stuck: split by discipline, check inside the discipline, check across disciplines, normocontrol before anything leaves the building. I work in one of those institutes. Yes, I know how this sounds.

Setup: six local models, all 1-2B (qwen2.5-coder-1.5b, llama-3.2-1b, internvl3-2b, qwen3-1.7b, gemma-3-1b, smollm2-1.7b). 40 test tasks, 18 records each, cut into 3 blocks of 6 by a script. Not by a model, an LLM planner scored 22% in an earlier run of mine. Every model reads every block. An id that isn't in its block gets dropped as a hallucination by a dumb check. Votes for a record only count from its own block. 720 calls, about 10 minutes on a GTX 1660 SUPER 6GB. Thresholds written down before the run.

Recall, head vs tail:

| records | full context | blocks |
|---|---|---|
| positions 0-5 | 0.508 | 0.595 |
| positions 12-17 | 0.304 | 0.574 |
| gap | 0.204 | 0.021 |

Gap down 10x. Position stopped mattering.

The part I didn't expect. Exact set match per model:

| model | full context | blocks |
|---|---|---|
| qwen2.5-coder-1.5b | 0.175 | 0.700 |
| llama-3.2-1b | 0.400 | 0.725 |
| internvl3-2b | 0.575 | 0.825 |
| qwen3-1.7b | 0.725 | 0.850 |
| gemma-3-1b | 0.000 | 0.050 |
| smollm2-1.7b | 0.000 | 0.000 |

The weaker the model, the bigger the jump. The coder went 4x and nearly caught the leader. Which is literally Kahn's whole pitch: divide the work right and mediocre workers produce good output. gemma and smollm can't do the task at all, and no org chart fixes that.

Now the embarrassing part. The actual goal was emergence, the swarm getting right what no single model could. My pre-registered threshold said 34 of 40, champagne. Then I looked at what I was comparing: the full pipeline against a raw single model. Put the best single model (qwen3-1.7b) through the same pipeline: 0.850. All six: 0.900. Oracle "any of the six got it": 0.975. Strict emergence: zero. The swarm adds +0.05 and never steps outside what at least one model already had. The 34 stays in the json next to the 0, deleting it would be cheating.

So out of seven Kahn components one carried the load, the split. The in-block check caught 5 hallucinated ids for free. The cross-discipline check had nothing to do, blocks in this task are independent. That's the next experiment.

Caveats before someone asks:

- 1-2B only, I have 6GB. Gains shrink as the model gets stronger (qwen3-1.7b: +0.125), so on 7B+ I'd expect less. Not measured.
- 40 tasks, one task family, one split. Block size 6 came from the recall curve, never tuned.
- Overall exact set 0.750 -> 0.900. The final yes/no hit 1.000 but that's the softer metric, 0.900 is the honest one.

If you feed a small model a pile of documents in one go: don't. Cut it, run the pieces, check ids with code, not with another model. Model-as-judge is a separate post, short version: worse than you think.

Report, protocol and scripts are in the comments, links in posts aren't allowed here.

If you run multi-agent setups: did you ever put the swarm against its best single agent run through the same pipeline? Curious if anyone got more than noise.

The experiment, the numbers and the dumb emergence threshold are mine.

(not a native speaker, an LLM helped with the English)

Edit: u/QuanTradin suggested logging the yes rate by position, so I did, on the saved runs, zero new calls. It barely moves: 0.21 on positions 0-5, 0.19 on 12-17, same share of matching records at both ends. Recall drops 0.50 -> 0.30 and yes on non-matching records goes up 0.14 -> 0.16. So the model doesn't start saying "no". It keeps saying yes about as often and lands on the wrong records more. The struck line above was my wrong explanation, left in place.

document fetched 2026-10-05 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1wxjg71/ran_a_sixmodel_swarm_like_a_1930s_design_bureau