a year of running a multi-model router for real work: seven things that survived, three that died
retained text
background: three models behind one interface (a drafter, a critic, a long context search), a lessons file with provenance, and a garbage collector that deletes any lesson not cited in 30 days. ran it daily for about a year on real work. what survived: 1. roles, not personalities. the critic only reads failures, the drafter never sees its own wins. separation of memory beats naming your agents like pokemon. 2. verification gates. nothing exits the system without a checker: tests, a second model, a human reviewing. an unchecked model output is just a rumor with good formatting. 3. the lessons file. every failure becomes a rule with a date and a provenance tag. rules without provenance get dropped by the gc, and honestly that gc is the hardest worker in the house. 4. small hot context. 64k resident, everything else gets searched and cited, never stuffed. the model listens better when it is not drowning. 5. temperature per task, not per model. 0.2 for extraction, 0.7 for drafting, 0.9 for brainstorming with a human picking the survivors. what died: 6. agent names and personas. cute for a week, useless after. the moment roles exist as memory scopes, names add nothing. 7. voting between models. three models agreeing is not verification, it is peer pressure. they share failure modes because they learned from the same internet. 8. autonomous chains longer than three steps. failure compounds and the audit trail becomes unreadable. three steps, checkpoint, continue. the funniest failure: the critic eventually started rejecting everything and had to be given a quota, a minimum of one approved solution per day, otherwise it optimized itself into a wall. that is the most human thing any of these machines has done. what is your minimum viable stack right now, one model and a dream, or a whole org chart?
document fetched 2026-10-20 00:20 · canonical https://reddit.com/r/artificial/comments/1x9nei2/a_year_of_running_a_multimodel_router_for_real