I run a news pipeline unattended, three episodes a day. The eval loops are the system, not the models.
retained text
Six thousand plus episodes in about 19 months, roughly three a day on a six-hour cadence, and people always ask which model I use. It's the wrong question. The model is swappable. The checks are what keep it from publishing something embarrassing at 6 AM while I'm asleep. How it runs: newsletters land in email. The system decides whether there are enough topics for an episode, writes the script, builds the avatar video in my cloned voice, then publishes a video, a blog post, and a LinkedIn post. About $3 to $3.50 an episode in inference. What I've learned: Evaluation loops have to block, not report. Every output gets checked against the required behaviors and the prohibited ones. Missing behavior, no publish. Reviewing after the fact only means the bad episode already shipped. A database beats a prompt for deduplication. A Postgres topic store, checked before anything gets written. Prompts drift, queries don't. Cross-review the architecture with a different model. One model writes the plan and the reasoning, another critiques it, I feed the critique back. That's how I found exposed API keys. Deterministic where correctness matters, probabilistic where it doesn't. Thumbnails are generated imagery with brand placement composited deterministically. The typography is never left to a model. Write handoffs, not long contexts. Small units of work, a written handoff at the end of each, then compact before switching tools. Otherwise, the agent forgets what it decided an hour ago. Where it breaks: distribution. 6,400 episodes, under 250 subscribers. I built the machine and skipped the audience, which is what I'm fixing now. Happy to go deeper on the eval setup if anyone wants it.
document fetched 2026-10-22 00:19 · canonical https://reddit.com/r/AI_Agents/comments/1xbp0nq/i_run_a_news_pipeline_unattended_three_episodes_a