Building quality into the agentic workflow

retained text

Building quality into the agentic workflow
AI coding agents changed how fast teams ship. They didn’t change who has to verify the output.
Last week, Qodo field CTO Shlomo Dalezman and solutions engineer Jonathan Klick ran a live session on what happens when code review stops waiting for the pull request. They walked through the Agentic Toolbox, demoed an install-to-review on a real codebase, and made the case that the PR is now the wrong place to catch most issues. Here’s what came out of it.
The first pass is no longer enough
A year ago, running AI review on a pull request still felt novel. The pitch was simple: code volume had outgrown human review capacity, so let AI take the first pass.
That model assumed the workflow stayed the same. Write code, push, open a PR, review there. As engineers moved deeper into agentic development, the assumption no longer held. When an agent writes most of a change, the PR is where problems surface too late. The work is done, the context is cold, and the burden lands back on a human reviewer.
Jonathan put the question directly: if you’re already leaning on AI to review your code, and the AI is sitting right there in your coding agent, why wait for the PR at all?
Trust is the real bottleneck
The counterargument to shifting review left is usually that teams are already shipping faster, so why add a step?
Data presented at LeadDev New York by DX, the engineering intelligence platform, complicates that. Across their customer base, organizations report real productivity gains from AI, even as trust in the resulting code sits at an all-time low. Velocity went up. Confidence went down.
That gap is the constraint. Teams that don’t trust their agents’ output end up re-reviewing everything manually, which erases the speed they gained. Closing the gap is what makes the next stage of adoption possible.
Software factories have levels
Shlomo offered a frame for where teams sit today, borrowed from autonomous vehicles: software factories run on a spectrum from L1 to L5.
At the low end, an agent takes a well-scoped ticket and implements it. At L5, the agent identifies the customer issue, writes the spec, designs the change, implements it, and merges it end-to-end without a human in the loop.
Most organizations are somewhere in the middle. Jonathan described the current moment as a semi-autonomous factory, or a dress rehearsal for the fully autonomous one. Verification is moving into the development process itself, and humans are being routed to the changes where their judgment matters. Everything you build into that loop now is what determines whether the next level is safe to attempt.
The context agents are missing
The reason agents produce work that needs heavy review is rarely model quality. It’s that the agent is working from a single repository while your system spans many repositories.
Qodo indexes that broader picture on day one: your organizational rules, your cross-repo dependencies, and, importantly, the contracts between repos, not just the fact that repo A relates to repo B. It also reads PR history, including the discussions between engineers, to learn which issues your team actually cares about. The default lookback is one year or 1,000 pull requests per repo, whichever comes first, and it’s configurable.
Shlomo framed the difference this way. A coding harness gives you context from a repository. This gives you context from an organizational perspective: how your team thinks about software from a governance, rules, and behavioral perspective.
Review, moved into the coding session
The Agentic Toolbox puts those capabilities inside the coding agent as skills, available through a slash command in Claude Code, Codex, Cursor, Kiro, and others. Install is a single command or a direct install from the plugin marketplaces.
Two points came up repeatedly in the demo.
The local review is the full review. It isn’t a prompt wrapped around a model call. The skill routes to Qodo’s backend and runs the same review engine used on your pull requests, with the same multi-model approach, rules, and codebase context. Findings still flow into your usage and acceptance metrics. Nothing is trimmed down for running locally.
Your agent can reason over repos it doesn’t have on disk. Because the underlying infrastructure runs on shallow gits, an agent working in one repository can still search across the rest. For an organization with hundreds or thousands of repos, that closes a real gap. Change a schema in repo A and break fifteen producers downstream, and you find out while planning rather than in production.
Humans and agents need different reviews
One of the sharper points of the session was that review depth and review noise pull in opposite directions. Go deeper, and false positives rise. Suppress false positives, and you surface shallower issues.
The resolution is that humans and agents sit on opposite sides of that tradeoff. An agent can work through a long list of findings without fatigue, so local review should surface everything, including the deep and complex issues. A human reviewer opening a PR needs the opposite: noise reduced, attention pointed at what matters. Tune each surface for its reader.
That extends to how reviews run during generation. Fast reviews are cheap enough to loop while the agent is still writing, similar to an engineer checking their own abstractions mid-task. A balanced review runs once the agent believes it’s finished, before the PR opens.
Developers fix more when review happens locally
A behavioral observation from Jonathan that resonated with the audience: once a change reaches the pull request, developers reliably fix the blockers and reliably skip everything else. Reliability, maintainability, performance, and defensive coding. The improvements that aren’t strictly required tend to get deferred and forgotten.
In the coding session, the calculation changes. The developer is already in the file with the context loaded. Fixing the four blockers is obvious, and adding the quality improvements costs almost nothing more. The result is a codebase that improves at the edges instead of accumulating skipped suggestions.
The PR becomes the final gate
None of this removes review from the pull request. It changes what that review is for.
With most findings resolved before the PR opens, PR review becomes the last gate rather than the first pass. A place to auto-approve low-blast-radius changes, block the ones that need a human, and label the rest so reviewers know where to spend attention. Everything learned in that process, including which findings teams accept and which they dismiss, feeds back into the system on merge.
Try it yourself
Local reviews through the Agentic Toolbox are free through the end of November. No credits consumed.
Install it, run a review on a change you’re working on now, and tell us where it falls short.

document fetched 2026-10-24 10:06 · canonical https://qodo.ai/blog/building-quality-into-the-agentic-workflow