loading
loading
Engineers write tests for the logic that worries them and let the small details slide. One of our commerce engineering teams now has a scheduled agent that finds the thinly tested files, writes the missing tests, and opens a pull request for review.
Nobody decides to skip a unit test. The core logic gets covered, the small details don’t make the cut, and most days that’s a fair trade. Every so often it isn’t, and the test nobody wrote is the one that would have caught a problem before it shipped.
We’d already made test generation easier for developers. A one-click local command could ask an AI coding agent to write tests for the file open in the editor. It was useful, but it still depended on someone noticing the gap and choosing to run it.
We handed that gap to an agent. A scheduled pipeline finds the files with low coverage, writes tests for what’s missing, and raises a pull request. The repository was new and coverage went from about 70% to 82% in about a month.
We started with one file, not the whole repository
“Improve test coverage” sounds like a goal, but it is a poor task description. It leaves too many decisions open: which code matters, how much to change, what counts as good coverage, and when to stop.
We gave our Agentic Pipeline a much smaller job. On each run, it selected one TypeScript or React file with no test file or with coverage below the target, wrote tests for that file, and checked the result against the repository’s normal development rules.
That narrow scope was important. A reviewer could understand the change without wading through a large generated diff, and a failed run affected one test target rather than a broad sweep of the codebase.
The first version asked the agent to cover the selected file’s statements, branches, and functions with tests that asserted real behavior: edge cases, error paths, and the regressions we’d want caught, not just lines that happened to run. Once the workflow had run in practice and the tests kept holding up in review, we raised the per-file target. Raising it was a one-line change to the prompt file, reviewed like any other PR.
How the scheduled workflow worked
The implementation used a custom Bitbucket Pipeline and a prompt stored in the repository. A simplified configuration looks like this:
definitions:
agents:
test-coverage-maintainer:
provider: rovodev # or claude, codex
prompt: "!agent-prompts/improve-test-coverage.md"
pipelines:
custom:
improve-test-coverage:
- step:
name: Add one focused set of tests
auth:
system:
scopes:
- write:repository:bitbucket
- read:pullrequest:bitbucket
- write:pullrequest:bitbucket
script:
- npm ci
- npm run build
- agent: test-coverage-maintainer
The provider line is the only part that depends on your agent. Agentic Pipelines currently supports rovodev (the default if you leave it out), claude, and codex. The schedule, auth scopes, and prompt file stay the same whichever one you pick.
We ran it from a pipeline schedule every weekday morning, so PRs landed while people were around to review them.
A sketch of the prompt, for a repository that already knows how to measure coverage:
# Add the unit tests we skipped
Run the coverage command this repo already uses. Read the report.
Pick one source file below the per-file threshold in this file. Work on that one file only.
If none is below it, stop. Do not open a pull request.
Otherwise add unit tests for uncovered behavior, matching the style of
the tests already here. Leave production code unchanged.
Run the repo's style check, lint, and test suite.
If they fail and you cannot fix the tests, stop without a pull request.
When they pass, open one pull request. Say which code was uncovered
and what the new tests assert.
The real flow had five parts:
- Prepare the repository. The pipeline completed the normal build setup before starting the agent. That gave the agent a working environment instead of asking it to rediscover installation steps on every run.
- Choose one gap. The prompt directed the agent to select one source file with no tests, or with coverage below the target. It did not attempt to refactor the application or improve coverage everywhere at once.
- Write and verify the tests. The agent iterated until the tests passed and met the per-file coverage target. It also ran the repository’s type checking and linting rules.
- Check the boundaries. The prompt prohibited new dependencies and business-logic changes. It asked the agent to reuse existing types, avoid lint suppressions, and remove mocks that were not needed.
- Open a pull request. When the checks passed, the agent created a branch and a pull request with a predictable title, the relevant tracking work item, and the reviewers who needed to make the final call.
Running it while people were at their desks made the first few weeks feel less like unattended automation and more like onboarding a new teammate. Engineers could watch the pull requests arrive, spot weak patterns, and improve the instructions while the context was fresh.
Coverage improved through small, reviewable changes
Over about four weeks, reviewers merged around 34 agent-opened PRs and repository-level coverage went from about 70% to 82%.
That many PRs could easily wear reviewers down, and we designed the workflow so it wouldn’t. Each run opened at most one pull request, scoped to a single file, so each one was small enough to review properly in one sitting. Every PR had a predictable title, a linked work item, and a description saying which code was uncovered and what the new tests asserted, so reviewers knew what they were checking before they opened the diff. If nothing was below the threshold, the agent stopped without opening a PR, so nobody got noise. And when reviewers flagged the same problem twice, we fixed the prompt once instead of leaving the same comment on every PR after it.
Early pull requests added focused test files for individual components. Later runs also expanded existing tests when a component still had uncovered behavior.
The pull request was the useful unit of work. Engineers could see which behavior the tests exercised, question an unnecessary mock, reject a brittle assertion, or ask for a better case before merging. Passing checks were necessary, but they were not a substitute for review.
A test can raise a coverage number while saying very little about the behavior people rely on. Keeping the changes small gave reviewers a realistic chance to judge both.
The prompt became part of the test system
Getting an agent to write Jest tests was the easy part. The prompt decided whether the tests were worth keeping.
The prompt captured details that a developer would normally apply without writing them down: where tests belonged, which commands to run, how imports were structured, which shortcuts were unacceptable, and what the agent was allowed to change in production code.
Those rules evolved after real pull requests. Early PRs mocked modules the component never imported. The tests passed and coverage went up, but the mocks were just noise. We added one line to the prompt: delete any mock the test still passes without. It hasn’t come back.
We tightened the coverage target, made type and lint validation explicit, and kept narrowing the work to one reviewable change.
Change a convention, update the prompt in the same PR, and the next run follows it.
What we’d tell another team
If you try this in your own repository, start smaller than you think.
- Give the agent a small job with a clear finish line. “One source file below coverage” is a task; “improve quality” isn’t. One bounded change per run keeps the diff small enough for a person to review properly.
- Let your repository define “done.” Run the same tests, coverage, type checks, and lint your engineers run. Write your local conventions, like where tests live, mocking rules, and what production code is off-limits, into the prompt beside the code. That’s what turns a general coding agent into a useful contributor to your repo.
- Keep people on the merge and on the prompt. Engineers decide whether each test is worth keeping. When the same weak pattern shows up twice, fix the prompt instead of accepting it as normal agent behavior.
The agent didn’t take testing away from us. It took away the start-up work: finding the next gap, writing the first test file, and putting together a change someone could review.
Try it now with Agentic Pipelines
Test coverage suits this treatment because a number tells the pipeline where to look. If you have a recurring, bounded job like that, plus checks that already define “done” in your repository, it’s a good candidate. Pick a repository where you already track coverage, run the pipeline once by hand, and read the first pull request.
Agentic Pipelines is in beta on Bitbucket Cloud. To turn it on, go to Workspace settings, open AI, and choose a provider. Rovo Dev requires a paid Bitbucket Cloud plan with Pipelines enabled, plus a Rovo Dev Standard subscription. Claude and Codex use an account your organization already has, and that provider bills for the tokens. With Claude or Codex, your code is processed under your organization’s agreement with that provider; see how Agentic Pipelines handles data.
We use the same pattern, a narrowly scoped agent with a person checking the result, for code reviews, vulnerability fixes, stale feature flag cleanup and documentation updates.
Start with one agent. For setup, variables, MCP tool scopes, and example agents, see:
- Agentic Pipelines documentation
- Step-by-step guide
- Interacting with Bitbucket via MCP
- Example Agents
One file. One pull request. Then learn from the result.document fetched 2026-10-24 11:26 · canonical https://atlassian.com/blog/bitbucket/using-agentic-pipelines-to-turn-test-coverage-into-daily-habit