A coding agent can turn a ticket into a pull request before your coffee cools. Reading that pull request properly still takes as long as it always did, and on a small team, where the same two or three people write, review, and deploy, that gap is the whole problem. Generation got cheap; verification didn’t, and verification now decides whether agents make a team faster or just make it busier.
This is the AI coding agent verification workflow my team uses to keep agent-written code shippable. None of it is clever, and little of it is new: it’s old engineering discipline, applied earlier in the process and enforced more strictly than before, because the volume of code arriving demands it. I’ll walk you through what the research says, the six steps, and a one-page checklist you can hand your team on Monday.
Disclosure: I own CodeFlamme, a software development agency that uses AI coding agents in client work. No tools or vendors mentioned in this article paid for it.
The Evidence Says Speed Isn’t the Problem
Before deciding what to fix, it’s worth looking at what the research says.
- Developers don’t trust the output, and they’re right to be careful. In Stack Overflow’s 2025 Developer Survey, 46% of respondents said they distrust the accuracy of AI tools, compared with 33% who trust it. The top frustration, cited by 66%, was “AI solutions that are almost right, but not quite,” and 45% said debugging AI-generated code takes more time.
- Feeling faster isn’t the same as being faster. In METR’s 2025 randomized controlled trial, 16 experienced open-source developers took 19% longer on real tasks when AI tools were allowed, yet believed afterward that AI had sped them up by 20%. METR frames this as a snapshot of early-2025 tools on mature codebases rather than a permanent verdict, but the perception gap is the part worth keeping.
- Delivery stability takes the hit. Google’s DORA 2024 Accelerate State of DevOps report estimated that every 25% increase in AI adoption came with a 7.2% drop in delivery stability. DORA’s own explanation is that the fundamentals (small batches, solid testing) still decide outcomes, AI or not.
- The newest data points the same way, and most teams are improvising. Stack Overflow’s 2026 survey, as reported by IT Brief, found that nearly half of developers trust AI output when they can verify it, and only 6.6% would trust it with an important decision. Just 24% said their employer had approved tools or formal AI guidance, even though 26.2% already use automated agent workflows.
Put those together, and the diagnosis is clear. “Almost right” code shows up quickly, feels like progress, and costs you later in review, debugging, and incidents. Better prompts won’t fix that; a verification workflow that keeps pace with the agent will.
One caution: METR and DORA measured earlier generations of tools, so treat their exact percentages as directional. The 2026 numbers suggest the underlying trust gap hasn’t closed.
The New Trap: “It Compiles and the Demo Works”
Agents are very good at producing code that runs. They’re much less reliable at producing code that’s correct for your system: your tenancy rules, your billing edge cases, the migration that has to be reversible, the endpoint a mobile client still calls with the old field name.
So the classic signals of “done” (a green build, a working click-through) tell you less than they used to. An agent will happily make both true while quietly changing behavior nobody asked it to touch, whether it’s a general-purpose harness like OpenClaw or a vendor-specific agent like Devin, which we cover in our Devin Desktop review. The fix is to decide, before the agent starts, what “done” means in terms a machine can check.
Here’s the map before the detail: six steps, each aimed at a different way agent-written code goes wrong.
Step | What It Catches | Who or What Runs It |
|---|---|---|
1. Tests first | Misread requirements, before any implementation exists | Human reviews the tests; CI enforces them |
2. Contract and integration tests | API drift and broken critical flows at the system’s edges | CI |
3. Preview environments and screenshot diffs | Unintended UI changes nobody spots in a code diff | CI, plus a quick human look |
4. Risk checklist | Secrets, access control, data deletion, migrations, dependencies, oversized diffs | Human reviewer |
5. Review budget | Queue overload and rubber-stamp reviews | Team process |
6. Ownership and boundaries | Lock-in, lost history, runaway agent actions | Team policy and permissions |
Step 1: Tests Before Code (Make the Agent Prove It)
The single most useful change we made was reversing the order of work. Instead of “implement X,” the first instruction is “write or extend the tests that describe X, and stop.” A human reads those tests; only then does the agent write the implementation, and the job is finished when those tests pass, and nothing else breaks.
This works for three reasons. Reviewing a test is faster than reviewing an implementation, since you’re checking intent rather than mechanics, and it catches misunderstandings while they’re still cheap: a test that asserts the wrong behavior costs you five minutes, not a day. It also gives the agent a target it can’t argue with, because agents are at their best against a tight, executable spec.
A practical rule follows: if the agent changes an existing test to make its own code pass, that diff gets flagged for a human every time. Sometimes the old test was wrong. Often it wasn’t.
Step 2: Contract and Integration Tests at the Edges
Unit tests protect the inside of a module. Agents tend to break the outside: an API response shape, a queue message format, a webhook payload. Two more layers help here.
- Contract tests for every API other people depend on. If you publish an OpenAPI spec, run a check in CI that fails when the implementation drifts from it, and keep consumer-side tests for the endpoints your frontend or mobile apps actually use.
- A thin set of integration tests against real dependencies. Use a real database in a container, not a mock, for the flows that matter most: sign-up, payment, permission checks, data export.
You don’t need hundreds of these. You need the ten or twenty that would wake you up at night if they broke.
Step 3: Preview Environments and Screenshot Diffs for UI
UI changes are where “looks fine to me” does the most damage, because nobody reviews pixels in a diff. Every pull request, human or agent, should get a preview deployment with its own URL.
Pair that with visual regression checks: capture screenshots of key pages and states on the main branch, capture them again on the PR, and flag differences. Open-source tools like Playwright can handle the capture and comparison in a few lines of config. The aim isn’t to block every pixel change; it’s to make unexpected ones visible.
When an agent fixes a button on the settings page and the pricing table on another page shifts by 40 pixels, you want to know before your customers do.
Step 4: A Human Checklist for the Risky Categories
Automation catches regressions; it’s weak at judgment. So we keep a short review checklist and apply it to any PR, whoever wrote it, that touches these areas:
- Secrets and Config: No new credentials in code, logs, or test fixtures, and no widened permissions “just to make it work.”
- Auth and Access Control: Every new endpoint or query is checked for who can call it and whose data it returns; multi-tenant filters are verified, not assumed.
- Data Deletion and Retention: Anything that deletes, overwrites, or bulk-updates data gets a second reviewer and, ideally, a dry-run mode.
- Migrations: Reversible, tested on a copy of realistic data, and safe to run while the old version of the app is still serving traffic.
- Dependencies: New packages are real, maintained, and actually needed; agents sometimes suggest packages that don’t exist or aren’t the ones you think they are.
- Size: If the diff touches more than a handful of files across unrelated areas, send it back and ask for smaller slices.
That last point matters more than it sounds. DORA has said for years that small batches are safer, and agents make large batches effortless. A team has to enforce small ones on purpose.
Step 5: Budget Review Time Like It’s a Resource (Because It Is)
Here’s the uncomfortable math: if agents double the number of pull requests and review capacity stays flat, either review quality drops or the queue grows. Usually both. Three habits help.
- Cap work in progress. Don’t let agents open more PRs than the team can review that day, since a queue of unreviewed agent PRs is inventory, not output.
- Spend review time where the risk is. A copy change and an auth change shouldn’t get the same attention, so label PRs by risk and route them accordingly.
- Make the agent do the boring prep. Ask for a PR description that lists what changed, what was tested, and what it didn’t test, then check that description against the diff instead of trusting it.
Step 6: Keep Ownership Where It Belongs
The last piece is less about code and more about control. The repo, the CI pipeline, and the cloud accounts should belong to the team that owns the product, not to a tool, a vendor, or one engineer’s personal account. The same goes for the record of how the code got that way: the specs, the tests, the PR discussions, the decisions.
If that history only lives inside an agent session or a chat log nobody can export, the next person to touch the code starts from zero.
Agents also need boundaries. Decide what they can read and run, and remember that protocols like MCP, which standardize how agents connect to tools and data, make those connections explicit but don’t set your permissions for you; our explainer on the Model Context Protocol covers the basics. Those limits still come from how you configure the host and what credentials you hand over.
Production credentials don’t belong in an agent’s environment. Any agent action that can spend money, send messages, or delete data should require a human approval step.
A One-Page Summary for Your Team
If you only take one thing from this, take this list:
- Tests first, reviewed by a human, before the agent writes implementation code.
- Changes to existing tests are always flagged for review.
- Contract tests on every API someone else depends on, failing CI on drift.
- Integration tests against real dependencies for the critical flows.
- A preview environment and screenshot diff on every UI pull request.
- A human checklist for secrets, auth, data deletion, migrations, and new dependencies.
- Small PRs, enforced. Work-in-progress limits on agent output.
- Repo, CI, cloud, and history owned by the team, not the tool.
FAQs
It’s the set of checks that decides whether agent-written code is safe to merge: tests written and reviewed before implementation, contract and integration tests at the edges, visual diffs for UI, and a human checklist for risky changes.
It depends on what you measure. METR’s 2025 trial found experienced developers took 19% longer with AI tools while believing they were faster, and DORA’s 2024 report linked higher AI adoption to lower delivery stability. Both studied earlier tools, so the sensible reading is that speed gains only count once review keeps up.
Have the agent draft them, but make a human read them before any implementation exists. Reviewing tests checks intent, which is faster than checking mechanics, and a wrong assertion costs minutes instead of days.
Fewer than you’d think. Aim for ten or twenty covering flows that would wake you up at night if they broke, such as sign-up, payment, permission checks, and data export, and run them against real dependencies rather than mocks.
No. Production credentials stay out of the agent’s environment, and any action that can spend money, send messages, or delete data should wait for human approval.
Yes. We apply the risk checklist to any pull request that touches those areas, whoever wrote it. Agents just make skipping it more expensive.
Check Your Bearings Before You Ship
The cost of skipping verification has changed shape. Before agents, a missing review cost a team a slow week; now it can cost a fast month of “almost right” code that someone has to unwind later, usually under worse conditions than if the review had happened up front. The teams that get real value from agents won’t be the ones that generate the most code; they’ll be the ones that can verify it at the speed it arrives.
So start small. This week, pick one repo and apply Steps 1 and 4: tests first, reviewed by a human, plus the risk checklist on every PR, then add the rest as your queue shows you where the pain is. Verification capacity, not the agent itself, is the bottleneck worth investing in, and for more on how AI tools actually behave under the hood, and where that behavior should and shouldn’t be trusted, our AI Unboxed section is the natural next stop.
Speed gets you to the pull request; judgment gets you to production, so keep sharpening yours at YourTechCompass.com.
About the Author: Umair Abbas is the founder of CodeFlamme, a senior-only software development agency in Lahore, Pakistan, that builds web, mobile, and AI products for startups and growing companies. He writes about engineering ownership, clean handoffs, and getting AI features to production. Find him on LinkedIn.
