Alignata
Daily DigestShare
A person at a desk checks code on a screen through stacked review layers, beside a small round counter and a stop sign.

Daily Digest

Layered agent code review with a round cap

Verified AI review, risk-scaled approval, and a cap on review rounds.

Daily Digest · Edited by Osbel Morell

Deep dive · · 4 min read

Based on "How the Claude Code team does code review" by Addy Osmani, Morgan Westlee Lunt and Qing Wang, published on claude.dev on 15 October 2026.

The problem

Agents write code faster than people can read it. The authors report that changes landing per day in Anthropic's monorepo went "from about 2,500 in May to more than 30,000 in September". They argue "did a human read the code?" is "an imprecise proxy" for asking which check, automated or human, should catch a given risk. Their goal: most PRs merge "with at most one human in the loop", "with no increase in reverts or incidents".

The technique

1. Prompts and instruction files

The post does not publish its reviewer prompts. What it does describe is two instruction files:

  • CLAUDE.md describes the project: "the stack, the conventions, the goals and the non-goals". Every review loads it, so keep it short and point to other docs.
  • REVIEW.md, written by code owners, tells the reviewer what matters. Good ones "say what failure looks like in that repo", "cap the nits, list paths to skip, and ask for a file and line as evidence", and ask for a summary "that starts with a count".

The Code Review docs confirm that REVIEW.md and CLAUDE.md tune what gets flagged.

OUR INFERENCE: example REVIEW.md (not from the post, 70% that this shape follows its guidance):

# REVIEW.md: payments-service
Failure here means: money moves wrong (double charge, lost refund).
Report: correctness bugs, missing idempotency, unhandled currency/rounding.
Max 3 nits per review. Skip: /generated/, /fixtures/, *.snap
Every finding needs file:line evidence, not a guess from a name.
Start the summary with a count: "N findings (X Important)". If none, say "Nothing to fix."

OUR INFERENCE: example local verify prompt (65%):

Run the test suite, build and linter. Paste the exact commands and their output.
Do not say "done" without output. Then spawn a subagent with fresh context to
review only `git diff main` against CLAUDE.md and REVIEW.md.

2. Setup

  • Give the agent checks it can run (tests, a build, a screenshot to compare against) and ask it "to show the output instead of saying it's done".
  • Run /code-review and /security-review locally, or a fresh-context subagent review.
  • Require CI. Add "Testing", "Risk" and "How to revert" to the PR template.
  • Add an AI reviewer (Code Review for Claude Code or the Claude Code GitHub Action) on one repo, tune REVIEW.md "for a few weeks, then widen". The Action repository exists.
  • Set a review depth per repo: Quick, Standard or Deep. The docs list the same levels: https://code.claude.com/docs/en/code-review.
  • Add automated security review and block merges while any thread is open. The open-source security-review Action exists.
  • Risk tiers: Low needs no human once checks are green, Medium needs one person, High needs two. "The change sets the tier, not the author". Use CODEOWNERS and branch protection so auth, billing, CI/release workflows and installers always need a person. A bot approval never replaces a required human's, and approvals are dismissed on new commits.

The post says a classifier assigns tiers, but it gives no implementation. Without one, path-based rules in CODEOWNERS are a reasonable starting point.

3. The review loop

Several agents hunt for bugs in parallel, then a separate group votes on each finding, and only findings enough of them agree on get posted. Findings about style alone are dropped, and the rest are tagged Important, Nit or Pre-existing. Every finding gets fixed, or its thread is resolved with a reason. Convergence rules:

  • "After three rounds, the reviewer asks the author to step back and reconsider the approach."
  • Diff-only re-review after the first few passes.
  • Flag the class of bug so every instance gets fixed.
  • Never repeat findings already posted.

If you build this yourself, the post gives no vote threshold or agent count. Pick values and tune them against your eval set.

4. Verification

  • Turn incidents and reverts into an eval set, with each bug confirmed by experts. Rescore the reviewer whenever its prompts, tools or model change, and keep a change only if the score goes up.
  • Measure noise by sorting every comment by the author's response: fixed, wrong, intentional or didn't matter. Also run the reviewer on known-clean PRs.
  • Track approvals per PR (median and 95th percentile), human comments, wait time and rounds-to-pass. Revert and incident rates are the guardrail.

5. Failure modes

  • Infinite loops: one CLI PR went through "more than fifty rounds" overnight.
  • Stale rediscovery: about 59% of repeat flags were code present since the first pass.
  • Noise is costlier with agent authors: "An agent fixes it, which means another commit and another round".
  • Recurring flags: fix them at the source, in a skill, a lint rule or CLAUDE.md.

What we saw

These are the authors' figures. We ran no tests of our own.

  • From June to September 2026, merged PRs to Claude Code "more than doubled" while the revert share "fell by more than 40 percent". This is a correlation across all their layers, not proof that AI review caused it.
  • At the research-preview launch, the share of internal PRs with substantive review comments rose "from 16% to 54%".
  • After the convergence changes: passing after one round rose from 65% to 78%, PRs needing five or more rounds fell from 8.9% to 1.5%, and mean reviews to pass fell from 2.08 to 1.32.
  • The authors say Boris Cherny estimates verification makes the final result two to three times better. This is an estimate, not a measurement.

Limits

  • No prompts, vote thresholds or classifier details are published. Our examples are inference.
  • The team has no circuit breakers yet that tighten approval when revert rates move, and engineers "want them before more code moves to zero-human merges".
  • Zero-human merges rely on flags, staged releases, internal use and on-call. Teams without fast rollback should not copy the Low tier.
  • The figures come from one team, measured by the vendor itself.
  • Code Review is available on Team and Enterprise plans. Pricing and limits for other plans are UNVERIFIED.

Try it

  1. This week, write a 6-line REVIEW.md and a short CLAUDE.md for one repo.
  2. Require your agent to paste test and build output, and to run a fresh-context subagent review before opening PRs.
  3. Turn on an AI reviewer for that repo only. Cap re-review at three rounds and make later rounds diff-only.
  4. Collect your last ten reverts as an eval set and score the reviewer against it.
  5. Track rounds-to-pass and reverts for a few weeks before you loosen any human approval.

Sources

Found this useful? Send it to someone.

Share