Round Trip Gate for Reviewing Agent-Written Patches
A round-trip gate is a review workflow that sends an agent-written patch through independent checks, returns actionable feedback to the agent, and requires a fresh review before the patch is accepted. It is useful when agents can edit code but should not be trusted to approve their own work: the gate combines automated tests and policy checks with human review for consequential changes.
What a round-trip gate means
“Round trip” describes the feedback loop, not one standardized product or protocol. An agent proposes a diff; a gate evaluates it; failures or review comments go back to the agent; the agent revises the patch; and the updated diff is checked again. The gate is the boundary that decides whether the change may proceed to merge or another controlled step.
This is different from asking an agent to inspect its own code once. A self-review can catch omissions, but the same model may repeat its original assumptions. A meaningful gate applies checks that are separate from the patch-writing step, records their results, and prevents unresolved requirements from being silently treated as satisfied.
How the review loop works
- Define the task and constraints. State the intended behavior, affected areas, tests expected, and restrictions such as “do not change public interfaces.” Keep the task narrow enough to evaluate.
- Generate a patch in an isolated branch or workspace. The agent should provide a diff and a concise account of what changed. Avoid granting write access to unrelated repositories, credentials, or production systems.
- Run deterministic checks. Build, tests, formatting, static analysis, and repository-specific policy checks can identify reproducible failures. Use the same checks expected for ordinary contributions where practical.
- Review the actual diff. A human reviewer, or a separate review process followed by a human where needed, checks whether the patch meets the task, handles edge cases, and changes only what is justified.
- Return specific findings. Give the agent precise locations and expected outcomes, not just “fix the bug.” For example: “When the input list is empty, this function should return an empty result; add a test for that case.”
- Recheck the revised patch. New edits can introduce regressions or invalidate earlier checks. Run the relevant checks again and review the final diff, not merely the agent’s summary.
- Accept, revise, or stop. Merge only after required checks and approvals pass. If the agent cannot resolve a finding safely, stop the loop and hand the issue to a person.
What belongs in the gate
A useful gate separates objective checks from judgment. Tests can establish that specified cases pass; they cannot prove that the feature is the right one or that every important case was specified. Code review can assess design and intent, but should not replace repeatable checks.
- Patch scope: Confirm that changed files and generated artifacts are expected. Flag unexplained edits, dependency changes, or broad formatting churn.
- Correctness checks: Run focused tests first, then the relevant wider suite. A passing test suite is evidence, not a guarantee of correctness.
- Quality and compatibility: Check formatting, type or static analysis where used, API compatibility, migrations, and documentation requirements relevant to the repository.
- Security-sensitive review: Pay particular attention to authentication, authorization, input handling, secrets, dependency changes, and data access. Automated scanning may help, but cannot replace a qualified review for high-impact changes.
- Traceability: Preserve the task, patch revision, check results, and reviewer decisions so it is clear what was approved.
For example, if an agent changes a date parser, the gate might run parser tests, add cases for invalid and boundary dates, and ask a reviewer to verify the intended timezone behavior. If the agent then changes the parser again to fix a failing case, the tests must run on that new revision. A green result from the earlier patch does not certify the later one.
Choosing a practical design
Start with the risk and the cost of a mistake, rather than trying to automate every part of review. A small documentation correction may need a diff check and a quick human glance. A change to permissions, data deletion, or a public interface deserves stricter checks and an explicit accountable reviewer.
- Set clear pass conditions. Decide which checks are mandatory, which findings block approval, and who can waive a check. Avoid vague criteria such as “looks good.”
- Keep review feedback actionable. Tie comments to lines or behaviors, explain the expected result, and distinguish required changes from suggestions.
- Limit repeated attempts. A bounded number of repair cycles helps prevent an agent from making endless, increasingly broad edits. Escalate unresolved issues instead of weakening the gate.
- Keep a person responsible. Assign an owner for approving changes, especially when consequences are difficult to reverse. An agent’s confidence or a successful run is not approval.
- Make the process visible. Show the exact patch revision and check status to reviewers. Do not let a new commit inherit approval intended for an older diff without revalidation.
There is no universal number of review rounds. One may be enough for a simple, well-specified patch; ambiguous tasks can require clarification before implementation even begins. More rounds are not automatically safer if feedback is inconsistent or the agent is allowed to alter the tests merely to make them pass.
Common failure modes and safeguards
A common misconception is that “all tests pass” means a patch is correct. Tests only cover the cases they contain, and an agent can produce code that satisfies existing tests while missing the requested behavior. Reviewers should ask whether the tests exercise the important requirements, not just whether the test command ended successfully.
- Reviewing a summary instead of the diff: Summaries can omit important details. Inspect the source changes and generated files directly.
- Stale approvals: Any material change after review should trigger the relevant checks and a fresh look at the updated diff.
- Overbroad permissions: Use an isolated workspace and the minimum access needed. Do not expose secrets to an agent or allow routine patch generation to deploy directly.
- Untrusted repository instructions: Treat text found in issues, source files, or external documents as data to evaluate, not as authority to override the task or review rules.
- Weak or altered tests: Check that tests reflect the requirement and were not simply deleted, loosened, or rewritten to conceal a failure.
- Endless repair loops: If the agent repeatedly changes direction or creates new failures, pause and have a person clarify the requirement.
Where it fits compared with ordinary code review
A round-trip gate does not replace a pull request, continuous integration, or code ownership. It organizes them around an agent’s ability to respond to feedback. Ordinary review may be a single human-led decision; a round-trip workflow makes the revision-and-recheck cycle explicit. Continuous integration supplies automated evidence, while the gate decides what evidence and approvals are required before proceeding.
It is also not the same as automatically merging any patch produced by an agent. Automation can safely handle mechanical checks and report results, but acceptance still depends on the repository’s risk policy and accountable reviewers. For low-risk, well-tested changes, teams may permit more automation; higher-impact changes warrant stricter human oversight.
Frequently Asked Questions
Does a round-trip gate mean the agent approves its own patch?
No. The agent can use review findings to revise its patch, but independent checks and the designated reviewer determine whether it passes.
Should every agent-written patch get human review?
That depends on the change and the organization’s policy. Human review is especially important for security-sensitive, externally visible, destructive, or otherwise high-impact changes. Automated checks alone do not establish that the change is appropriate.
What happens when the agent cannot fix a finding?
Stop the loop, preserve the failing revision and feedback, and assign the issue to a person. Do not waive a blocking requirement merely to make the workflow finish.
Do tests need to run after every revision?
Run the relevant checks on the final revision before acceptance. Focused checks can run during iteration, but earlier results do not cover code changed afterward.
Can the gate be fully automated?
Some parts can be automated, including builds, tests, formatting, and policy checks. Whether a patch satisfies intent, handles untested cases, or is safe to accept may require human judgment.
Related
Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.