James Duffy

Rewriting Our Pull Request Process for AI-Written Code

A vast majority of the pull requests and commits on my team are now generated by AI. I don’t see that as a problem, but our pull request process was built for a time when a person wrote every change, and I wanted to start treating it differently.

The old process assumed the person who opened the PR wrote the change and could explain it. That isn’t always true now. The reviewer needs different information, and the human who opens the PR still needs to own what is in it.

We have started rolling out a few changes, and the first one is the pull request template.

#Pointing the agents at the template

A template only helps if it gets used, and these days that mostly means the agents have to use it. The first thing I did was update our AGENTS.md and CLAUDE.md files so that any agent opening a pull request fills out the template instead of making up its own format. Every PR now has the same shape, which means I can find what I’m looking for without reading the whole description.

#The template

Here is what we ended up with:

 1## ๐Ÿ“‹ Issue
 2
 3<!-- Link the JIRA ticket this PR resolves. One line, no extra commentary. -->
 4[TICKET-XXX](https://example.atlassian.net/browse/TICKET-XXX)
 5
 6## ๐Ÿ“– Context
 7
 8<!-- Plain sentences, no tables or lists (this is what Slack previews show). Say what this touches and which environment (dev, staging, prod, global). Then the why: the problem being solved or the capability enabled. A few sentences, not a restatement of the diff. -->
 9
10## ๐Ÿ“ Changes Made
11
12<!-- What changed, in plain language. One bullet per logical change, not per file. Don't repeat Context. Name the specific roots, modules or workflows touched, and anything that behaves differently after merge. -->
13-
14
15## ๐Ÿ”— Dependencies
16
17<!-- Other PRs, repos, or pinned version bumps this depends on or blocks. Write "None" if there are none. -->
18-
19
20## ๐Ÿงช Testing
21
22<!-- Evidence this is safe to merge: terraform plan output for the affected root(s) and/or CI results. Merging to main applies automatically, so plan output is the main evidence. Don't claim manual testing beyond what you ran. -->
23
24## ๐Ÿค– Authorship
25
26<!-- One line: which agent wrote this, which harness, and which human reviewed the diff before it was opened. Write "Human only" if no agent was involved. -->
27Authored by: model + version via harness + version. Reviewed by: name
28
29<details>
30<summary>Prompt and transcript (optional)</summary>
31
32<!-- Include when an agent wrote most of the change: the prompt that started the session and the key parts of the transcript (decisions, corrections, dead ends). Redact secrets, account IDs and customer/patient data. -->
33
34</details>

The HTML comments are instructions for the agent as much as for the human. The parts I care about most are the last two sections. Authorship says which model and harness wrote the change and which human reviewed the diff before it was opened. The prompt and transcript show what the agent was asked to do and where it went wrong along the way, so when a PR turns out to be bad I can tell whether the prompt was vague, the agent misread it, or our AGENTS.md needs to change. They go in a collapsed block so they stay out of the way, and the template reminds people to redact secrets, account IDs, and customer or patient data.

#One PR, one thing

The second change is about size. I tell the agents that PRs should be extremely focused and solve only one thing, which means I end up with a lot more of them.

This has always been good advice, but AI makes it matter more. An agent can produce a thousand-line diff in a few minutes, and a person can’t meaningfully review that in a few minutes. If writing code gets faster and reviewing stays the same speed, review is what gives, and people start skimming and approving.

It is much easier to review five small, focused pull requests than one large one. Each can be read start to finish, the reviewer only has to decide on a single idea, and if one of the five is wrong it doesn’t hold up the other four.

It also helps the template, since a short Context section and a short list of changes only work when the PR does one thing. And because merging to main applies automatically, a small PR is easier to roll back and easier to reason about when a plan looks wrong.

#Where this is going

I’m not trying to slow anyone down or add ceremony. I just want us to be honest about how the work gets done now. If most of our code is written by agents, then the pull request is where a human takes responsibility for it, and the process should make that easy to see.

It is still early and we are adjusting as we go.