Skip to content

A Tech Lead's AI Pull-Request Review Checklist

Ask a chatbot for a PR review checklist and it generates a fresh, generic one each time: check naming, check tests, check edge cases. That list is reasonable and forgettable, and it is regenerated differently on the next prompt, so it never becomes the standard your team reviews against. A useful checklist is a vetted artifact you copy once and reuse, with the judgment behind each item written down, because reviewing AI-generated code fails in different places than reviewing a human's.

How is reviewing an AI PR different

A human author has a mental model of the change and can answer "why did you do it this way." An AI-generated diff has no such author behind it, so two specific risks rise. First, the code is fluent — it reads well, names things sensibly, and looks reviewed already, which makes a skim feel sufficient when it is not. Second, the failure mode is plausible-but-wrong logic that the diff actively hides: a swapped condition, an off-by-one, a dropped error path that the surrounding clean code makes invisible. Reviewing AI code well means reading for correctness on the critical path rather than for style, and treating "it looks fine" as the start of the review rather than the end.

What should I check in an AI-generated PR

The procedure below is ordered: a failure at an early step is a hard stop, so you do not spend attention on later steps for a PR that should not merge.

  1. Did CI stay green without being weakened? Open the diff for the test config, the CI workflow, and any # type: ignore / eslint-disable / skipped-test markers. An agent under pressure to make checks pass will sometimes delete or skip the check rather than fix the code. Any weakening of CI or a test is a hard stop.
  2. Trace the critical path by hand. Pick the one path that carries the change's intent and read it line by line — inputs, the branch that does the work, the error path, the output. This is the step a skim skips and the step where plausible-but-wrong logic lives.
  3. Do the tests assert behavior, or do they assert the implementation? A test that re-states what the code does passes by construction and proves nothing. Confirm at least one test would fail if the behavior regressed.
  4. Is the change scoped to what the PR claims? AI diffs often include unrequested refactors, renamed symbols, or a reformatted file. Each unrelated change is surface area you now have to review and a place a regression can hide.
  5. Are the failure and edge paths handled beyond the happy path? Empty input, the upstream call that errors, the value that is null. Generated code is reliably strong on the happy path and thin here.

What are the red flags

These are the patterns that should slow you down regardless of how clean the diff reads.

  • A test file changed in the same PR as the behavior it covers, with the test now matching the new behavior. Confirm the test still fails against the old code.
  • A check turned off: a deleted assertion, a skip, a loosened type, a lowered coverage threshold.
  • A comment that explains intent the code does not implement — the agent described the right thing and wrote something else.
  • A diff much larger than the task warranted, or one that touches files the task never mentioned.
  • Confident-sounding handling of a case you cannot find a test for.

How do I review AI PRs at scale

The point of writing the checklist down is that it stops being something each reviewer reinvents. Copy the list above into your repository — a PULL_REQUEST_TEMPLATE.md or a CONTRIBUTING section — so every PR is reviewed against the same bar, and so the items that are mechanical can move into CI instead of into a human's attention. The skim-feels-enough trap is not a discipline problem you can fix by trying harder; the METR study found experienced developers were about 19% slower (METR) with AI tools while they felt faster, which is what an unverified "looks fine" loop produces. The durable fix is to make the load-bearing items unskippable. A failing test makes correctness a fact rather than a judgment call, and a gate makes that check run whether or not the reviewer remembered it. That is the subject of Module 3, whose lab has you write a red test for a real bug in Triage, drive the self-correcting loop to green, and install a pre-commit gate that refuses the bug if it comes back. Start by copying the checklist into one repository and moving its first item into CI.