Ten thousand files. Nine thousand are a codemod.

Run an agent over ten thousand files and every single one becomes non-deterministic. Two runs produce different diffs. A file that was correct on Tuesday gets rewritten differently on Thursday. Nothing is reviewable in bulk, because no two changes are textually identical even where the transformation was identical. And you have paid a per-file inference cost for nine thousand transformations that a syntax tree would have done perfectly, for free, in eleven seconds.

The opposite failure is just as common. A team writes a codemod, it handles eighty percent, and the remaining two thousand sites become a spreadsheet that three engineers grind through for a quarter. That is the work agents are genuinely good at, and it is being done by hand because the codemod was framed as the whole solution rather than the first stage.

ALL AGENT      10,412 units · non-deterministic · unreviewable in bulk
                pays inference for 9,100 transforms an AST does for free

ALL CODEMOD    9,100 done in one diff · 1,312 left in a spreadsheet
                a quarter of three engineers grinding the tail by hand

SPLIT          codemod: 9,100 in one reviewable diff, deterministic
                agent:   1,312 units, one decision each, individually reviewed
                the expensive tool only touches the expensive problem

Before any transform, write a script that classifies every call site into a shape. Not a sample: all of them. This costs an afternoon and it is the difference between a plan and a hope.

What the census actually tells you

• Shape D is not a migration. Every one of those tolerance comparisons exists because somebody was papering over float error. In integer cents the tolerance is meaningless, and each site is a question: was this hiding a real discrepancy? Some of them were. That is a bug hunt wearing a migration's clothes.
• Shape C is a policy question first. Where exactly does rounding happen, and in whose favour? Until a human answers that, no tool of any kind can do the work correctly. This is the brick-coloured category and it is what actually sinks migrations.
• The ratio is the plan. 87/13 says write the codemod. If the census had come back 40/60 the whole approach would be wrong, and you would want to know that before building anything.

def transform(node, ctx):
    if not is_money_expr(node):
        return None                      # not ours

    if has_tolerance_compare(node) or crosses_boundary(node, ctx):
        ctx.refuse(node, reason="shape-C" if ... else "shape-D")
        return None                      # hand to the agent, do not guess

    return to_cents(node)                # one correct output

A codemod that tries hard produces the worst outcome available: a diff where most changes are provably correct and a few are silently wrong, mixed together, at a scale nobody can review. A codemod that refuses cleanly produces two artifacts, one of which you can review in bulk and one of which is a worklist.

{"path":"billing/invoice.py","line":211,"reason":"shape-C",
 "expr":"round(subtotal * tax_rate, 2)","sha":"6c1a2e0"}
{"path":"billing/recon.py","line":88,"reason":"shape-D",
 "expr":"abs(paid - owed) < 0.01","sha":"91be004"}

1. Scope One file. Not one module, not one "area". A unit that touches four files cannot be reverted without considering the other three.
2. Input The refusal record, the file, the policy decision from §1, and nothing else. No repository-wide context: it is not needed and it is where scope creep enters.
3. Output One commit, plus a one-line justification naming which policy branch it applied. That line is what makes 1,312 reviews survivable.
4. Escape hatch The unit may return UNCLEAR with a question. Roughly one in twenty will, and those are the genuinely interesting ones. An agent forced to always produce a change will produce a plausible wrong one.

def work(item):
    # idempotence key: path + the sha it was queued against
    key = f"{item.path}@{item.sha}"
    if done.has(key):
        return SKIP                       # re-runs are free

    if head_sha(item.path) != item.sha:
        return REQUEUE                    # file moved under us

    result = agent_unit(item)
    if result.status == "UNCLEAR":
        dlq.put(item, result.question)    # human worklist
        return

    commit(result.diff, msg=f"cents: {item.path} ({result.branch})")
    done.put(key)

Four properties that make this survivable

• One writer per file. Partition the queue by path. Two workers on one file is the only true race here, and partitioning removes it without locks.
• Idempotence on path plus sha. A rerun is a no-op, so you can kill the whole fleet at any moment and start it again. Migrations run for days; this will happen.
• Requeue on drift. The repository keeps moving while you work. A file whose sha changed goes back in the queue rather than getting a diff computed against a stale reading.
• The dead-letter queue is the deliverable. Not a failure log. It is the ranked list of sites that genuinely need a person, which is the output you actually wanted from the whole exercise.

Unit tests check the cases somebody thought of. A differential test checks the cases that actually happen, which for a billing system means a year of real transactions. Replay them through both implementations and compare.

for txn in replay(last_365_days):          # 4.1M recorded inputs
    old = legacy.total(txn)                # float dollars
    new = cents.total(txn) / 100           # integer cents, for compare

    if abs(old - new) > 0.005:
        report(txn, old, new)              # a real behaviour change

# Expected output is not "zero differences". It is a small set of
# differences you can explain one by one, because the point of the
# migration was that the old behaviour was wrong.

Where no recorded inputs exist, generate them: property-based testing over the input domain, comparing old and new. Weaker than real traffic and vastly stronger than the unit tests you already have, because both implementations are available to disagree with each other.

1. First Add the new representation alongside the old. A total_cents field written in parallel with total, both maintained. Nothing reads the new one yet.
2. Then Migrate leaves before callers. Modules nobody imports go first, so each merge has the smallest possible blast radius and the dependency order does the sequencing for you.
3. Per merge One shape, one module, differential replay attached to the pull request. Reviewers read twenty representative changes plus every residue commit individually.
4. Last Delete the old field. This is the only irreversible step and it happens weeks after the code is done, once the parallel writes have agreed in production for long enough to believe them.

Running both representations in parallel feels wasteful and is the cheapest insurance available. It gives you a production differential test: every write compares the two answers, and a mismatch alerts. That catches the paths your replay corpus missed, which are precisely the paths nobody thought about.

Two thirds of the human time went to the thirteen percent that needed judgment, and the sixty escalations got two full days. That distribution is the entire point. The comparison is not against doing it by hand, it is against the all-agent version, which would have cost eight times the inference to produce a diff nobody could review in bulk and would have answered those sixty questions confidently rather than asking them.

1. Purely mechanical If the census comes back 100% mechanical, you need a codemod and a reviewer, and nothing else. Building a fan-out for it is elaborate procrastination.
2. A rename Your language server does this correctly, instantly, for free, with full type awareness. There is no version of this where an agent is the right answer.
3. No oracle No recorded inputs, no old implementation to compare against, no property you can state. Then nothing verifies the result at scale and you are shipping ten thousand unreviewed changes with extra steps. Build the oracle first or do not start.
4. Unresolved policy The 890-site question from §1, unanswered. No amount of tooling substitutes for somebody deciding where rounding happens, and starting anyway produces inconsistency that is far harder to find than the original problem.