7 · Judgment & Economics — Know the Limits

Reaching for an agent is not free. Every task spends tokens, wall-clock time, and the time it takes to review output you did not write. On some tasks that spending pays off; on others, writing the change yourself is faster and cheaper. Without a way to tell the two apart, the agent gets used by reflex, and the cost of that habit stays invisible because most of it never appears on an invoice. This module is about developing the judgment to know when an agent is worth it and when it is not, and grounding that judgment in measured numbers rather than impressions.

The agent is worth it when the cost to drive and verify is less than the cost to do it yourself, including your review time. For small, well-understood changes, that is often false. The skill is knowing which case you are in.

Concept

Running an agent spends three things, and only one shows up on an invoice.

1. Tokens — the metered cost. Cost per task scales with context size times number of turns. A big context and a long self-correcting loop spend more; a tight context and a clean run spend less. For real engineering work this is usually the smallest of the three costs. It draws the most attention because it is the only one with a number attached.

2. Latency — the wall-clock cost. An agent that takes ten minutes of back-and-forth on a change you could type in two has cost you eight minutes rather than saving them. Latency is cheap when you are out of the loop (background or parallel agents, Module 5) and expensive when you are sitting there watching it think.

3. Review time — the hidden, dominant cost. Verifying output you did not write is the real bill; Module 3 exists because of it. This cost does not shrink as models improve: more fluent, plausible-but-wrong output is harder to review, not easier. It is the cost that decides most break-evens, and the one no token counter shows you.

The unit you measure matters as much as the costs themselves. Cloud cost management offers a durable lesson here: optimize the unit cost rather than the total bill. The unit here is cost per completed, verified task — tokens plus latency plus your review time — not monthly API spend. A task that burns tokens but removes an hour of your work is cheap. A task that looks cheap on tokens but then takes forty minutes to review is expensive. The monthly bill tells you nothing actionable; cost-per-task tells you whether to reach for the agent on the next task of the same shape.

These costs combine into a single decision. Reach for the agent when the cost to drive and verify is less than the cost to do it yourself. That inequality flips on three axes:

• Specifiability. Can you state "done" precisely (Module 2)? A vague task costs more to drive and verify than to do directly.
• Verifiability. Is there machinery to check the result (Module 3)? With no test or eval, review time dominates and the agent usually loses.
• Size and repetition. Large or repeated work amortizes the spec-and-setup cost; a one-line change you already understand does not.

The widget below puts numbers on that inequality. Set your own minute estimates for one task and watch the verdict flip as you raise review time, the cost that decides most break-evens.

Interactive: set minute estimates for doing a task by hand versus specifying, driving, and reviewing an agent, then watch the verdict flip as review time rises. (Enable JavaScript to try it.)

Where the agent pays — and where it doesn't

PaysDoesn't

• Mechanical breadth. A rename or refactor across 40 files, each change obvious and testable.
• Well-specified and gated. A feature with acceptance criteria and a test it can self-correct against.
• Exploration you would otherwise skip. Finding every call site that does X, or drafting three approaches.
• Out-of-the-loop work. Anything you can run in the background or in parallel (Module 5), where latency stops being your cost.

• The one-line fix you already understand. Typing it is faster than specifying it.
• Unverifiable judgment calls. Naming, API taste, or a tricky tradeoff, where review is the work itself.
• Load-bearing, irreversible changes. This is the never-delegate line (Module 3.3), not an economics question. A cost argument should not talk you across it.
• Anything you would spend longer reviewing than doing.

Several failure modes add cost that the token counter never surfaces. Thrashing is the agent burning tokens going in circles; the fix is to kill the run and re-specify rather than wait it out. Over-trust is skipping verification to save time, then paying it back as a production bug at a much worse exchange rate. Under-scoping is a vague spec sending the agent in the wrong direction, costing a full redo. Harness sprawl is covered in Module 4: tools, permissions, and external systems added without repeated need. The root anti-pattern this module exists to address is reflexive reach: using the agent because it is available rather than because the math favors it.

The honest version is that "the agent is always faster" is a claim this course does not make. Sometimes it is not faster. The useful move is to determine which case you are in before you start, and to say so plainly.

Guided Lab

The Practice Repo economics slice is pending. The Triage steps below are method practice: they show what to measure, but they do not yet include token/cost instrumentation or a dedicated answer key. Run the full measurement on your own codebase, where the real review time and wall-clock cost matter.

On the Practice Repo, use Triage to rehearse the decision before doing it on your own repo.

1. Pick two tasks on Triage: one a good fit for an agent (a mechanical change across several files) and one a poor fit (a one-line tweak you already understand).

   Expected: you can say in advance which task you expect the agent to win, and why.

2. Do each task both ways, by agent and by hand, and record the real cost of each: tokens, wall-clock, and the minutes you spent reviewing.

   Expected: four cost figures — agent and by-hand, for each of the two tasks.

3. Compute cost-per-task for both and note which way each one broke.

   Expected: a clear read on which task the agent helped and which it did not, backed by numbers rather than impression.

On your own codebase, the next time you reach for an agent on real work, run the tally before you start: estimate drive-plus-verify against do-it-yourself, then check that estimate against what it actually cost. Calibrating that prediction is the skill this module trains.

Keep the artifact

The artifact is a cost/fit decision log: a running note of which task types paid off and which did not, with the real cost-per-task you measured, so the call becomes a lookup instead of a re-derivation. A one-line "reach for the agent when…" rule for your CLAUDE.md falls straight out of it. Keeping the log is what makes this judgment compound, since each recorded post-mortem informs the next decision.

Self-check

You can state, with numbers, which task the agent helped on and which it didn't, and why each broke the way it did. A gut feeling that one felt faster isn't the bar; the cost-per-task figure is.

Recall

Before moving on: what's the break-even test for using an agent on a given task?

Answer

Reach for it when the cost to drive and verify — including your own review time — is less than the cost to do it yourself. For small, well-understood, hard-to-verify changes, that is often false, and the skill is knowing which case you're in before you start.