Trade-offs
← Home
The bargain you're making.
Loops trade a one-shot answer for the chance to be right. That trade costs money, latency, and complexity — and pays out only if you engineered the verifier.
PROS
- Persistence beats intelligence. A mediocre model that retries with feedback often beats a stronger model taking one shot.
- Verifiable outcomes. Tests, checkers, or graders let the loop know when it's actually done — not when the model feels done.
- Composability. Loops nest. A supervisor loop coordinates evaluator loops that wrap tool-use loops. Structure grows to fit the problem.
- Recovery from partial failure. A well-designed loop treats each step as replayable. Transient errors stop being outages.
- Autonomy on tap. Async, long-running, "come back when done" workflows unlock work that was too tedious to prompt one step at a time.
CONS
- Cost is quadratic on a bad day. Iterations multiply tokens. Context grows with each pass. Runaways burn dollars before they burn wallclock.
- Latency is now measured in minutes. Every extra pass adds a model call plus tool time. Users need progress signals, not spinners.
- Self-grading is weak. Models are lenient graders of their own work. Prefer deterministic checks or an independent critic model.
- Context poisoning compounds. A wrong observation early can steer every later step. Small errors become confident bad answers.
- Observability is not free. Traces, replays, cost dashboards, run diffs — you'll build them, and you'll wish you had built them sooner.
The conductor's rule: start small. Play the passage by hand first — then codify it into a skill, add verification, wrap it with a trigger and hard limits. Only then let it run without you.