Oracle First: Routing AI-Generated Diffs With a Glossary and Four Leaves

Consider this scene. It is a composite, not a personal war story. An agent ran overnight on a leftover prompt. Morning git status showed fourteen files. Two of them implemented the requested endpoint. The rest were a new logger, a renamed helper, a rewritten Dockerfile, and a README that now contradicted the tests. Generation cost was close to zero. The next four hours were not. That gap is the actual product problem. When a patch is cheap to produce, the expensive work is routing: keep, quarantine, or rewrite. Skip the routing and the cheap code becomes expensive debt with extra files attached. This article is a glossary, a routing tree, and a worked example at each leaf. The artifact is a small classifier plus a quarantine command sequence. Treat the code as a proposal unless you run it on your own repo. The problem the scoreboard hides Free-model loops optimize for “a diff appeared.” Reviewers optimize for “this diff is safe to merge.” Those are different objective functions. A green unit test on a helper you did not ask for is not evidence that the architecture still holds. Cheap generation also changes failure shape. The common failure is no longer “the model wrote nothing.” It is silent expansion: extra modules, extra dependencies, extra comments that drift from the contract. Routing has to detect that shape before anyone debates style. Glossary Use these terms as they are defined here. Nearby words in vendor blogs do not override them. Cheap diff. A change whose generation cost is negligible next to the human time needed to decide its fate. Cost here means review and rollback, not GPU invoices. Oracle. An automated check that can reject a patch without reading every line. A contract test, a typecheck, a golden fixture, or a linter with a frozen config can be an oracle. A vibe is not. Surface area. The set of paths, exported names, and config files the diff touches. Count files, but also count contracts: HTTP shapes, CLI flags, schema versions. Silent expansion. Files or refactors the prompt did not request. Expansion is a routing signal, not a taste argument. Quarantine run. Execute the generated tree off the developer workstation, with no production secrets and no write access to the source remote. Allowlist apply. Bring only named paths from the generated tree onto a local branch. Everything else stays discarded or parked. Architecture-touching change. A patch that alters auth, persistence, process boundaries, or public contracts. These patches fail closed: rewrite, do not quarantine-and-hope. Review budget. The hours a human can spend before the cheap patch costs more than writing it by hand. When the budget is exceeded, the tree’s answer is rewrite, not “one more prompt.” The routing tree Walk the questions in order. Do not skip to a leaf because the diff “looks small.” Step 1 — Is there an oracle that already fails, or an oracle you can add in under fifteen minutes? No oracle, and you cannot add one quickly: go to Leaf D (rewrite the work, or rewrite the prompt into a smaller contract). Ungrounded generation is not a review task. Oracle exists or can be added: continue. Step 2 — Is surface area bounded? Bound means: requested paths only, or requested paths plus test files. A hard cap helps. A working default is “three production files and their tests.” Silent expansion beyond the cap: go to Leaf A (discard, tighten the prompt, regenerate). Bounded: continue. Step 3 — Does the patch need network, secrets, or write access outside a temp directory? Yes, and the need is real (migrations, signed webhooks, vendor APIs): go to Leaf D unless you already have a dedicated staging path that is not your laptop. Yes, but the need is accidental (the model added telemetry, a download, or .env reads): go to Leaf A. No: continue. Step 4 — Can an isolated process execute the oracle? Isolated execution is available (container, spare VM, or a throwaway server): go to Leaf B (quarantine run). Isolated execution is not available, and the patch is a pure function with an allowlist of one or two files: go to Leaf C (local allowlist apply). Isolated execution is not available, and the patch is larger than that: go to Leaf D. The tree is deliberately biased toward discard and rewrite. Cheap generation makes “try it locally” the risky default, not the brave one. Artifact: a proposed classifier The script below does not prove safety. It only encodes the tree’s cheap heuristics so a human does not re-litigate them every morning. Label: proposal, unexecuted on your tree until you run it. #!/usr/bin/env python3 """classify_patch.py — proposal heuristic, not a security scanner.""" from __future__ import annotations import subprocess import sys from pathlib import Path ALLOWED_PREFIXES = ("src/", "lib/", "tests/", "test/") ARCH_HINTS = ("auth", "middleware", "migration", "dockerfile", "compose", ".github/") SECRET_HINTS = ("os.environ", "getenv(", "api_key", "BEGIN ", ".env") MAX_PROD_FILES = 3 def git_names(diff_range: str) -> list[str]: out = subprocess.check_output( ["git", "diff", "--name-only", diff_range], text=True ) return [line.strip() for line in out.splitlines() if line.strip()] def patch_text(diff_range: str) -> str: return subprocess.check_output(["git", "diff", diff_range], text=True) def classify(diff_range: str, requested: set[str]) -> str: names = git_names(diff_range) body = patch_text(diff_range).lower() prod = [n for n in names if not Path(n).parts[0].startswith("test")] extra = [n for n in names if n not in requested and not n.startswith("test")] if any(h in n.lower() for n in names for h in ARCH_HINTS): return "LEAF_D_REWRITE_architecture_touch" if any(h in body for h in SECRET_HINTS): return "LEAF_A_DISCARD_secret_or_env_touch" if extra or len(prod) > MAX_PROD_FILES: return "LEAF_A_DISCARD_silent_expansion" if not names: return "LEAF_A_DISCARD_empty" if all(n.startswith(ALLOWED_PREFIXES) for n in names) and len(prod)