Stop Orphaning AI Shortcuts: A Team SOP You Can Paste Into the Wiki

You merge a Friday afternoon pull request that an assistant drafted in about twenty minutes. The tests are green, the description sounds confident, and nobody wants another meeting before the weekend. On Monday a cache key collides in production, and the shortcut lives in a helper nobody remembers writing. The thread then asks a painful question: who actually owns the debt that cheap generation just created? Cheap generation does not make maintenance cheap, and that gap is now a team operations problem. Assistants draft glue code, copy nearby patterns, and leave TODOs that never become tickets. Your wiki probably explains how to open a pull request, yet it rarely explains who files, ranks, and retires the shortcuts those pull requests leave behind. This article gives you a paste-ready SOP, a decision table, and a proposed overnight scanner you can run without inventing a new process religion. Why this SOP is not another review checklist Pre-merge review still matters, but it cannot see debt that only appears after traffic, retries, and a second feature land on the same file. Reviewers already drown in volume, so they approve local correctness and miss ownership. You need a post-merge queue with named humans, not another comment macro that everyone learns to ignore. Treat AI-shipped shortcuts like incoming incidents: they need a clerk, an author, a reviewer, and a service owner who can accept risk in writing. The workflow below assumes your team already ships with assistants somewhere in the loop. It does not require you to ban those tools, and it does not pretend a linter can replace architecture judgment. It only forces every detected shortcut onto a ticket with an owner before the next merge wave buries it. If that sentence feels heavy, you are the audience this SOP is for. Roles that keep the queue honest Name four seats and rotate the first one weekly so the queue cannot stick to one volunteer. Write the names on the wiki page, not in a chat pin that disappears after two standups. If a seat is empty, you do not run the scanner that night; an unowned queue is how this process rots. Debt Clerk (rotating). Opens or updates tickets from the overnight queue, and refuses to close items that lack an author. Authoring Engineer. The person who merged the change; they confirm intent, add reproduction notes, and propose a repayment patch or an accept-risk statement. Debt Reviewer. Someone other than the author; they challenge severity, reject fuzzy titles, and send thin tickets back to the clerk. Service Owner. Accepts documented risk, schedules repayment on a dated backlog, or escalates when the shortcut sits on a production path. On-call is not a fifth planner. Page on-call only when a queue item is tagged prod-risk and a customer-facing symptom already exists. Everything else waits for working hours so you do not train the team to treat TODOs like pages. Decision table you can enforce in standup Use this table as the only ranking language allowed on tickets. If a clerk cannot pick one row, the item goes back to needs-triage instead of getting a heroic guess. Proposed example: copy the table into your wiki and refuse screenshots of it in Slack. Signal on the change Default severity Next handoff Allowed exit Comment or name contains TODO, HACK, FIXME, or ai-generated s3-cleanup Clerk → Author Repayment PR or dated won't-fix New except Exception, swallowed error, or retry without jitter s2-correctness Author → Reviewer Test plus patch, or owner accept-risk Cache key, authz check, money math, or PII field touched s1-prod-risk Reviewer → Service Owner Owner-dated plan or rollback Duplicate helper beside an existing module s3-cleanup Clerk → Author Delete or merge the duplicate Scanner noise, generated lockfile, or vendor folder s4-ignore Clerk closes with reason No ticket You should read severity as cost of delay, not as moral failure. An s3-cleanup that sits for a month is still cheaper than a vague s1 that nobody believes. The reviewer exists to stop severity inflation, which is how these queues die. Seven numbered steps from clone to wiki Follow the steps in order the first week, then keep only the overnight job and the standup pass. Label this a proposed workflow until your team has run it against one real repository. Do not start by scanning the entire monorepo history or you will drown the clerk on day one. Pick a blast radius. Limit the first scan to the service that merged the most assistant-touched diffs this month, not the whole company org. Freeze the marker list. Start with TODO, FIXME, HACK, XXX, nosec, type: ignore, and ai-generated so arguments happen in the wiki, not in the scanner. Run the scanner once by hand. File nothing until you have spent fifteen minutes rejecting obvious noise with the clerk and the service owner together. Turn on the overnight job. Write JSON to a known path, then let the clerk open tickets from that file during the first hour of their rotation. Handoff in the ticket, not in chat. Author, reviewer, and owner fields are required; a missing owner blocks the next merge to that package if you already use CODEOWNERS. Close only through the table. Every closed ticket cites a row, a commit, or a dated accept-risk note signed by the service owner. Review the SOP monthly. Add a marker only when two incidents share it; delete a marker when it produces mostly s4-ignore for two weeks. Those seven steps are the entire operating cadence. If your team wants a dashboard before step three, you are decorating the process instead of staffing it. Resist that urge for one month and judge the SOP by closed s1 and s2 items, not by chart polish. Proposed artifact: an overnight debt queue scanner The script below is a proposed example you can adapt; it is not a claim about any production deployment. It walks a repository, records matching lines, and writes a stable JSON queue the clerk can read. Optional summarization uses whatever OpenAI-compatible chat endpoint you already export in the environment, and it stays silent when those variables are absent. #!/usr/bin/env python3 """Proposed overnight scanner for AI-era shortcuts. Adapt before you cron it.""" from __future__ import annotations import json import os import re import sys import urllib.request from dataclasses import asdict, dataclass from datetime import datetime, timezone from pathlib import Path MARKERS = ( "TODO", "FIXME", "HACK", "XXX", "nosec", "type: ignore", "ai-generated", "generated-by-assistant", ) SKIP_PARTS = {".git", "node_modules", "dist", "build", "vendor", ".venv"} CODE_SUFFIXES = {".py", ".ts", ".tsx", ".js", ".go", ".rb", ".java", ".rs"} SEVERE_HINTS = re.compile( r"cache|authz|authoriz|token|password|pii|gdpr|money|invoice|retry", re.I ) @dataclass class Finding: path: str line: int marker: str text: str severity: str def iter_files(root: Path): for path in root.rglob("*"): if not path.is_file() or path.suffix.lower() not in CODE_SUFFIXES: continue if any(part in SKIP_PARTS for part in path.parts): continue yield path def classify(text: str) -> str: if SEVERE_HINTS.search(text): return "s1-prod-risk" if "except Exception" in text or "rescue StandardError" in text: return "s2-correctness" return "s3-cleanup" def scan(root: Path) -> list[Finding]: findings: list[Finding] = [] for path in iter_files(root): try: lines = path.read_text(encoding="utf-8", errors="replace").splitlines() except OSError: continue for idx, raw in enumerate(lines, start=1): for marker in MARKERS: if marker.lower() in raw.lower(): findings.append( Finding( path=str(path.relative_to(root)), line=idx, marker=marker, text=raw.strip()[:240], severity=classify(raw), ) ) break return findings def maybe_summarize(findings: list[Finding]) -> str | None: base = os.getenv("LLM_BASE_URL") key = os.getenv("LLM_API_KEY") model = os.getenv("LLM_MODEL") if not base or not key or not model: return None payload = { "model": model, "messages": [ { "role": "system", "content": "Summarize debt findings for a clerk. No extra advice.", }, { "role": "user", "content": json.dumps([asdict(item) for item in findings[:40]]), }, ], "temperature": 0, } req = urllib.request.Request( base.rstrip("/") + "/chat/completions", data=json.dumps(payload).encode("utf-8"), headers={ "Authorization": f"Bearer {key}", "Content-Type": "application/json", }, method="POST", ) with urllib.request.urlopen(req, timeout=45) as resp: body = json.loads(resp.read().decode("utf-8")) return body["choices"][0]["message"]["content"] def main() -> int: root = Path(os.getenv("REPO_ROOT", ".")).resolve() out = Path(os.getenv("QUEUE_PATH", "debt-queue.json")) findings = scan(root) report = { "generated_at": datetime.now(timezone.utc).isoformat(), "root": str(root), "count": len(findings), "findings": [asdict(item) for item in findings], "model_summary": maybe_summarize(findings), } out.write_text(json.dumps(report, indent=2), encoding="utf-8") print(f"wrote {len(findings)} findings to {out}") return 0 if __name__ == "__main__": sys.exit(main()) Run it once in the foreground so you can see the noise before any ticket exists. Keep the first command boring, local, and easy to paste into the wiki beside the SOP. python3 debt_queue.py REPO_ROOT=. QUEUE_PATH=/tmp/debt-queue.json python3 debt_queue.py python3 - Author (ticket assignee) - Author -> Reviewer (severity + patch or accept-risk draft) - Reviewer -> Service owner (s1 always, s2 when disputed) - On-call only for s1 with a live customer symptom Exit rules - Close with commit SHA, or with owner-dated accept-risk - No Slack-only closures - Empty seat means the scanner stays off Out of scope - Full architecture redesigns - Vendor folders and lockfiles - Personal laptops as the nightly host That page is deliberately dull. Dull pages get followed; clever pages get debated. If someone wants to add a flowchart, ask them to close three s2-correctness tickets first and then propose one extra sentence, not a new microsite. Limitations you should write under the paste Marker scanning is noisy, and noisy queues train clerks to stop reading. Generated files, vendored code, and copied error handlers will dominate unless you maintain SKIP_PARTS with the same seriousness you maintain CODEOWNERS. A free chat model can summarize the JSON, but it can also rank a harmless TODO above a silent authorization bypass if your hints are weak. This SOP does not detect missing abstractions, wrong domain boundaries, or product bets that should never have been generated in the first place. The process also fails when tickets outrun staffing. If the clerk needs more than one hour a morning, you shrank the blast radius too late or you allowed s4-ignore items onto the board. Stop the timer, clean the board with the service owner, and only then widen the scan. A stalled board is worse than no scanner, because it creates the fiction that debt is already being managed. Who should not use this approach Skip this SOP if you are a solo hobbyist with no wiki and no second reviewer, because you will just file mail to yourself. Skip it if your organization already runs a formal architecture review board that must sign every production path; a marker scanner would duplicate that authority and leak unofficial severity language into audits. Skip it if you cannot keep a read-only clone and a secret-free environment for the overnight host. Skip it if leadership wants a dashboard this week more than they want named owners on ten real tickets. Teams that benefit are small enough to name four seats, and busy enough that assistant-authored glue is already landing every day. You will know it is working when Monday incidents point at a ticket that already had an owner, not at a helper nobody remembers writing. Run the scanner on a host you already have, paste the one-page SOP, and judge the next two weeks by closed s1 items rather than by how clever the summary sounds.