Make Your Weekend Agent List Unknowns First

You sit down Friday with one messy GitHub issue. You want a usable plan before Monday morning. You paste the text into a chat agent. The reply looks polished, complete, and very sure. It adds Kafka, Redis, and a new auth service. The issue never named those systems. You do not have a brownfield budget this weekend. You also do not want fake architecture. You need a gate that blocks invented specs. The failure you are fixing Agents fill gaps with confident guesses. That habit wrecks small side projects. You ship a plan that your repo cannot support. A recent wave of agent write-ups repeats the same pain. The model assumes missing APIs, queues, and owners. You then spend Sunday deleting those guesses. This article builds a tiny assumption gate. You feed an issue. You get facts, unknowns, and a pass or fail. You skip dashboards, vector stores, and multi-agent graphs. What you will ship tonight You will ship a stdin tool in Python. It extracts claimed nouns from the issue. It asks a model for a plan only after that list exists. The tool rejects any plan that introduces new systems. It prints a JSON report you can paste into a PR. It runs without a database. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If you need free model access and a free server for this weekend check, MonkeyCode is one place to run it. You can keep the same script on your laptop either way. Scope you will cut on purpose You will not call GitHub. You will not store history. You will not rank models. You will not add embeddings. You will not add a web UI. You will not chain tools. Those cuts keep the demo honest. A weekend agent dies from extra moving parts. You want a gate, not a platform. Decision table before you write code Use this table before you add any feature. Idea Keep this weekend? Why Stdin issue text Yes You already have the file. Fact list from the issue Yes The gate needs ground truth. Unknowns list Yes This is the actual product. Plan JSON from a model Yes You still want a draft. Invented-system scanner Yes This is the fail path. GitHub API No Auth and rate limits steal hours. Vector database No One issue does not need retrieval. Multi-agent debate No Two models invent twice as much. Pretty dashboard No JSON in the terminal is enough. If a row says no, you delete that ticket. You do not park it in a backlog. Weekend scope is a knife, not a parking lot. Step 1: Freeze the contract Create assumption_gate.py. Keep the contract tiny and strict. The script must print JSON and an exit code. #!/usr/bin/env python3 """Reject AI plans that invent unstated systems.""" from __future__ import annotations import json import os import re import sys from pathlib import Path from typing import Any STOP = { "a", "an", "the", "and", "or", "to", "of", "in", "on", "for", "with", "we", "you", "it", "this", "that", } SYSTEM_HINTS = { "redis", "kafka", "rabbitmq", "postgres", "mysql", "mongo", "s3", "sqs", "lambda", "kubernetes", "auth0", "okta", "elasticsearch", "opensearch", "graphql", "grpc", } def load_issue(path: str) -> str: text = Path(path).read_text(encoding="utf-8") if len(text.strip()) < 40: raise SystemExit("issue file is too short to gate") return text You now have a hard floor on input size. Short prompts invite fiction. The gate should refuse thin issues early. Step 2: Pull facts without a model You extract candidate systems with boring regex. This list is your ground truth. The model may not expand it. def tokenize(text: str) -> set[str]: words = re.findall(r"[A-Za-z][A-Za-z0-9+_.-]{1,}", text.lower()) return {w for w in words if w not in STOP} def stated_systems(text: str) -> set[str]: tokens = tokenize(text) found = {name for name in SYSTEM_HINTS if name in tokens} # Keep explicit TitleCase services from the issue only. titled = re.findall(r"\b([A-Z][A-Za-z0-9]+(?:Service|Queue|Store|API))\b", text) found.update(name.lower() for name in titled) return found This extractor is deliberately dumb. Dumb is a feature here. You want recall of named systems, not creative inference. Step 3: Define the plan schema You force the model into three lists. Plans, facts, and unknowns must stay separate. Mixing them is how Kafka appears. SCHEMA = { "type": "object", "required": ["facts", "unknowns", "steps", "systems"], "properties": { "facts": {"type": "array", "items": {"type": "string"}}, "unknowns": {"type": "array", "items": {"type": "string"}}, "steps": {"type": "array", "items": {"type": "string"}}, "systems": {"type": "array", "items": {"type": "string"}}, }, } PROMPT = """You write an implementation plan. Use only systems named in the issue. If a system is missing, put a question in unknowns. Never invent queues, databases, or vendors. Return JSON with keys facts, unknowns, steps, systems. ISSUE: {issue} """ You should paste that schema into your notes. The gate scores systems against the issue. unknowns is the success path, not a failure. Step 4: Add a local mock so the demo runs You may lack an API key tonight. The mock still proves the gate. It invents Kafka on purpose so you see a fail. def mock_plan(issue: str) -> dict[str, Any]: return { "facts": ["The issue asks for a faster digest email."], "unknowns": [], "steps": [ "Add a Kafka topic for digest events.", "Write a worker that sends email.", ], "systems": ["kafka", "email"], } Run the mock first. You want a red result before any network call. A green-only demo teaches nothing. Step 5: Optional live call, no secrets in git Keep the live path optional and boring. Read the base URL from the environment. Never print the key. def live_plan(issue: str) -> dict[str, Any]: import urllib.request url = os.environ.get("LLM_URL", "").rstrip("/") key = os.environ.get("LLM_KEY", "") model = os.environ.get("LLM_MODEL", "") if not url or not model: raise SystemExit("set LLM_URL and LLM_MODEL, or use --mock") payload = json.dumps({ "model": model, "messages": [{"role": "user", "content": PROMPT.format(issue=issue)}], "temperature": 0, }).encode("utf-8") req = urllib.request.Request( url + "/chat/completions", data=payload, headers={"Content-Type": "application/json"}, ) if key: req.add_header("Authorization", "Bearer " + key) with urllib.request.urlopen(req, timeout=45) as resp: body = json.loads(resp.read().decode("utf-8")) content = body["choices"][0][message_key(body)]["content"] return json.loads(extract_json(content)) def message_key(body: dict[str, Any]) -> str: return "message" def extract_json(text: str) -> str: start = text.find("{") end = text.rfind("}") if start < 0 or end < 0: raise ValueError("model did not return JSON") return text[start:end + 1] You still own the URL and model name. This article will not invent product model names. Point the env vars at whatever endpoint you already trust. Step 6: Score invented systems The scorer is the whole product. Compare plan systems to stated systems. Unknowns do not fail the run. Inventions do. def score(issue: str, plan: dict[str, Any]) -> dict[str, Any]: allowed = stated_systems(issue) claimed = {s.lower().strip() for s in plan.get("systems", [])} invented = sorted(s for s in claimed if s and s not in allowed and s not in {"email", "http"}) unknowns = [u for u in plan.get("unknowns", []) if str(u).strip()] ok = not invented return { "pass": ok, "allowed_systems": sorted(allowed), "plan_systems": sorted(claimed), "invented_systems": invented, "unknowns": unknowns, "steps": plan.get("steps", []), "facts": plan.get("facts", []), } Email and HTTP stay on a tiny allowlist. Almost every issue implies them. Kafka does not get that courtesy. Step 7: Wire the command Finish with a small CLI. Exit 1 on invented systems. Exit 0 when the plan stays inside the issue. def main() -> None: if len(sys.argv) < 2: raise SystemExit("usage: assumption_gate.py ISSUE.md [--mock]") issue = load_issue(sys.argv[1]) use_mock = "--mock" in sys.argv[2:] plan = mock_plan(issue) if use_mock else live_plan(issue) report = score(issue, plan) json.dump(report, sys.stdout, indent=2) sys.stdout.write("\n") raise SystemExit(0 if report["pass"] else 1) if __name__ == "__main__": main() Save a sample issue next to the script. Keep it ugly and incomplete. Perfect specs hide the bug you are hunting. # Digest email is late Users wait until noon for the daily digest. We already send mail from `mailer.py`. Please make the existing job faster. No new vendors. Step 8: Run the red path, then the live path chmod +x assumption_gate.py python3 assumption_gate.py sample_issue.md --mock; echo exit:$? You should see kafka under invented_systems. The process should exit 1. That is the demo working. Then try a live model only if you already have an endpoint. export LLM_URL="http://127.0.0.1:8080/v1" export LLM_MODEL="local-or-hosted-model" python3 assumption_gate.py sample_issue.md; echo exit:$? A good live result lists unknowns instead of vendors. "What is the current job runtime?" is a win. "Add Redis" is a fail. What you skipped, and why it matters You skipped a second agent that critiques the first. Debate often adds more invented systems. One strict scorer beats two polite liars. You skipped issue fetch from GitHub. Tokens, permissions, and pagination eat the weekend. A pasted markdown file is enough evidence. You skipped a database for plan history. History is tempting and useless tonight. You need one honest report, not analytics. Limitations you should say out loud The noun list will miss odd product names. Teams invent unique service titles. Extend SYSTEM_HINTS when your repo uses them. The JSON parse is brittle. Models still wrap objects in markdown. extract_json is a seatbelt, not a parser library. The allowlist can hide real scope creep. "Email" might mean a new vendor. Read the steps, not only the score. This gate does not measure code quality. It only blocks unstated infrastructure. You still review the steps as a human. Who should not use this Do not use this on regulated production agents. You need evals, logs, and an owner. A weekend script is not that layer. Do not use this when the issue is a research spike. Exploration needs guesses. The gate is for implementation plans. Do not use this if you cannot name current systems. The extractor needs words in the issue. Empty context makes every plan look invented. A short checklist for Monday Paste the real issue into sample_issue.md. Run --mock and confirm a red Kafka fail. Run a live model with temperature zero. Treat unknowns as tickets, not as shame. Delete any step that names a new system. Ship the smallest change inside the repo you have. You now have a working demo with a brutal scope cut. The agent may still be wrong on details. It is much less free to invent a platform. Keep the script in the repo root. Run it before you accept any AI plan. That habit is the whole weekend win.