Building an AI bug-fixing agent that runs inside your GitHub Actions runner

Disclosure: I'm building a product around this (bugzero). This article was drafted with AI assistance, then reviewed and edited by me. Production errors are a good fit for an AI agent. There is a stack trace, there is a repository, and "done" has a clear definition: a pull request that a human reviews. What bothered me was the usual shape of these tools. You give a SaaS read access to your repositories, and often your AI key as well, and hope for the best. So I started from one rule: The service that coordinates the agent should never be able to read the code. Everything below follows from that rule. I'll walk through the design, the code that enforces it, and the things that surprised me. The shape of it Two parts. A control plane knows about errors, configuration and run history. A runner, your own GitHub Actions runner, has the code and the AI key. Sentry / PostHog / webhook │ new error ▼ control plane ── workflow_dispatch ──▶ your GitHub Actions runner (errors, config, run history) · checks out your repo ▲ · runs the agent (your AI key) │ · opens a pull request └── status · PR link · summary · token counts ◀──┘ An error arrives from the error tracker. The control plane picks the mapped repository and starts a workflow in it. In your runner, the agent explores the checked-out code and writes a fix. The fix goes to a new branch, and a pull request is opened. The runner reports status, PR link, a summary and token counts back. The control plane can only press a button The control plane is a GitHub App with two permissions: Metadata: read and Actions: read & write. No Contents. It cannot clone your repository or read a file. All it can do is start a workflow with workflow_dispatch. The workflow lives in your repo, so you can read exactly what runs: on: workflow_dispatch: inputs: run_id: description: bugzero run id required: true type: string base_branch: description: Branch to fix required: true type: string token: description: Per-run callback token required: true type: string control_plane: description: URL to report back to required: false type: string permissions: contents: write pull-requests: write jobs: fix: runs-on: ubuntu-latest timeout-minutes: 25 steps: - uses: actions/checkout@v4 with: ref: ${{ inputs.base_branch }} fetch-depth: 0 - uses: runde-software/bugzero-action@v1 with: run-id: ${{ inputs.run_id }} callback-token: ${{ inputs.token }} control-plane: ${{ inputs.control_plane }} env: ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }} The AI key is a secret in your repository. The control plane never sees it. A token that is good for exactly one run The runner needs to fetch the error context and report the result, and GitHub gives it no identity toward my service. So the control plane mints a per-run token and passes it as a workflow input: It is an HMAC signature over the run id and an expiry, valid for 30 minutes. It is scoped to one run. It lets the runner read that run's error context and post that run's result, nothing else. It is masked in the job logs (::add-mask::). The part I had to think about is that workflow inputs are shown on the run page. Anyone who can read your repository's Actions runs can see the token until it expires. That is why it is short-lived and narrowly scoped, rather than a long-lived credential I hope nobody looks at. An agent with four tools and no shell The agent has four tools: Tool What it does get_file_contents Reads a file in the checkout list_directory Lists a directory search_code git grep over the checkout create_pull_request Writes the fix to a new branch and opens a PR There is no shell and no way to make network requests of its own. The reason is that the model's input is partly attacker-influenced: error messages, stack traces and code comments can contain text written by someone who is not you. An agent that can only read files and produce a diff has a small blast radius, and the diff still goes through code review. Reads go through a path guard that rejects ../ escapes out of the checkout: private resolve(rel: string): string { const root = path.resolve(this.opts.cwd); const abs = path.resolve(root, rel); if (abs !== root && !abs.startsWith(root + path.sep)) { throw new Error(`Path "${rel}" is outside the repository.`); } return abs; } Search is git grep -l -I -F, capped at 20 file names. Fixes are written in one place only: create_pull_request creates a new branch, commits as a bot user, pushes, and opens the PR. It never pushes to your base branch and never merges. The cost of this design is real: the agent cannot run your tests. It can read a test file and mention it, but it cannot execute anything. I think that is the right default for now, and I'm not sure it is the right default forever. More on that at the end. The loop has a budget The loop stops at 30 tool calls or 20 minutes of wall-clock time, whichever comes first. The workflow's job timeout is 25 minutes, which leaves headroom to open the PR and report back. The detail that mattered: every model call gets only the time that is left, not a fixed timeout. From the loop (trimmed): const remaining = deadline - Date.now(); if (remaining