AI Development Rules Needed Regression Tests Too

I had already reached an uncomfortable conclusion: Giving an AI coding agent better instructions was not enough. So I started building repository-level governance around things like source-of-truth ownership, review behavior, evidence, scope, and lifecycle. The repository became more consistent. The agents behaved better. And then I ran into another problem. How did I know the rules were actually responsible for the improvement? An AI agent can produce a better result for many reasons: the prompt was slightly different, previous conversation context leaked the expected answer, the agent saw later commits, I accidentally hinted at the problem, or the model simply took a different reasoning path that day. At some point, “this seems better” stopped being good enough. So I started regression-testing the rules themselves. The repository The project is AIDDSkeleton. It is a repository skeleton for AI-driven development. Most of its interesting behavior does not come from scripts or a framework. It comes from repository-local governance: Markdown files that tell an AI agent how to interpret project information, what counts as authority, how work moves through its lifecycle, how evidence should be treated, and how review findings should affect current work. That creates a strange testing problem. If I change a TypeScript function, I can run a test. If I change a rule written in natural language, what exactly do I run? Commit 08ded669 — feedback stopped meaning “do this now” 08ded669 — docs: govern feedback triage and review recall One early change separated two things I had been mixing together: How serious is this finding? and What should we do with it now? A review comment could be valid without automatically becoming current work. The governance started distinguishing dispositions such as: Accept now Reject Defer Observe from classifications such as: Blocker In-scope deficiency Follow-up This made it harder for every interesting suggestion to silently expand the task. It also exposed a deeper problem. I could read the new rule and think: Yes. This is clearer. But that still did not tell me whether an AI agent would actually behave differently. So I started applying candidate governance to another repository that already used AIDDSkeleton-style rules and observing what the agent did. That became the basic idea: A governance change is not validated only by reading the governance repository. It also needs behavioral evidence from a repository actually using it. My first regression tests had a bug At first, the experiments looked promising. I would give the agent a review finding and ask things such as: Is there a common cause? Are there sibling surfaces with the same problem? Should this be reconsidered at a higher level? The agent generalized. It found related problems. It produced broader corrections. Great. Except for one thing. I had told it what kind of reasoning I wanted. The candidate governance might have been causing the behavior. But the test prompt was carrying the same idea. I had effectively written the expected answer into the input. That meant the experiment could answer: Can the model follow this hint? But not: Did the governance change the model's behavior? The regression test itself had a bug. Commit 0611d61b — a finding became more than a patch target 0611d61b — docs: shift review learning into adversarial self-review The review model then changed more substantially. A valid finding was no longer just something to patch. It could also reveal something about the reasoning that had failed to detect the problem earlier. The idea was roughly: finding | v what did we miss? | v does that missed perspective matter elsewhere? That sounded useful. It also made the testing problem harder. If I explicitly asked the agent to inspect sibling paths or look for a shared cause, I would no longer know whether the governance itself was responsible. So the prompt had to become more neutral. Freezing the past The next version of the experiment started from a real historical commit. Not the latest repository. Not a manually reconstructed example. An actual past state. I treated that commit as a fixture. historical consumer commit | +-- Control | old governance | +-- Treatment candidate governance Both worlds started from the same repository state. The important part was what the agent was not allowed to see. Everything after the fixture was treated as nonexistent: later commits, later Pull Requests, later review comments, known fixes, previous regression results. In other words, I tried to create two artificial timelines from Git history. The agent could see the past. It could not see the future. If a later review had already identified a bug, allowing the agent to discover that comment would destroy the experiment. It would no longer be testing whether the governance helped the agent find the issue. It would be testing whether the agent could read the answer. Control and Treatment did not behave the way I wanted This was probably the point where I started trusting the method. The candidate governance did not simply win every round. In one MediaStorage scenario, the Control branch explored a collision problem more aggressively. The original issue was effectively: destination already exists The Control agent spontaneously pushed further toward: what if two writers race for the same destination? The Treatment agent improved the requested surface, but in that run it did not expand as far into the race condition. That was inconvenient. It was also excellent evidence. If every experiment had ended with: new rule good I would have trusted the experiments less. Instead, I had an actual behavioral concern. The new governance seemed stronger in some kinds of generalization, but possibly weaker in immediate exploration breadth. So I did not declare victory. I kept changing the governance. Commit 94d163b8 — a useful experiment that I deliberately removed At one point I tried persistent review knowledge. The idea was attractive. If one task teaches the agent a useful lesson, why not save it and reuse it later? The experiment worked. Almost too well. The retained knowledge materially changed later reasoning. It helped the agent go deeper. But it also influenced decisions outside the original task strongly enough that the boundary between: useful prior knowledge and: accidental design authority became difficult to control. So I removed it. 94d163b8 — docs: remove knowledge base governance That commit changed how I thought about successful AI testing. A test did not need to prove that a new feature was good. It could tell me: This idea clearly affects behavior, but the side effects are not bounded well enough. Do not ship it. That is still a successful test. The test became part of the development loop By this point the workflow looked suspiciously familiar. change governance | v run consumer regression | v observe unexpected behavior | v change governance again | v rerun regression That is just software development. The implementation language happened to be English. Commit e5033e6d — refactoring natural language broke behavior e5033e6d — docs: compact adversarial self-review escalation Next I tried to make the review rules shorter. The existing responsibilities were compressed into a smaller structure. The new version looked cleaner. I believed the semantics were preserved. Then review started finding things I had lost. The compacted version had weakened or dropped details such as: diagnosing why a problem escaped detection, retaining the concrete context that exposed it, reopening review when later evidence changed the basis, keeping review proportional and bounded, reopening broader review after a broad correction. This was a useful reminder that natural-language refactoring can fail exactly like code refactoring. The result may look equivalent. The behaviorally important conditions may not be. Eventually I stopped treating “fewer bullets” as a goal. Commit 1f5dafd6 — structure won over compression 1f5dafd6 — docs: separate adversarial review responsibilities The rule was reorganized around five responsibilities: 1. Initial adversarial review 2. Signal assimilation 3. Correction challenge 4. Review continuation / re-entry 5. Structural reassessment The text became slightly larger again. But the responsibilities became clearer. At that point I had another problem. Most of the behavioral evidence had been collected against earlier candidate commits. The governance had changed during review. So the final merge candidate needed to go back through the consumer test again. Evidence from an earlier revision was not enough. The final candidate went back into the frozen past This time I did not need another full Control/Treatment A/B. The earlier comparisons had already given me a baseline. What I wanted to know was simpler: Does the actual final candidate still behave well in the historical fixture? So I ran it again. No hints about the behaviors I hoped to see. No later review comments. No historical answer key. This time the earlier breadth concern did not reproduce. The agent independently moved from an existing-key collision problem into concurrent same-key publication. That was encouraging. Then it did something even more interesting. It explored media reads and introduced requirements around: avoiding whole-artifact buffering, backpressure, client-disconnect cancellation, handling storage read failures after response start. Technically? Reasonable. Possibly even good engineering. And that was exactly the problem. The AI found a good idea that it was not authorized to adopt I went back to the historical requirements. Those read-performance behaviors had not actually been required. The approved design did not establish them as current acceptance criteria. The agent had crossed a subtle boundary. It had moved from: Find deficiencies in the adopted design. to: Improve the design because this would be better. This is one of the failures I now find most interesting in AI-driven development. The generated result can look better and still be wrong. An agent does not only need to discover useful things. It also needs to distinguish: this is a real deficiency in current work from: this is a good idea for some other scope A benchmark based only on code quality could easily reward the wrong behavior here. I sent one finding back into the same session I did not tell the agent to revert the change. I did not tell it which parts to keep. I gave it one narrower challenge: Are these new read-performance contracts actually required by the existing approved scope, or were useful improvements promoted into formal design without sufficient authority? The agent re-evaluated its own previous work. It removed the new read-performance requirements. But it did not revert everything from the previous correction. The collision changes stayed. Why? Because those were supported by an already-adopted invariant: do not silently overwrite an existing key Concurrent publication could violate that invariant. So collision safety remained an in-scope correction. The new read-performance contract did not. The resulting behavior was effectively: finding | v re-evaluate authority and scope | +-- read performance requirements | no existing requirement | -> remove | +-- collision safety existing invariant -> keep That was a much stronger result than simply finding more bugs. What I learned from black-box testing AI governance The obvious lesson is that prompts can be tested. But that is not really the interesting part. The more important lesson is that repository governance behaves like executable policy. Changing a few sentences can alter: what an agent inspects, when it expands its search, what it considers evidence, when it reopens review, when it stops, and what it believes it is allowed to change. Those are behavioral changes. So I now treat significant governance changes more like code changes. I want: fixed historical fixture controlled candidate historical isolation black-box execution behavioral comparison regression after refinement final-candidate regression Not because this proves the rules are correct. It does not. LLM behavior is non-deterministic. One successful run proves very little. But it is much stronger evidence than: I rewrote the instructions and they look clearer now. Natural language does not mean “untestable” I used to think of repository instructions as documentation. Now I think that is incomplete. When AI agents are active participants in development, those instructions are part of the system that produces changes. They deserve regression tests. Not necessarily traditional unit tests. Sometimes the most useful test is: freeze a real historical repository state, hide everything that happened afterward, change only the governance, give the agent the same task, observe what it does differently. And most importantly: do not tell the agent what improvement you are hoping to see. Otherwise you are not testing the rule. You are testing your hint. That led directly to the next problem. Once the agent became better at questioning its own work and noticing that a local finding might point beyond the exact line that failed, I had to decide: How much of that reasoning should happen autonomously? That became the next set of commits. This article was drafted with AI assistance based on the actual development history, commits, reviews, and regression experiments of AIDDSkeleton. The experiments and repository changes described here were performed against real project history; AI was used to help structure and edit the prose. Repository: joyrswd/AIDDSkeleton