How to Review AI-Generated Test Cases Before You Trust Them

I'm the only QA engineer on a large development team, so I use AI a lot. ChatGPT, Claude and Copilot all help me get a first draft of test cases down far faster than I could type them. But here's the thing I've learnt the hard way: AI drafts look more complete than they are. They're well formatted, confidently worded and neatly numbered. That polish makes it very tempting to skim, nod and paste them into the test management tool. [MY REAL EXAMPLE] The AI's job is to draft. My job is to verify, not rubber-stamp. Below are the seven checks I run on every AI-generated set of test cases before I trust any of them. Trace every test back to a requirement Every test case should exist because of something in the story, spec or acceptance criteria. If I can't point to where a test came from, I don't know whether it's testing the product or testing the AI's imagination. What to look for: a test like "Verify the user can export results to PDF" when the story only mentions CSV export. It reads perfectly sensibly. It's just not in scope. Quick tip: ask the AI to add a "Source" line to each test quoting the exact requirement it covers. Then check those quotes against the real document — the AI can misquote too. Hunt for invented details This is the one that catches people out most. When the AI doesn't know a detail, it often fills the gap with something plausible: a field name, an error message, a validation rule, a timeout, a role name. What to look for: Field labels that don't match the UI ("Email Address" when the screen says "Work email") Exact error message text nobody gave it Business rules stated as fact ("accounts lock after 5 failed attempts") that appear nowhere in the requirements Quick tip: highlight every specific value in the draft — numbers, strings, names — and confirm each one against a source. Anything you can't confirm becomes a question for the product owner, not an expected result. Check negative and edge cases AI drafts lean heavily towards the happy path. You'll often get six variations of "valid input succeeds" and one token "invalid input shows an error". What to look for: missing coverage for empty fields, boundary values, maximum lengths, special characters, duplicate submissions, expired sessions, permissions for different user roles, and what happens when a dependency is slow or unavailable. Quick tip: after the first draft, ask specifically: "List the negative, boundary and error-handling cases that are missing from these tests." Then apply your own judgement — you know where this system has broken before. The AI doesn't. Make sure expected results are testable An expected result should let two different testers reach the same pass/fail verdict. AI loves vague outcomes that sound like results but can't actually fail. What to look for: phrases like "the system works as expected", "the page loads correctly", "an appropriate message is displayed" or "data is saved successfully". Quick tip: for each expected result, ask yourself "how would I prove this failed?" If there's no clear answer, rewrite it as something observable: a specific message, a specific state, a specific record in the database. Remove duplicates and padding Ask for twenty test cases and you'll usually get twenty, whether the feature needs them or not. Volume isn't coverage, and padded suites cost real time to run and maintain. What to look for: tests that differ only in wording, tests that check the same rule with an equivalent input, and filler like "Verify the page title is displayed" that adds nothing to the risk you're covering. Quick tip: group the tests by the rule or behaviour they cover. Where two tests land in the same group with no meaningful difference in input or risk, merge them or delete one. Check for sensitive data in prompts and test data This applies in both directions. What did you paste into the prompt, and what has the AI put into your test data? What to look for: Real customer names, emails or account details copied from production into a prompt API keys, connection strings or internal URLs in pasted logs or config Generated test data that looks like real personal data and might end up in shared environments or screenshots Quick tip: anonymise before you prompt, and use clearly fake data in tests (for example, addresses on a reserved domain like example.test). Check your organisation's policy on which AI tools are approved for which kinds of data. Record what you verified If a test case was drafted by AI and reviewed by you, that should be visible. It protects you, helps your team understand how much to trust the suite, and matters a great deal in regulated environments where you may need to show your working. What to look for: test cases with no indication of where they came from, who checked them, or what was changed. Quick tip: keep it light. A short review note — "AI draft, reviewed against AUTH-112, two invented rules removed, three negative cases added" — is enough. You don't need an essay; you need a trail. Before and after: one test case Here's a typical AI draft for a password reset story: text TC-014: Verify password reset works Steps: Go to the login page Click "Forgot Password" Enter a valid email Click Submit Expected: User receives a reset email and the system works as expected. And the same test after review: text TC-014: Reset email is sent for a registered address Source: AUTH-112, acceptance criterion 1 Preconditions: Account exists for test.user01@example.test Test mailbox is accessible in the QA environment Steps: On the login page, select "Forgot password?" Enter test.user01@example.test Select "Send reset link" Expected: Confirmation message matches the copy in AUTH-112 Exactly one reset email arrives in the test mailbox The email contains a link to the reset page for that account Review notes: AI draft, reviewed by CB Link label corrected to match UI Invented message "Check your inbox!" removed Unregistered email and expired link split into TC-015 and TC-016 The reviewed version is traceable, uses real labels, has expected results that can fail, splits out the negative cases, and leaves a record of what changed. Wrapping up AI has made me faster at drafting tests, and I'm not going back. But speed only helps if what comes out is right. The polish on an AI draft is not evidence of quality — it's just formatting. Treat every AI-generated test case as a first draft from a confident colleague who hasn't read the spec. Trace it, question it, tighten it, and write down what you checked. That's where the actual testing happens. AI drafts. You verify. I've put together a kit with 38 QA prompts and a 15-point checklist for this: https://trustworthy-ai-testing.lovable.app/