From Spec to Suite: An AI-Powered Test Generation Workflow That Actually Works

The workflow that works is not "paste code into ChatGPT and ask for tests." It is a staged pipeline: extract testable requirements from the spec first, generate test skeletons against those requirements before implementation exists, then use an AI coding agent with repo context and a run-loop (generate, execute, read failures, fix) to reach passing, meaningful coverage. Skipping the spec-extraction step is the single biggest reason generated suites end up shallow.

Why "just ask for tests" fails in practice

If you point an LLM at a 400-line service class and say "write unit tests," you get tests that mirror the implementation, not the intent. The model reads if user.age > 18 and writes a test for age > 18, but it has no idea the business rule is actually "18 in the US, 21 in some states, configurable per tenant." The code is a compressed, lossy encoding of the spec. Generating tests from code alone just re-encodes the same bugs as green checkmarks.

I have watched teams ship AI-generated suites with 90%+ line coverage that still missed the actual regression that shipped two weeks later, because the missed case was a business rule that existed only in a Jira ticket and a Slack thread, never in the code path the model could see. Coverage percentage is a code-shape metric, not a correctness metric. Treat it as a floor, not a goal.

Step 1: Turn the spec into a requirements table

Before any test code gets written, feed the AI the spec (PRD, RFC, API doc, ticket description, whatever exists) and ask it to produce a structured requirements table, not tests yet. This forces the ambiguity in the spec to surface early, when it is cheap to fix, instead of during code review.

1Prompt:
2You are extracting testable requirements from a product spec.
3For each requirement, output: ID, description, inputs, expected
4behavior, edge cases implied but not stated, and a confidence
5rating (explicit in spec / inferred / assumed).
6Do not write test code. Flag any requirement where two readings
7are plausible.
8
9Spec:
10<paste spec text>

The "flag ambiguity" instruction matters more than any other line in that prompt. Models default to confident, singular interpretations of ambiguous requirements. Forcing them to surface the ambiguity turns a silent guess into a visible question you can route to the product owner or the original ticket author.

I keep this table in the PR description or a linked doc, not buried in a prompt history, because it becomes the audit trail when someone asks "why does this test exist."

Step 2: Generate test skeletons before implementation, when possible

If you are doing anything close to spec-first or TDD-adjacent development, generate the test skeletons (names, arrange/act/assert structure, empty assertions) from the requirements table before the implementation is written. This is the highest-use moment for AI assistance because the model has no implementation to anchor on and is forced to reason from behavior.

1def test_discount_applies_when_cart_total_exceeds_threshold():
2 # Requirement R-14: 10% discount on carts over $100 (pre-tax)
3 # TODO: implement once threshold config is finalized
4 pass
5
6def test_discount_does_not_apply_to_gift_cards_in_cart_total():
7 # Requirement R-14a (inferred edge case, confidence: assumed)
8 pass

Leaving pass and TODO comments is deliberate. It gives you a checklist and a diff surface: when the implementation lands, every skeleton either gets filled in or explicitly deleted with a reason, and that deletion shows up in code review.

Step 3: Fill in implementation-aware assertions with an agentic loop

Once code exists, switch to an agent that can read the repo, run the test command, and iterate on failures rather than a single-shot chat completion. This is where tools like GitHub Copilot's coding agent, Claude Code, and Cursor's agent mode earn their keep, because filling in a real assertion value often requires actually running the code path and observing output, not guessing what a function returns.

A concrete loop that works well with Claude Code or a similar CLI agent:

11. Agent reads requirements table + skeleton test file
22. Agent fills in one test body, runs it
33. On failure: agent reads stack trace, decides if the test
4 assertion is wrong or the implementation is wrong
54. Agent reports which one it believes and why, waits for
6 confirmation on anything touching business logic
75. Repeat per test, batch commit

Step 4 is the part teams skip and regret. An agent that silently "fixes" a failing test by loosening the assertion is worse than no test at all, because it produces a green build that asserts nothing. Configure the agent's instructions (a CLAUDE.md, .cursorrules, or Copilot custom instructions file) to explicitly forbid weakening assertions without flagging it.

1## Test generation rules
2- Never change an assertion's expected value to make a test
3 pass unless you also cite the requirement ID that justifies
4 the new value.
5- If a test fails against current code, default assumption is
6 that the code is wrong, not the test. State this explicitly
7 before changing either.
8- Do not delete a test skeleton; mark it SKIP with a reason.

What about mutation testing and coverage checks?

Line and branch coverage tell you what ran, not what was checked. Mutation testing (tools like Stryker for JS/TS, mutmut for Python, PIT for Java) deliberately breaks your code in small ways and checks whether any test fails. If a mutant survives, some line executed during a test but nothing asserted on its outcome, which is exactly the failure mode AI-generated tests fall into when they were built by pattern-matching on existing code shape rather than reasoning from requirements.

Running full mutation testing on every PR is usually too slow (mutation runs typically take multiples of your normal suite runtime, and on a mid-size codebase that can mean 20 to 40 minutes versus a 3-minute unit run). A workable compromise: run mutation testing only on files touched in a PR, in CI, as a non-blocking report, and treat a low mutation score on new code as a signal for human review rather than an automatic gate.

Do AI-generated tests actually catch regressions later?

This is the question that matters more than initial pass rate, and it is harder to answer than "did the suite pass." My experience across several mid-size codebases (roughly 50k to 300k lines) is that AI-generated tests catch regressions well when they were generated from the requirements table method above, and catch regressions poorly when generated purely from reading implementation code, because implementation-derived tests tend to encode the current behavior as correct by definition, bugs included.

A cheap sanity check before trusting a generated suite: revert the actual bug fix from a recent postmortem and see if any generated test fails. If none do, the suite has a coverage gap in exactly the area that has already burned you once, which is the area most likely to burn you again.

Cost and latency realities worth planning around

Generating a full test file for a moderately complex module (say, 200 to 400 lines with several branching paths) through an agentic tool typically involves multiple tool calls: reading the file, reading related files for context, running tests, reading failures, revising. In practice this can take two to six minutes wall-clock and, depending on model and context size, cost anywhere from a few cents to around a dollar in API usage on models like Claude Opus or GPT-4 class models, versus a fraction of a cent on faster, cheaper models that produce noticeably shallower tests. If you are generating tests across a large legacy codebase in one pass, budget for this multiplicatively, not linearly, because context windows fill up with related files fast and many agents will re-read files across turns.

For CI-triggered generation (a bot that writes tests on every PR lacking coverage), rate limits and per-call latency matter more than raw quality, so a common pattern is a cheaper, faster model for the skeleton pass and a stronger model reserved for filling in assertions on business-critical paths only.

When this workflow is the wrong choice

Do not run a full spec-extraction and agentic test-generation pipeline on throwaway prototypes, internal scripts with a lifespan of weeks, or code where the spec itself is still churning daily. The requirements table becomes stale faster than you can act on it, and you end up maintaining a shadow spec document nobody reads. In that situation, a much lighter touch, quick AI-assisted test generation directly from code with a human skim, is the better trade. Reserve the full workflow for code with real blast radius: billing logic, auth, data migrations, public APIs, anything where a silent regression costs more than the time spent building the requirements table in the first place.

Putting it together on a real PR

On a recent feature (multi-tenant rate limiting with per-plan overrides), the sequence looked like this: spec extraction surfaced two ambiguous cases the original ticket never addressed (what happens when a tenant downgrades mid-window, and whether limits reset on plan change or at the next natural window boundary), which got resolved with the product owner in about ten minutes over Slack instead of during a bug report three weeks later. Skeleton tests went in in the same PR as the interface definitions, before the rate limiter itself was implemented. Claude Code filled in assertions once the implementation landed, flagged one case where it believed the test skeleton's expected value was wrong given a later clarification, and that flag turned out to be correct. Mutation testing on the touched files ran in CI as a non-blocking report and showed one survived mutant in an edge case around clock skew, which prompted one more test by hand.

None of that is exotic. It is the same discipline good engineers applied before AI tools existed, spec, skeleton, implement, verify, just with an agent doing the mechanical parts of steps two and four fast enough that skipping them stops being tempting.