Test Planning
The test plan is the contract between testing and the project. It says what will be done, by whom, by when, and what "done" actually means.
1 The Hook — Why This Matters
In 2022, a Dunedin healthcare provider launched a patient portal that let users book appointments, view test results, and message their GP. The project had a test strategy. It had testers. It had Jira. What it did not have was a test plan with clear entry and exit criteria.
System testing started before the test environment was stable. Smoke tests failed for four days, but testing continued anyway because "we were behind schedule." Defects were found, fixed, and retested in a chaotic loop with no defined resumption criteria. When the release date arrived, the team had no idea whether the product was ready. They shipped anyway.
Within 48 hours, patients saw other patients' test results. The privacy breach was reported to the Privacy Commissioner. The portal was taken offline for three weeks. The strategy was sound. The planning was absent. A test plan turns intent into accountable execution.
Senior engineer insight
The shift that changed how I write test plans happened when a programme director asked me to justify my exit criteria in a steering committee meeting — in front of the CTO. I realised I had no objective basis for "95% pass rate": I had copied it from a previous project. Now I derive every threshold from historical data: actual defect density on comparable releases, environment stability metrics, mean time to fix. When you can say "this exit criterion is based on three prior releases that shipped without P1 incidents," stakeholders stop negotiating and start trusting the gate.
The most common mistake Test Lead testers make: writing test plans after planning has already happened, then reverse-engineering the document to match a timeline that was never theirs to control.
From the field
On a New Zealand government digital services programme — a high-profile Benefits NZ client-facing rebuild — the team assumed that having a TMMi Level 2 process framework meant test planning was covered. It was not. The TMMi artefacts described what a test plan should contain; nobody had agreed who had authority to declare entry criteria met. When the build arrived with 40% of smoke tests failing, three senior testers continued executing functional tests because "we'd already started" and the plan had no suspension trigger. The project governance board only discovered this at the weekly checkpoint, five days later, after 200 test hours had produced results against an unstable build. The fix was simple: a one-page RACI appended to the plan, naming the Test Lead as the sole person with authority to suspend or resume. That change alone eliminated two similar incidents in the next release cycle — and became a standard requirement across all TMMi-aligned programmes on that account.
2 The Rule — The One-Sentence Version
A test plan is a detailed, agreed document that defines how testing will be executed, who will do it, when it will happen, and what conditions must be met to start and stop.
It is not a wish list. It is not a schedule in a project manager's spreadsheet. It is the binding agreement that lets you say "we are not ready to test" when the build is broken, and "we are ready to ship" when the exit criteria are met.
3 The Analogy — Think Of It Like...
A ship's captain filing a voyage plan with the harbourmaster.
The voyage plan says: departure time, route, crew roster, fuel check, weather thresholds for delay, and the conditions under which the captain will turn back. It doesn't say "sail well." It says "if wind exceeds 40 knots, we do not leave the harbour." The harbourmaster and the captain both sign it. A test plan is the same: it defines the route (schedule), the crew (resources), the fuel (test data), the weather thresholds (entry criteria), and the turn-back conditions (suspension criteria). Without it, you are sailing blind.
4 Watch Me Do It — Step by Step
Here is how to build a test plan for an NZ healthcare patient portal. Follow these steps for any release you lead.
- Define scope and objectives What features are in this release? What are the quality objectives? Be specific: "Enable appointment booking, result viewing, and GP messaging with zero critical defects at launch."
- Build the Work Breakdown Structure (WBS) Decompose testing into tasks with estimates. Example: planning (2d), environment setup (3d), functional testing (10d), regression (3d), UAT (5d), defect retest (3d), reporting (1d) = 27 days.
- Set entry criteria The conditions that must be true before system testing begins. Code complete. Unit tests pass. Smoke tests pass. Test environment ready. Test data available and anonymised. Requirements signed off.
- Set exit criteria The conditions that must be true to release. All P1/P2 defects resolved. Defect Detection Percentage (DDP) > 90%. No new critical defects in 72 hours. Performance benchmarks met. Regression suite green. UAT sign-off received.
- Define suspension and resumption criteria When do you stop testing? When the environment is down for more than four hours, or when a blocking defect stops all progress. When do you resume? When the environment is stable and the blocker is resolved and deployed.
- List deliverables, dependencies, and risks Deliverables: test plan, test cases, defect reports, coverage metrics, release certificate. Dependencies: environment provisioning, third-party API availability, test data from BI team. Risks: environment instability, late feature drops, defect volume higher than forecast.
| Criterion | Detail |
|---|---|
| Entry: build | Deployed from CI/CD pipeline with green unit-test stage |
| Entry: smoke | All smoke tests pass in the target environment |
| Entry: data | Test data anonymised and validated by privacy officer |
| Exit: defects | All P1/P2 resolved; P3/P4 have accepted workarounds |
| Exit: coverage | DDP > 90%; functional coverage mapped to requirements |
| Exit: stability | No new critical defects in 3 consecutive days |
Audit-Proofing Your Test Collateral
In regulated sectors (Fintech, Healthcare, Govt), your test plan and results must stand up to an external audit. To ensure your collateral is audit-ready, include these "Proof of Quality" steps:
- Traceability Matrix: A clear map linking every requirement/story to a specific test case and its final result. If an auditor asks, "How do you know Requirement X was tested?", you point to this matrix.
- Evidence Retention: Don't just record "Pass". Store timestamped screenshots, API logs, or database snapshots for critical business flows. Auditors look for contemporaneous evidence.
- Immutable Sign-offs: Ensure all exit criteria sign-offs are recorded in a way that shows who approved it and when (e.g., an e-signature, a locked Jira comment, or a PDF summary).
- Deviation Logs: If you ship with known issues (P3/P4), document the business justification for the exception. An auditor cares less about the bug and more about the decision-making process used to accept it.
5 When to Use It / When NOT to Use It
✅ Use a formal test plan when...
- A release has multiple features and dependencies
- Stakeholders need visibility into test progress
- Entry and exit criteria must be agreed before testing
- Multiple testers need coordinated task assignments
- Compliance or audit requires documented planning
❌ Don't over-plan when...
- The team uses pure continuous delivery with full automation
- A single hotfix needs one hour of verification
- The plan is written after testing has already started
- The plan is so rigid it cannot adapt to scope changes
- No stakeholder will read or enforce it
Before you write a test plan, ask:
- Are requirements stable enough to plan against, or are they still fluid?
- Will this plan be reviewed and enforced, or is it a compliance document nobody reads?
- Is the scope fixed or flexible? (Rigid plans are worthless in Agile environments.)
- Who is the audience—testers, management, auditors, or all three?
6 Common Mistakes — Don't Do This
🚫 Planning in isolation
I used to think: the Test Lead writes the plan and the team follows it.
Actually: A plan written without developer input misses environment constraints. A plan written without product input misses priority shifts. A plan written without tester input misses realistic estimates. Involve the whole team in planning. Ownership produces commitment.
🚫 No buffer time and optimistic estimates
I used to think: if everything goes smoothly, we can finish in ten days.
Actually: Nothing goes smoothly. Environments fail. Builds break. Requirements change. A plan built on best-case estimates is a plan built on hope. Use three-point estimation for uncertain tasks, add contingency for retest cycles, and track actuals against estimates to calibrate future plans.
🚫 No clear exit criteria
I used to think: "all tests pass" is a good exit criterion.
Actually: "All tests pass" ignores defect severity, coverage gaps, and performance. Good exit criteria are multi-dimensional: defect thresholds, coverage targets, stability windows, and stakeholder sign-off. Define them before testing starts, or you will argue about readiness under release pressure when everyone is tired and biased toward shipping.
When this technique fails
Test planning fails when it is treated as a ceremonial document written once and forgotten. It also fails when exit criteria are vague or absent: you ship based on feeling, not objective gates. Finally, it fails when the plan is so rigid that scope changes invalidate it: a waterfall plan in an Agile project creates friction, not value.
7 Now You Try — Plan Review Exercise
Scenario: A test plan for an NZ retail website's holiday sale release includes: functional testing (5d), regression (2d), UAT (3d). Total: 10 days. There is no buffer, no retest cycle, no suspension criteria, and the exit criterion is "all tests pass."
Question: List three specific problems with this plan and what you would change as the Test Lead.
Three problems and fixes:
- No retest cycle. Defect fixes need verification and regression. Budget 30% of test time (3 days) for retest.
- No buffer or contingency. Holiday releases are high-pressure and high-defect. Add 20% buffer (2 days) for environment issues and late drops.
- Weak exit criteria. "All tests pass" ignores severity and coverage. Replace with: 0 critical open, 95% test cases executed, no new critical defects in 48 hours, performance benchmarks met, and product owner sign-off.
A 10-day plan becomes a 15-day plan with integrity. A Test Lead who defends realistic planning protects both the team and the release.
Why teams fail here
- Exit criteria are written to be achievable rather than meaningful — "100% test execution" gets hit on day one because the team excludes blocked tests from the denominator.
- The test plan is authored by the Test Lead in isolation and never reviewed by developers, product, or the business — so estimates are fantasy and scope assumptions are wrong before testing begins.
- Suspension criteria are absent, so testing continues on a broken environment and the data produced is worthless — contaminating defect counts and coverage metrics that the release decision will rely on.
- The plan is treated as a one-time deliverable: written in Week 1, never updated after scope changes in Week 3, and then used to sign off a release it no longer accurately describes.
Key takeaway
A test plan earns its authority not from the template it follows, but from the evidence behind every threshold — the Test Lead who can defend each criterion with data is the one stakeholders listen to when it matters most.
8 Self-Check — Can You Actually Do This?
Click each question to reveal the answer. If you got all three, you're ready for practice.
Q1. What are good entry criteria for system testing?
Code complete; unit tests pass; smoke tests pass in the target environment; test environment is stable and matches production configuration; test data is available, validated, and privacy-compliant; requirements or user stories are signed off and traceable to tests.
Q2. Why should exit criteria be defined before testing starts?
Because exit criteria defined under release pressure become negotiation points, not quality gates. When the deadline is tomorrow and everyone is tired, "all tests pass" becomes "well, most tests pass." Pre-defined, agreed criteria remove emotion from the release decision and give the Test Lead objective grounds to delay if quality is insufficient.
Q3. What percentage of total test time should typically be reserved for defect retest cycles?
Typically 25–35% of total test time, depending on historical defect density and team maturity. A team with high code quality and fast feedback loops may need less. A team with frequent integration issues or large legacy codebases may need more. The key is to use your own historical data rather than guessing.
9 Interview Prep — Lead-Level Q&A
Q. What are good entry criteria for system testing?
Code complete and merged; unit tests passing; smoke tests green in the target environment; test environment stable and configuration-matched to production; test data available, validated, and privacy-compliant; requirements or user stories signed off and traceable to test cases. I also like to see a risk register review complete so the team knows where to focus first.
Q. How do you handle a project manager who wants to cut two days from your test plan?
I ask what they want to cut: coverage, retest time, or buffer? I present the consequences of each. If we cut coverage, these features go untested. If we cut retest time, verified fixes may regress. If we cut buffer, any environment issue becomes a delay. I then propose alternatives: parallel testing streams, risk-based scope reduction (test high-risk features deeply, accept shallow testing on low-risk features), or moving non-critical tests to post-release monitoring. The goal is a conscious trade-off, not an arbitrary cut.
Q. What do you include in a test plan that junior leads often forget?
Suspension and resumption criteria; a clear RACI for who can override exit criteria; dependency mapping with named owners and fallback plans; historical velocity data to justify estimates; and a communication plan (who gets what report, how often, and in what format). Juniors often focus on "what we test" and forget "what stops us testing" and "who decides when we're done."
Q. How do you adapt a test plan when scope changes mid-sprint?
I reassess the WBS with the team: what drops out, what adds in, and what is the impact on estimates and exit criteria. I update the risk register because scope changes introduce new risks. I communicate the revised plan, the new timeline, and any coverage gaps to stakeholders in writing. A plan that cannot adapt is a plan that becomes obsolete. The discipline is not rigidity; it is transparent replanning.