Risk-Based Testing
Time is always limited. Risk-based testing uses risk exposure to decide what to test first, how deeply to test it, and what to deprioritise when time runs out.
1 The Hook
A Wellington council ships a new online rates portal two weeks before the annual rates instalment is due. The team had four weeks of test time and a backlog twice that size, so they tested in the order the tickets happened to be filed: help text first (easy wins), then the FAQ, then login, and they ran out of time before they got to the rate-calculation engine.
On go-live day the portal looked polished. Then ratepayers started phoning: the new calculation code applied last year’s rating valuation to this year’s charges, over-billing thousands of households. The most dangerous part of the system — brand-new code that touches every ratepayer’s money — got the least testing, because nobody had ranked the work by how much damage a failure would cause.
This is the pattern behind a huge share of public quality failures: limited test time spent evenly, or worse, spent on whatever was easiest, instead of on what would hurt most if it broke. The team had the hours to test the calculation engine — they just spent them on the FAQ.
2 The Rule
You never have time to test everything — so rank every area by risk exposure (likelihood × impact) and spend your test effort top-down, testing the highest-exposure areas first, deepest, and earliest.
3 The Analogy
A WOF inspection at a VTNZ station.
A vehicle inspector has a fixed slot to check your car. They do not spend it polishing the wing-mirror glass and admiring the paint. They go straight for the things that kill people if they fail: brakes, steering, tyres, seatbelts. A cracked tail-light gets a note; worn brake pads stop the car passing. The inspector is ranking checks by consequence — how likely is this to fail, and how bad is it if it does — and spending the limited time where a miss matters most.
Risk-based testing is running your test plan like a WOF inspector runs the bay: brakes before bodywork. The high-exposure areas get the deep checks; the cosmetic ones get a glance, and if the clock runs out they are the things you can defend leaving untested.
What it is
Risk-based testing (RBT) is an approach to test planning where you use risk assessment to prioritise test activities. The areas with the highest risk get the most test effort; low-risk areas get less or none. This is how experienced leads make defensible decisions about coverage when there isn’t time to test everything.
Product risk vs project risk
- Product risk — risks that the software will fail to meet stakeholder needs. These become test targets. Examples: payment processing errors, data loss, security vulnerabilities.
- Project risk — risks that affect the testing process itself. Examples: late delivery of test environments, incomplete requirements, team capacity issues.
Risk-based testing primarily addresses product risk. Project risk is managed in the test plan.
Risk exposure = likelihood × impact
Risk exposure is the product of two factors:
- Likelihood — how likely is this to fail? (technical complexity, code churn, new developer, known defect history)
- Impact — what’s the consequence if it fails? (financial loss, user harm, reputational damage, regulatory breach)
| Feature area | Likelihood (1–5) | Impact (1–5) | Exposure | Priority |
|---|---|---|---|---|
| Payment processing | 3 | 5 | 15 | High |
| User authentication | 2 | 5 | 10 | High |
| Product search | 4 | 3 | 12 | Medium |
| Wishlist feature | 2 | 2 | 4 | Low |
| Email preferences | 1 | 1 | 1 | Very Low |
Building a risk register
- Identify risks — brainstorm with stakeholders, developers, and the team. Review defect history. Check industry-specific risks.
- Assess likelihood — consider: code complexity, number of changes, developer experience, module age, integration points.
- Assess impact — consider: user importance of the feature, business criticality, regulatory requirements, number of users affected.
- Calculate exposure — multiply the scores. Sort descending.
- Map to test effort — high risk = deep testing + automation + early testing. Low risk = light testing or none.
How risk drives test decisions
- Test order — test high-risk areas first so critical bugs are found early.
- Test depth — high-risk areas get full EP/BVA/decision table coverage. Low-risk areas may get smoke tests only.
- Automation priority — high-risk, frequently run paths should be automated first.
- Exit criteria — if time runs out, you can demonstrate that high-risk areas were fully tested and low-risk areas were consciously deprioritised.
ISTQB mapping
| Ref | Topic | Level |
|---|---|---|
| CTFL 5.2 | Risk management overview — product vs project risk | Foundation |
| CTAL-TM Ch. 2 | Risk-based testing — full framework, risk register, mitigation | Advanced / Lead |
| CTAL-TA 2.3 | Risk-based testing tasks for the Test Analyst | Advanced / Senior |
Practice this technique: Try Test Lead Practice 02 — Risk-based prioritisation, Senior Practice 09 — Cross-level regression.
NZ example — risk register for a NZ e-commerce platform
For a New Zealand e-commerce platform, a practical risk register might include:
- Payment processing via POLi (NZ bank-to-bank payment) — High likelihood if newly integrated, High impact (revenue loss, customer trust) = exposure 25, test first and deepest.
- NZ address validation (RD addresses, apartment formats) — Medium likelihood (complex input rules), High impact (failed deliveries) = exposure 15.
- GST calculation correctness — Low likelihood (stable code), High impact (tax compliance breach, IRD penalty) = exposure 10.
- Product image display — Low likelihood, Low impact = exposure 4, test last or not at all if time is cut.
Risk-based testing means: when the sprint is shortened, you can show the stakeholder exactly which risks were fully tested and which were consciously deferred. “We tested payment and address validation; we deferred product image testing — here is the residual risk.”
4 Now You Try
Three graded exercises — spot, fix, then build. Write your answer, run it for AI feedback, then compare to the model answer.
A NZ health-board patient portal has these four feature areas. For each, assign a likelihood (1–5) and impact (1–5), calculate exposure, and rank them in test order. Areas: (a) prescription repeat ordering — new code, touches medication; (b) appointment booking — mature code, high traffic; (c) news/notices page — static content; (d) login via RealMe — external dependency, gates all access.
Show model answer
A defensible ranking (exact scores can vary, the reasoning matters): - Prescription repeat: Likelihood 4 (new code) x Impact 5 (patient safety, medication) = 20 — test FIRST and deepest. - Login (RealMe): Likelihood 3 (external dependency) x Impact 4 (blocks all access if broken) = 12 — test early; mature dependency lowers likelihood. - Appointment booking: Likelihood 2 (mature code) x Impact 4 (high traffic, but a missed booking is recoverable) = 8. - News/notices: Likelihood 1 (static) x Impact 1 = 1 — test last, or defer and document the residual risk. Test order: prescription repeat, login, appointment booking, news/notices. The key call is that new code touching medication outranks high-traffic-but-mature booking. Likelihood is driven by code churn and complexity; impact by harm if it fails. A safety-of-life consequence pushes impact to 5.
A junior tester built the risk register below for a Kiwibank-style transfers feature. It is wrong: it confuses project risk with product risk, scores impact by how hard the code is rather than the consequence of failure, and ranks a cosmetic item above a money-movement one. Rewrite it as a correct product-risk register with sensible exposure scores and test order.
Test environment is late — Likelihood 5, Impact 5, Exposure 25 (test first)
Inter-account transfer — Likelihood 2, Impact 2, Exposure 4
Dashboard colour theme — Likelihood 4, Impact 4, Exposure 16
Transaction history export — Likelihood 2, Impact 1, Exposure 2
Rewrite as a correct product-risk register:
Show model answer
Correct product-risk register (project risks removed, impact scored by consequence): - Inter-account transfer: Likelihood 3 x Impact 5 (money movement, loss and trust) = 15 — test first and deepest. - Transaction history export: Likelihood 2 x Impact 3 (users rely on records, but errors are visible and recoverable) = 6. - Dashboard colour theme: Likelihood 2 x Impact 1 (cosmetic) = 2 — test last or defer. What was wrong with the original: - "Test environment is late" is a PROJECT risk, not a product risk. It belongs in the test plan's risk section, not the product risk register that drives test coverage. Remove it from this register. - Impact was scored by how hard the code is, not by the consequence of failure. A colour theme cannot be Impact 4 — nobody is harmed by the wrong shade of teal. - The ranking was upside down: a cosmetic theme outranked moving customer money. Money movement is the highest-impact area and must be tested first.
A NZ supermarket is launching online click-and-collect with two weeks of test time. Build a short risk-based test plan: list at least four feature areas, give each a likelihood × impact = exposure score, then state for each how that exposure changes your test depth, test order, and automation priority. Finish with what you would defer if the two weeks were cut to one.
Show model answer
A strong plan for click-and-collect (scores indicative):
- Payment and order creation: Likelihood 4 x Impact 5 = 20 — deepest coverage, test first, automate the critical path.
- Stock/availability sync (shown in-store vs online): Likelihood 4 x Impact 4 = 16 — deep testing, early; over-selling damages trust.
- Collection slot booking: Likelihood 3 x Impact 3 = 9 — moderate depth, automate the happy path.
- Loyalty points on the order: Likelihood 2 x Impact 2 = 4 — light testing.
- Marketing banner on the confirmation page: Likelihood 1 x Impact 1 = 1 — smoke only.
How exposure drives the plan:
- Depth: exposure 20 areas get full EP/BVA/decision-table coverage; exposure 1 areas get a single smoke test.
- Order: highest exposure first, so critical defects surface while there is still time to fix them.
- Automation: high-exposure, frequently-run paths (payment, stock sync) are automated first because they will run on every build.
If cut to one week: defer loyalty points and the marketing banner; document the residual risk explicitly ("loyalty point accrual untested — worst case is a points correction, not a failed order"). Keep payment, stock sync, and slot booking fully tested. The whole value of the approach is being able to show stakeholders exactly what was tested and what was consciously left out.
Self-Check
Click each question to reveal the answer.
Q1: What is the difference between product risk and project risk, and which one does risk-based testing primarily address?
Product risk is the risk that the software fails to meet stakeholder needs (payment errors, data loss, security holes) — these become test targets. Project risk affects the testing process itself (late environments, incomplete requirements, capacity). Risk-based testing primarily addresses product risk; project risk is managed in the test plan.
Q2: How is risk exposure calculated, and what drives each factor?
Exposure = likelihood × impact. Likelihood is driven by technical factors — code complexity, churn, integration points, defect history, developer experience. Impact is driven by consequence — financial loss, user harm, reputational damage, regulatory breach. You multiply the two and sort descending.
Q3: A cosmetic dashboard theme is technically tricky to get right. Should it score a high impact?
No. Impact measures the consequence of failure, not how hard the code is to write. A cosmetic feature has low impact regardless of difficulty, because nobody is harmed if it fails. Confusing difficulty with impact is a classic scoring mistake that inflates low-value work up the test order.
Q4: Beyond test order, name two other test decisions that risk exposure should drive.
Test depth (high-risk areas get full EP/BVA/decision-table coverage; low-risk areas get smoke tests or none) and automation priority (high-risk, frequently run paths are automated first). It also informs exit criteria — you can demonstrate the high-risk areas were fully tested.
Q5: What is the practical payoff of a risk register when the schedule is cut?
It lets you make a defensible, transparent call about coverage. You can show stakeholders exactly which high-risk areas were fully tested and which low-risk areas were consciously deferred, along with the residual risk — instead of cutting tests at random and hoping.