Senior · Test Strategy Technique

Risk-Based Prioritisation

You can't test everything equally. Risk-based prioritisation tells you where to aim first, how deep to go, and when to stop.

Senior ISTQB CTAL-TA 2.3 (Risk-Based Testing) — K4 Analyse ~12 min read + exercise

1 The Hook — Why This Matters

In 2021, a Wellington fintech startup shipped a major update to their lending platform. The team had two weeks to test. They spent the first week on UI polish and form validation, leaving only three days for payment integration. On launch day, POLi transfers above $10,000 silently failed. No error message reached the user. The startup processed $2.3 million in loans that never actually transferred.

The root cause wasn't a lack of testing. It was equal weighting. The team treated "wishlist icon alignment" with the same urgency as "bank transfer settlement." A risk-based approach would have flagged payment flows as exposure 25 (likelihood 5, impact 5) and allocated the deepest testing there first.

Time is always finite. Risk tells you where to spend it.

2 The Rule — The One-Sentence Version

Test effort must be proportional to risk exposure, which is likelihood multiplied by impact.

A feature that is complex and business-critical deserves deep, multi-layered testing. A feature that is simple and low-impact deserves light testing or none at all. Equal effort across both is not fairness. It's negligence.

3 The Analogy — Think Of It Like...

Analogy

A hospital triage nurse on a Saturday night.

A patient with chest pain goes straight to resus. A patient with a sprained ankle waits. The nurse doesn't assess everyone for the same length of time. They assess urgency and severity, then allocate resources accordingly. Risk-based testing is triage for your product. If you spend twenty minutes on the ankle while the chest pain patient waits, people get hurt.

4 Watch Me Do It — Step by Step

Here is a real NZ e-commerce risk register. Follow these steps to build your own.

Identify risks across four categories Brainstorm functional risks (user journeys), technical risks (integration complexity), business risks (revenue, reputation), and regulatory risks (compliance, data privacy).
Assess likelihood and impact on a 1–5 scale Likelihood: how probable is a defect here? Impact: what happens if one slips through? Be specific. Use historical defect data where you have it.
Calculate risk exposure: Likelihood × Impact This gives you a score from 1 to 25. It is a prioritisation input, not a prophecy.
Map exposure to test effort
- 20–25 (Deep): exhaustive functional, edge case, regression, and exploratory testing
- 10–16 (Moderate): standard functional and regression testing
- 1–9 (Light / None): smoke test or skip, depending on capacity
Review and adapt throughout the project Risk is not static. New code, new requirements, and defect clusters change the picture. Update the register at sprint boundaries.

NZ e-commerce risk register

Feature area	Likelihood (1–5)	Impact (1–5)	Exposure (L×I)	Test effort
Payment (POLi/Stripe)	5	5	25	DEEP
GST calculation	4	3	12	MODERATE
Address validation	4	3	12	MODERATE
Product search	2	3	6	LIGHT
Wishlist	1	2	2	NONE

Pro tip: In New Zealand, health apps must consider HISF (Health Information Security Framework) compliance risks. Financial systems must consider RBNZ FMI (Financial Market Infrastructures) requirements. Regulatory risk often has an impact of 5 regardless of likelihood, because a breach can shut down the business.

Pro tip: Use a risk matrix heat map to make the register visual. Involve the whole team in scoring — developers often know technical likelihood better than testers do. Link risk scores to exit criteria: "No open defects in exposure 20+ areas before release."

Second Example: Christchurch Insurance Broker App

A Christchurch-based insurance broker is releasing a major update: new policy quote engine, integration with IRD for income verification, and a customer dashboard redesign. The team has identified the following risks:

Area	Likelihood	Impact	Exposure	Test Effort
Quote engine calculation (insurance premium logic)	4	5	20	DEEP — exhaustive; test edge cases (high income, dependants, pre-existing conditions); verify calculation accuracy against actual quotes from competitors
IRD income verification (automated data sync)	5	5	25	DEEP — integration testing with IRD test harness; verify end-to-end data flow; test timeout and retry logic for network failures; verify no customer data is cached insecurely
Dashboard redesign (UI only)	2	2	4	LIGHT — smoke test across browsers; check basic responsive behaviour

Rationale: The quote engine and IRD integration are both high-likelihood and high-impact. A calculation error means the broker issues incorrect policies, creating legal and compliance liability. IRD integration failures could lock customers out of the application. The dashboard redesign is purely cosmetic; if it breaks, users can still access quotes and policies through the old interface.

5 When to Use It / When NOT to Use It

✅ Use risk-based prioritisation when...

Time or budget is constrained (which is always)
You are testing a new feature with unknown stability
The system has regulatory or compliance requirements
You need to justify test coverage to stakeholders
Defect history shows clustering in specific modules
You are defining entry and exit criteria for a release

❌ Don't use it when...

The project is tiny and every feature is equally critical
You have no access to business or technical context
The team uses it as an excuse to skip testing entirely
You treat the first risk register as final and never revisit it
Scores are assigned by one person without team input

⚠ When It Fails

Risk-based prioritisation fails when low-exposure features cause high-impact failures. You skipped testing the login reset flow (exposure 6) and a user locked themselves out of their $50,000 investment account. It also fails when risk scoring is too subjective or outdated. A module you marked as low-risk has shipped three critical defects in the past two quarters, but you're still allocating it light testing. Re-validate your register every sprint using actual defect data. If predicted risk and actual defects diverge, update the scoring before making test allocation decisions.

📋 Decision Checklist: Should You Use Risk-Based Prioritisation?

Do you have limited testing time or budget? (Almost always yes.) Risk-based prioritisation helps you allocate effort where it matters most.
Can you access business and technical context? You need developers, product owners, and compliance leads to score likelihood and impact accurately.
Will you update the risk register throughout the project? If you create a register in week one and ignore it for twelve weeks, you'll miss emerging risks and defect clusters.

6 Common Mistakes — Don't Do This

🚫 Equal weighting everything

I used to think: Every feature deserves the same attention. It's fairer that way.
Actually: Equal effort means your payment flow gets the same three test cases as your "About Us" page. Risk is not about fairness. It's about probability and consequence. A defect in POLi processing costs revenue and trust. A defect in the wishlist costs almost nothing.

🚫 A static risk register that never updates

I used to think: We made the risk matrix in sprint zero. Job done.
Actually: Risk changes as code changes. A module that was stable in sprint two may become risky in sprint six after a major refactor. A module with zero historical defects may suddenly produce a cluster. Review the register every sprint. Treat it as a living document.

🚫 Only scoring technical risk

I used to think: Complex code = high risk. Simple code = low risk.
Actually: A one-line GST rounding rule is technically trivial, but a $0.01 error in every transaction is a business catastrophe. You must score business impact and regulatory exposure separately from technical complexity. Involve product owners and compliance leads in scoring sessions.

⚠ When It Fails

Risk-based prioritisation fails when low-exposure areas mask high-consequence bugs. You scored address validation as exposure 6 and tested it lightly. In production, an edge case (PO Box with no street number) caused all rural claims to be routed to the wrong office. It also fails when the risk register becomes obsolete. You built it in sprint zero and never updated it. By sprint eight, the payment module has a history of three critical defects, but you're still allocating it moderate effort because that's what the old register says. Review and re-score every two sprints based on actual defect data.

7 Now You Try — Build a Mini Register

🎯 Interactive Exercise

Scenario: You are testing a NZ health appointment booking app. You have one week. Identify the test effort for each area using L×I scoring.

Patient data sync with GP systems (HISF regulated)
Appointment reminder SMS
Clinic profile photo upload

Assign likelihood and impact scores, then reveal the suggested register.

Suggested risk register:

Feature area	Likelihood	Impact	Exposure	Effort
Patient data sync	4	5	20	DEEP
Appointment SMS	3	2	6	LIGHT
Clinic photo upload	2	1	2	NONE

Why: Patient data sync is HISF-regulated. A breach or data loss has regulatory and reputational impact of 5. SMS reminders are useful but easily fixed if broken. Photo uploads are cosmetic. Spend your week on data sync.

8 Self-Check — Can You Actually Do This?

Click each question to reveal the answer. If you got all three, you're ready for the interview prep.

Q1. What is the difference between product risk and project risk?

Product risk is the risk that the software will fail to meet user or business needs (defects, poor performance, security holes). Project risk threatens the delivery itself — schedule slip, skill gaps, tool failures. Risk-based testing focuses on product risk, but project risk can amplify it (e.g., time pressure increases the likelihood of defects).

Q2. A feature scores likelihood 2 and impact 5. Another scores likelihood 5 and impact 2. Both have exposure of 10. Should they get the same test effort?

Not necessarily. Exposure is a guide, not a rule. The high-impact / low-likelihood feature (2×5) might warrant deeper testing because the consequence of a miss is severe. The high-likelihood / low-impact feature (5×2) might be cheaper to fix if it fails. Use exposure as a starting point, then apply judgment based on context and fix cost.

Q3. Why should predicted vs actual defect cluster analysis inform the next risk register?

Because risk assessment is a prediction, and predictions improve with data. If you predicted high defects in module A but found none, while module B produced a cluster you didn't anticipate, that tells you your likelihood model was wrong. Update the register with this evidence before the next cycle.

9 Interview Prep — Questions You Might Get

Click each question to reveal a strong answer structure.

Q1. How do you handle a stakeholder who wants everything tested equally?

Show them the risk matrix. Explain that equal effort means a critical payment flow receives the same attention as a cosmetic wishlist icon. Use historical defect data to demonstrate where bugs actually cluster. Frame it as resource protection: "We have finite time. Risk-based prioritisation ensures the features that could damage revenue or compliance get the deepest coverage first." Offer to review the register together so they feel ownership of the scores.

Q2. What NZ regulatory frameworks affect risk scoring in software testing?

For health applications, HISF (Health Information Security Framework) governs data security and privacy. For financial systems, RBNZ FMI (Financial Market Infrastructures) sets resilience and continuity expectations. Both frameworks can force an impact score of 5 regardless of technical likelihood, because a compliance breach can result in loss of licence or operational shutdown.

Q3. How do you communicate residual risk to a project manager?

Residual risk is the risk that remains after testing. I communicate it explicitly: "We have tested the payment flow deeply. Residual risk is low-medium. We have done light testing on the photo upload feature. Residual risk is medium-high." I document it in the test summary report with the risk register attached, so the PM makes an informed go/no-go decision rather than assuming everything is equally covered.

Q4. When would you skip risk-based prioritisation and test everything exhaustively?

Almost never, because exhaustive testing is theoretically impossible for non-trivial systems. The only scenario close to it is safety-critical embedded systems where failure means loss of life, and even then engineers use model checking and formal verification, not manual exhaustive testing. For commercial software, risk-based prioritisation is the professional standard. The real question is how granular and evidence-driven your risk model is.

← All Senior learning Full reference (ISTQB deep-dive) → Defect Analysis →