Smoke Testing
A quick pass to confirm the build is worth testing. Walk the most critical paths — load, navigate, submit. If smoke fails, stop and send the build back.
1 The Hook
A Wellington fintech ships a new build to its test environment at 9am. The test lead, keen to get through a packed regression plan, dives straight into editing edge-case payment scenarios. Forty minutes in, nothing is saving. After another half hour of digging, the truth lands: the build never connected to the database at all. Login was broken from the first second. Every minute since 9am was spent testing a corpse.
The whole morning was wasted because nobody spent two minutes confirming the obvious first: does the app even start, can a user log in, does the main page load? Those are not deep tests. They are vital signs. And had anyone checked them, the build would have gone straight back to the developers at 9:02, not 10:30.
This is the trap a smoke test is designed to stop. Deep testing on an unstable build is worse than no testing — it produces noise, false bugs, and a wasted half-day. The cheap, shallow check up front protects the expensive, deep work that follows.
The failure mode nobody talks about: smoke passes, your team spends six hours testing, then discovers the build was pointing at last sprint’s database the entire time. Every defect you found was real — in an environment that no longer exists. I’ve seen this swallow full UAT days on NZ government projects where staging shares infrastructure with other agencies and a config swap goes unnoticed. Before you trust a passing smoke, spend sixty seconds confirming the environment itself — check the version banner, hit a health endpoint, verify the database seed date. In twenty years I’ve never regretted that minute. I’ve regretted skipping it many times.
2 The Rule
Before any deep testing, run a fast, shallow pass over the most critical paths — load, log in, navigate, complete the main flow. If smoke fails, stop immediately and send the build back; don’t test further.
3 The Analogy
The pre-flight walkaround at Wellington Airport.
Before an Pacific Air turboprop pushes back, the captain walks a circuit of the aircraft: control surfaces move, tyres have tread, no fluid pooling under the engines, pitot tubes uncovered. It takes a few minutes and it does not test the autopilot, the cabin service, or the in-flight entertainment. It answers one question — is this aircraft safe enough to fly at all? If a tyre is flat, the walkaround stops there and the plane does not leave the gate.
A smoke test is that walkaround for a build. It is wide and shallow, it checks the vital signs, and a single failure grounds the build before anyone wastes time on the deep stuff.
What it is
Smoke testing is a shallow, wide test pass that checks whether the most critical functions of a build work at all. The name comes from hardware: plug in the circuit, see if it smokes. If it does, don’t do any more testing — go back and fix it.
The goal isn’t thorough testing — it’s a fast decision: is this build good enough to test? If smoke passes, deeper testing begins. If smoke fails, the build is returned immediately.
What to cover in a smoke test
- Application launches and loads without errors
- Core navigation works (menus, links, page transitions)
- Primary user journeys complete end-to-end (login, main action, logout)
- Critical integrations respond (database connects, APIs return data)
- No crash-level errors in the browser console or server logs
Time target: 15–30 minutes maximum. If your smoke test takes longer, it’s too broad.
Real-world NZ Example: ListRight
Imagine you're testing the latest build of the ListRight iOS app. A smoke test suite would include:
- Does the app open to the Home screen?
- Can I search for "Tent"?
- Does the "Watchlist" load my saved items?
- Does the "Sell" button open the listing flow?
Smoke testing in CI/CD
In modern pipelines, smoke tests are automated and run on every deployment. They gate whether a build proceeds to further testing or gets rolled back automatically. A failing smoke in CI/CD means the deploy is blocked — no human decision required.
As a tester, you should be able to identify which tests belong in the smoke suite: high value, fast to run, deterministic.
Smoke vs sanity testing
| Dimension | Smoke Test | Sanity Test |
|---|---|---|
| Scope | Wide — covers the whole application shallowly | Narrow — focuses on a specific changed area |
| Depth | Shallow — happy paths only | Moderate — verifies specific functionality |
| When run | After every build deployment | After a specific bug fix or change |
| Goal | Is the build worth testing? | Did the fix actually work? |
| Scripted? | Usually (stable script, rerun each build) | Often ad hoc (targeted at the fix) |
What smoke testing doesn't cover
Smoke testing is deliberately shallow. It does not:
- Test edge cases, error states, or boundary values
- Validate business rules or complex logic
- Test performance, security, or accessibility
- Confirm all features work correctly
Common mistake: treating a passed smoke test as proof the application is ready to release. Smoke tests only confirm the build is stable enough to test further — not that it’s production-ready.
Practice this technique: Try Grad Practice 01 — Typos & wording, Grad Practice 05 — Navigation & footer.
4 Industry Reality
- The smoke suite is almost always out of date. In real teams, automated smoke suites are written once and rarely updated. A critical new flow ships and nobody adds it to the suite. Senior testers do a manual top-up on day one of each sprint to catch what automation misses.
- Pressure to skip it is constant. Stakeholders push to start deep testing immediately. On a Monday morning when the UAT window is tight, you will be told "just start — we don’t have time for a smoke check." A single broken-build story is usually enough to justify the 15 minutes; have that story ready.
- CI/CD smoke suites are flaky in practice. Automated pipeline checks that read clean on a laptop break in staging because of timing, seeded data mismatches, or environment differences. Expect to spend real time tuning the suite until it’s reliably green-on-green and red-on-red before you trust it as a gate.
- Legacy codebases have no clear "critical path". On a 15-year-old NZ government system, there may be 40 different entry points, three competing login mechanisms, and no documentation. Senior testers derive the smoke scope from call-centre escalation logs and incident history — not from a requirements doc that doesn’t exist.
- A passed smoke doesn’t mean what managers think it means. After a clean smoke run, stakeholders routinely assume the software is ready. You will need to correct that in writing — "smoke passed, proceeding to functional testing" — or the green result will be forwarded as a sign-off.
5 When to Use It — and When Not To
✓ Use it when
- A new build has just been deployed to any test environment — always run smoke first, before anything else
- You are gating a CI/CD pipeline — smoke is the lightweight first check before slower functional suites run
- After a hotfix in production — a quick 10-minute manual smoke confirms the patch didn’t break the core flow
- At the start of a UAT session with business stakeholders — you don’t want them wasting a morning on a broken build
- After a major infrastructure change (database migration, NZ data-centre move, cloud region switch) — critical paths need a vital-signs pass before deep testing resumes
✗ Skip it when
- You are testing a trivial content-only change (a typo fix, a colour update) — smoke is overkill; a targeted sanity check on the changed element is enough
- The automated pipeline already runs a smoke suite and it’s green — don’t duplicate the check manually unless you distrust the suite
- You are doing exploratory testing on a known-stable environment — the environment has been validated; switch straight to exploratory charters
- You are testing a backend-only change with no UI impact — a quick API health check may suffice rather than a full UI smoke run
- You only have 5 minutes before a meeting — a partial smoke is worse than no smoke; note the risk and schedule a proper run immediately after
Context guide
How the right level of smoke testing effort changes based on project context.
| Context | Priority | Why |
|---|---|---|
| Benefits NZ, Revenue NZ, or CoverNZ benefits portal — UAT before a fortnightly release | Essential | A broken login in a government payment portal wastes an entire UAT window and blocks processing for real clients. Smoke is non-negotiable before any UAT session begins. |
| CI/CD pipeline deploying multiple times daily (e.g. CloudBooks, ListRight, KiwiFirst Bank mobile app) | Essential | Automated smoke gates every deploy. Without it, a broken build reaches staging (or production) before anyone notices. Must be deterministic and under five minutes. |
| HealthNZ or TransitNZ post-hotfix verification in production hours | High | A 10-minute manual smoke after a live hotfix confirms the patch didn’t disrupt the core flow. The cost of missing a broken critical path at this stage is an incident report and public disruption. |
| Harbour Bank or Pacific Bank internet banking — major infrastructure change (cloud region migration, database failover) | High | Environment drift after infrastructure changes is invisible to the application layer. Smoke catches a misconfigured database endpoint or broken session store before functional testing embeds hours of work into an unreliable environment. |
| Spark or Vodafone consumer portal — content-only release (pricing page copy update, promotional banner) | Medium | Full smoke is likely overkill; a targeted sanity check on the changed element is faster. Reserve smoke for when the change touches application code, not just copy or images. |
| Internal admin tool with five users, no customer-facing risk (e.g. internal reporting dashboard) | Low | With a tiny user base and no public impact, a brief manual sanity check on login and core navigation is enough. Full smoke ceremony adds more overhead than value at this scale. |
Trade-offs
What you gain and what you give up when you choose smoke testing.
| Advantage | Disadvantage | Use instead when… |
|---|---|---|
| Stops wasted effort immediately — a broken build is caught in minutes, not hours | Gives false confidence if stakeholders mistake a green smoke for a release-ready signal | A targeted sanity check suffices — when the change is a single component fix and the rest of the environment is known stable |
| Fits CI/CD automation perfectly — deterministic checks gate deploys without human review | Automated suites go stale; critical new flows ship without being added to the suite, creating a false safety net | Post-deployment synthetic monitoring — when continuous production health checks are already running (e.g. a Spark or Pacific Air uptime monitor) |
| Protects business stakeholder time — UAT teams never waste a session on a fundamentally broken build | Suite creep is constant pressure — the suite grows over sprints until it takes 45 minutes, then teams skip it entirely | Pure regression testing — when the goal is proving correctness after a change, not confirming the build is alive |
| Works at any level — same concept applies to a 5-person startup app and a nationwide Revenue NZ system | Does not catch environment drift on its own — smoke can pass while the app silently points at the wrong database | Exploratory testing — when the environment is known stable and the goal is finding unknown unknowns, not confirming vital signs |
Enterprise reality
How smoke testing changes at 200–300-developer scale in NZ enterprise
- Smoke suites are fully automated and gated at the pipeline level — no human runs them manually. At CloudBooks and Pacific Air, every deployment to staging triggers a smoke run in CI; a red gate blocks promotion to production without a manually approved override, removing the "I'll just check it quickly" workaround that small teams rely on.
- Compliance frameworks turn smoke testing into an audit artefact. Under the NZ Information Security Manual (NZISM) and the Health Information Security Framework (HISF), agencies such as HealthNZ and CoverNZ must demonstrate that each release passed a documented verification check before touching production data — smoke test results are retained as evidence, not discarded after the sprint.
- Tooling shifts from ad-hoc scripts to governed platforms. Large NZ enterprises typically run smoke suites in Playwright or Cypress wired into GitHub Actions or Azure DevOps, with results published to a shared Allure or TestRail dashboard — so any squad lead, release manager, or auditor can see pass/fail history without asking the QA team.
- A failed smoke gate at enterprise scale cascades across 10+ squads simultaneously. When Revenue NZ's FIRST platform deploys a shared service layer, a broken smoke test doesn't block one team — it freezes every downstream squad waiting on that release. Organisations at this scale maintain a dedicated "smoke owner" on-call rotation precisely because the blast radius of a silent failure is measured in hours of lost developer time, not minutes.
◆ What I would do
Professional judgment — when to reach for smoke testing, when to skip it, and what to watch for.
The bottom line: Smoke testing is the cheapest insurance in a testing cycle. Two minutes confirming vital signs before deep testing begins saves hours of work on a build that was never worth testing. Always run it, always keep it short, and never let a green smoke result stand in for a release sign-off.
6 Best Practices
- ✓ Keep the smoke suite under 15 minutes, no exceptions. If it takes longer, you will start skipping it under pressure. Time-box it ruthlessly — cut anything that can wait for functional testing.
- ✓ Write it down, every time. A smoke test that lives in someone’s head is useless when they’re on leave. Maintain a simple checklist (even a shared Google Doc) so any team member can run it consistently.
- ✓ Stop and report immediately on any failure. Do not work around a failing smoke step, note it, and carry on. The rule is stop, log the failure, return the build. Every minute spent deep-testing after a smoke failure is waste.
- ✓ Cover the most-visited paths, not the most interesting ones. Senior testers pick smoke scenarios from analytics, support tickets, and business-critical transactions — not from which features are newest or most complicated.
- ✓ Include one end-to-end path, not just page loads. A common weak smoke suite checks that pages load but never completes a transaction. Always include at least one full journey: login → action → confirmation.
- ✓ Check the server and browser together. Inspect the browser console for JavaScript errors and check the server logs (or a health endpoint) for 5xx responses. Silent back-end failures are invisible to the UI but will corrupt the test data underneath you.
- ✓ Refresh the suite every sprint. New critical features should be added to the smoke suite the sprint they ship. Set a team agreement: if a feature is important enough to include in the regression suite, it’s worth a smoke entry.
- ✓ Distinguish environment failures from application failures. If smoke fails, confirm whether it’s the app or the environment (missing seed data, wrong config, a shared service down). Report which one clearly — the fix route is different.
- ✓ In CI/CD, keep smoke deterministic. Any check that sometimes passes and sometimes fails (due to timing, network, or shared state) poisons the gate. Fix or remove flaky checks before they erode trust in the entire pipeline.
- ✓ Communicate the outcome, not just the result. "Smoke passed" means something specific. Explicitly document in your test report: what was covered, what was excluded, and what the pass means for the team’s decision — not just a green tick.
7 Common Misconceptions
❌ Myth: A passed smoke test means the software is working correctly.
Reality: A smoke test only confirms the build is stable enough to test further. It deliberately skips business logic, edge cases, calculations, and security checks. A clean smoke run is permission to start proper testing — not a sign-off on quality. When a NZ health portal smoke passes at 9am, the testers still have a full day of functional work ahead of them; smoke just confirmed the platform isn’t on fire.
❌ Myth: Smoke testing and regression testing are the same thing, just faster.
Reality: They serve entirely different purposes. Regression testing verifies that existing functionality hasn’t broken after a change — it is thorough and covers wide feature sets, edge cases, and previously-reported bugs. Smoke testing is a pre-check that asks "is the build even alive?" A smoke suite might have 6 checks; a regression suite might have 600. Running them interchangeably is a false economy — you get neither the speed of smoke nor the coverage of regression.
❌ Myth: Automated CI/CD smoke suites eliminate the need for manual smoke testing.
Reality: Automated pipeline checks are valuable for catching build-level failures fast, but they test what was written months ago. Manual smoke testing at the start of a sprint catches newly introduced regressions, environment issues, and recently changed flows that haven’t been added to the automated suite yet. In most NZ teams, a hybrid approach is best: pipeline automation catches the obvious, manual smoke catches the recent. Neither replaces the other.
Senior engineer insight
The most useful shift in how I think about smoke testing is treating it as an environment contract, not just an app check. A passing smoke doesn't only mean the application started — it means the environment is coherent: the right database, the right config, the right version. On NZ government projects I've watched teams spend full UAT days finding bugs that turned out to be staging-environment drift. Smoke is your chance to catch that in five minutes.
Most common mistake: expanding the smoke suite every sprint until it takes 45 minutes to run. At that point teams start skipping it under pressure, which defeats the entire point. Keep it ruthlessly short — cut anything that isn't a vital sign.
From the field
A NZ telco I worked with had a CI/CD pipeline that ran a smoke suite on every deploy — green builds auto-promoted to staging. The suite had been written two years earlier and covered the old checkout flow. When the team shipped a new one-page checkout, nobody added it to the smoke suite. For three sprints, builds that broke the new checkout still promoted cleanly because smoke was checking paths that no longer mattered. The pipeline looked healthy; the product wasn't.
The lesson that generalises: an automated smoke suite is only as good as its last update. Treat it like production code — when a critical flow changes, the smoke suite changes with it, same sprint, same PR. If your smoke suite isn't reviewed during sprint planning, it's slowly becoming a false safety net.
8 Now You Try
Three graded exercises — spot, fix, then build. Write your answer, run it for AI feedback, then compare to the model answer.
A new build of the Revenue NZ myIR portal has landed in the test environment. From the checks below, pick the ones that belong in a smoke test and say why each one stays in or out:
(a) Home page loads without errors. (b) GST calculation is correct to the cent for a 15% rate on $1,234.56. (c) Log in with a test RealMe account succeeds. (d) The "File a return" button opens the return flow. (e) Session expires after exactly 30 minutes idle. (f) Browser console shows no JavaScript errors on the dashboard.
Show model answer
In smoke: a, c, d, f. Out of smoke: b, e. Reasoning: - (a) Home page loads — vital sign, the app is alive. In. - (c) RealMe login — core authentication; if it fails, nothing else is testable. In. - (d) "File a return" opens the flow — a critical path is reachable. In (it confirms reachability, not correctness). - (f) No JS console errors on the dashboard — fast, catches silent breakage affecting everything downstream. In. - (b) GST correct to the cent — that is business-logic / calculation accuracy. Deep, slow, and not a vital sign. Out (regression/functional). - (e) Session expires at exactly 30 minutes — a specific timeout value; too detailed and too slow for smoke. Out (functional/security). The split rule: smoke checks "is the build worth testing?" — wide and shallow, happy paths only. Anything testing correctness of a specific calculation or a precise edge value is deeper than smoke.
A grad wrote the "smoke suite" below for a KiwiFirst Bank mobile app build. It takes 90 minutes to run and half of it isn’t smoke at all. Rewrite it into a proper smoke suite (target: under 15 minutes) and say what you removed and why.
1. App opens to the login screen
2. Login with a valid test customer succeeds
3. Every error message across the transfer flow is checked for correct wording
4. Account balances load on the dashboard
5. Interest is calculated correctly for a 12-month term deposit
6. A payment can be made to a saved payee
7. All 14 settings toggles are tested for persistence after logout
Rewrite as a proper smoke suite:
Show model answer
Keep (the vital signs, ~10 min): 1. App opens to the login screen 2. Login with a valid test customer succeeds 4. Account balances load on the dashboard 6. A payment can be made to a saved payee (critical path completes end-to-end) Removed: - 3. Checking every error message wording — that is detailed functional/UX testing. Move to the functional suite. - 5. Term-deposit interest calculation — business-logic correctness. Move to regression/functional. - 7. All 14 settings toggles for persistence — feature-by-feature depth, far too slow for smoke. Move to functional/regression. Why it is now valid: it is wide and shallow, walks only the critical happy paths (open, log in, see balances, complete a payment), and runs well under 15 minutes. It answers "is this build worth deeper testing?" without trying to prove any feature is correct.
A NZ council rates portal deploys automatically several times a day. Design a smoke suite to run in the CI/CD pipeline that gates whether a deploy is promoted or rolled back. List 5–6 checks, and for each state what a failure should trigger. Remember: CI smoke must be fast, deterministic, and need no human decision.
Show model answer
Pipeline smoke checks (gate the deploy): 1. Home page returns HTTP 200 and renders — on failure: block promotion, auto-roll back. 2. Health/status endpoint reports database connected — on failure: roll back (the build can’t serve data). 3. Login with a seeded test account returns a valid session — on failure: roll back (auth down = portal unusable). 4. "Pay rates" page loads and the payment form is present — on failure: roll back (critical path unreachable). 5. A scripted test payment reaches the confirmation page — on failure: roll back (end-to-end path broken). 6. No 5xx errors in the app logs during the run — on failure: roll back. Why safe to automate: each check is a clear pass/fail with no judgement (HTTP status, element present, session returned, confirmation page reached), each runs in seconds, and each is deterministic across runs — so the pipeline can decide promote-or-rollback with zero human input. Deliberately excluded: calculation accuracy, edge-case validation, timeouts — those are too slow or too flaky to gate a deploy.
Why teams fail here
- Smoke suite creep — the suite starts at 10 minutes, grows to 45 minutes over two years, and teams start skipping it entirely under deadline pressure.
- Treating a green smoke as a release gate — a passed smoke is permission to test further, not sign-off on quality. This confusion gets teams into trouble with stakeholders who forward the green result as approval.
- Stale automated suites — the CI/CD smoke suite is written once and never updated when critical paths change. Builds that break new flows still pass smoke because smoke is checking old flows.
- Skipping environment verification — smoke passes on the application but nobody confirms the environment itself (database seed date, config version, connected service endpoints). The app looks fine but is running against last sprint's data.
Key takeaway
Smoke testing is the cheapest decision you make in a testing cycle — two minutes confirming the build is alive saves hours of testing a corpse.
How this has changed
The field moved. Here is how Smoke Testing evolved from its origins to current practice.
The term originates in electronics — power on a new circuit board and check whether smoke appears. If it smokes, something fundamental is wrong and further testing is pointless. Software adopted the concept directly.
Daily builds and smoke tests become standard at Microsoft (documented by Steve McConnell in Code Complete, 1993). The smoke test verifies that the build is testable before the full test suite runs — saving QA teams from discovering a broken build after hours of setup.
Agile adoption makes smoke tests continuous rather than daily. Every CI build triggers a smoke test as the first gate. Teams develop the muscle of keeping smoke tests fast (under 5 minutes) and representative of the critical paths.
Smoke testing extends to infrastructure and deployment verification. Post-deployment smoke tests verify that the deployment succeeded before routing real traffic. Blue-green deployment smoke tests prevent routing traffic to a broken new version.
In microservices architectures, smoke tests verify the entire service mesh is healthy — hitting each service's health endpoint and tracing a request end-to-end. Synthetic monitoring (running smoke tests against production continuously) blurs the line between testing and observability.
Self-Check
Click each question to reveal the answer.
Interview Questions
What NZ hiring managers ask about Smoke Testing — and what strong answers look like.
What should a post-deployment smoke test cover, and how long should it take?
Strong answer: A post-deployment smoke test should cover the critical paths that represent "the system is alive and functional" — user authentication (can users log in?), the primary user workflow (can users complete the core task the system exists to support?), external integrations (are downstream services reachable?), and health endpoints for all services. It should not attempt to cover all features — that is what the regression suite is for. Target runtime: under 5 minutes. A smoke test that takes 30 minutes provides delayed feedback and creates pressure to skip it. Automated smoke tests running immediately after deployment, before traffic is routed, prevent routing users to a broken release.
Junior/Mid
A smoke test fails in production after deployment. What are your next steps?
Strong answer: Immediately trigger rollback if the deployment configuration supports automated rollback — do not wait. Simultaneously check whether real users are already affected by monitoring error rates and user support channels. Preserve the deployment artefacts and logs before rollback overwrites them. Once stable on the previous version, reproduce the smoke test failure in a staging environment that mirrors production. Identify the root cause — did the deployment introduce the regression, or was the smoke test itself incorrect? If a smoke test is failing for a reason that does not affect real users, that is a flawed smoke test that needs updating. Document the incident and add the specific failure as a permanent smoke test case.
Mid/Senior
Q1: What single question is a smoke test trying to answer?
Is this build worth testing? Smoke is a fast, shallow pass over critical paths to decide whether deeper testing should begin — not whether the application is correct or release-ready.
Q2: A smoke test fails on the login step. What should you do next?
Stop and send the build back immediately. There is no value in deep-testing a build whose critical path is broken — you’d only generate false bugs and waste time. The build is returned to development before further testing.
Q3: How does a smoke test differ from a sanity test?
Smoke is wide and shallow, run after every build to confirm the whole app is stable enough to test. Sanity is narrow and a bit deeper, run after a specific change or bug fix to confirm that one area works. Smoke asks "worth testing?"; sanity asks "did the fix work?".
Q4: Why is checking a calculation’s exact result a poor fit for a smoke test?
Because smoke confirms the build is reachable and stable, not that business logic is correct. Calculation accuracy is deep functional/regression testing — it is slower and tests correctness, which is a different job from the vital-signs check smoke performs.
Q5: What makes a check suitable for an automated CI/CD smoke suite?
It must be high value, fast, and deterministic — a clear pass/fail with no human judgement (HTTP 200, element present, session returned). That lets the pipeline decide promote-or-rollback automatically. Flaky or slow checks, and anything needing interpretation, don’t belong in a gating smoke suite.
Q6: Your team has just received a new build of the Benefits NZ benefit payments portal on a Friday afternoon. There is one hour before the UAT window closes for the week. A stakeholder says “just skip smoke and go straight to the complex eligibility scenarios — we need the coverage.” What do you do and why?
A: Run smoke first, even if it only takes 10 minutes. If the build has a broken login or a failed database connection, every minute spent on eligibility scenarios is wasted effort on a system that cannot actually process payments. In a government portal like Benefits NZ, a failed smoke also means stopping the UAT cleanly rather than generating misleading defect reports that waste a developer’s weekend. The correct response to the stakeholder: “Smoke takes 10 minutes. If it passes, we have 50 minutes of real eligibility coverage. If it fails, we know immediately and can escalate to the developer today rather than Monday.”
Q7: What is the key difference between smoke testing and regression testing, and when would you run each on a KiwiSaver provider portal that has just received a hotfix to the contribution calculation engine?
A: Smoke testing is wide and shallow — it asks “is the build alive and testable?” and covers the whole application at a surface level (login, navigation, main flow). Regression testing is thorough and deep — it asks “did the change break anything that was working before?” and covers a broad set of scenarios including edge cases and previously reported bugs. After the hotfix lands on the KiwiSaver portal, you run smoke first: confirm login works, the dashboard loads, and the contribution summary page is reachable. Only once smoke passes do you run targeted regression on the contribution calculation engine — checking exact percentage splits, member vs employer contributions, and prior bug scenarios. Running regression first risks spending an hour on calculation tests inside a portal whose session management is broken.
Q8: When should you deliberately skip a smoke test, even after a new build arrives in the test environment?
A: Skip smoke when the risk and cost of running it outweigh the benefit. Two clear cases: (1) A trivial content-only change — for example, a label correction on the TransitNZ driver licence portal or a colour fix on the CoverNZ homepage. A targeted sanity check on the changed element is faster and more appropriate than a full smoke pass on a known-stable environment. (2) A green automated CI/CD smoke suite has already gated the deploy — if the pipeline ran smoke and promoted the build, duplicating the same checks manually adds delay without adding signal. Reserve your skip decisions for environments with a trusted, up-to-date automated suite; if the suite is stale or untrusted, run manual smoke regardless.
Q9: A developer on your Revenue NZ project says “our smoke suite passed, so we’re good to release to production this afternoon.” What is wrong with this and how do you respond?
A: A passed smoke test does not mean the application is release-ready — it means the build is stable enough to test further. Smoke deliberately skips business logic, calculation accuracy, edge cases, security checks, and accessibility. On an Revenue NZ portal, that means GST calculations, income tax thresholds, RealMe authentication edge cases, and WCAG compliance have not been touched by smoke at all. The correct response: “Smoke passed, which tells us the build is alive and the critical paths are reachable. We still need to complete functional testing, regression, and the security sign-off before we can make a release decision. I’ll document in the test report exactly what smoke covered and what remains outstanding.” Never let a green smoke result be forwarded as a release sign-off without that written clarification.
Try It — Build a smoke test suite
A new NZ council rates portal has just been deployed to the test environment. You have 20 minutes for a smoke test before the team begins deeper testing. Which of the following tests belong in your smoke suite? Tick all that apply.
Smoke suite answer
| Test | In smoke suite? | Why |
|---|---|---|
| Home page loads | Yes | Vital sign #1 — application is alive. |
| Login with RealMe succeeds | Yes | Core authentication — blocks all other testing if broken. |
| Rural rate calculation accuracy | No | Too deep — this is business logic testing, not smoke. Add to regression suite. |
| Payment page loads | Yes | Critical path milestone — confirms the payment flow is reachable. |
| 15-digit credit card rejected | No | Edge case validation — not smoke. Goes in functional testing. |
| Confirmation page after test payment | Yes | End-to-end critical path completion — confirms the whole flow works. |
| PDF receipt downloads | No | Nice to have feature, not a blocker. Add to functional suite. |
| Session timeout at exactly 20 min | No | Specific timeout value testing — too detailed for smoke. Functional/security testing. |
| Navigation links work | Yes | Core navigation — if menus are broken the portal is untestable. |
| No JS errors on home page | Yes | Fast check, catches silent errors that will affect everything downstream. |
Smoke suite = 6 tests. Total time: ~10 minutes. This confirms the build is stable enough to proceed. Tests 3, 5, 7, and 8 are valuable — just not smoke tests.