Library Automation/ Experience-Based/ Smoke Testing
Foundational · All Levels

Smoke Testing

A quick pass to confirm the build is worth testing. Walk the most critical paths — load, navigate, submit. If smoke fails, stop and send the build back.

Grad Junior Senior Test Lead

1 The Hook

A Wellington fintech ships a new build to its test environment at 9am. The test lead, keen to get through a packed regression plan, dives straight into editing edge-case payment scenarios. Forty minutes in, nothing is saving. After another half hour of digging, the truth lands: the build never connected to the database at all. Login was broken from the first second. Every minute since 9am was spent testing a corpse.

The whole morning was wasted because nobody spent two minutes confirming the obvious first: does the app even start, can a user log in, does the main page load? Those are not deep tests. They are vital signs. And had anyone checked them, the build would have gone straight back to the developers at 9:02, not 10:30.

This is the trap a smoke test is designed to stop. Deep testing on an unstable build is worse than no testing — it produces noise, false bugs, and a wasted half-day. The cheap, shallow check up front protects the expensive, deep work that follows.

2 The Rule

Before any deep testing, run a fast, shallow pass over the most critical paths — load, log in, navigate, complete the main flow. If smoke fails, stop immediately and send the build back; don’t test further.

3 The Analogy

Analogy

The pre-flight walkaround at Wellington Airport.

Before an Air New Zealand turboprop pushes back, the captain walks a circuit of the aircraft: control surfaces move, tyres have tread, no fluid pooling under the engines, pitot tubes uncovered. It takes a few minutes and it does not test the autopilot, the cabin service, or the in-flight entertainment. It answers one question — is this aircraft safe enough to fly at all? If a tyre is flat, the walkaround stops there and the plane does not leave the gate.

A smoke test is that walkaround for a build. It is wide and shallow, it checks the vital signs, and a single failure grounds the build before anyone wastes time on the deep stuff.

What it is

Smoke testing is a shallow, wide test pass that checks whether the most critical functions of a build work at all. The name comes from hardware: plug in the circuit, see if it smokes. If it does, don’t do any more testing — go back and fix it.

The goal isn’t thorough testing — it’s a fast decision: is this build good enough to test? If smoke passes, deeper testing begins. If smoke fails, the build is returned immediately.

What to cover in a smoke test

  • Application launches and loads without errors
  • Core navigation works (menus, links, page transitions)
  • Primary user journeys complete end-to-end (login, main action, logout)
  • Critical integrations respond (database connects, APIs return data)
  • No crash-level errors in the browser console or server logs

Time target: 15–30 minutes maximum. If your smoke test takes longer, it’s too broad.

Real-world NZ Example: Trade Me

Imagine you're testing the latest build of the Trade Me iOS app. A smoke test suite would include:

  • Does the app open to the Home screen?
  • Can I search for "Tent"?
  • Does the "Watchlist" load my saved items?
  • Does the "Sell" button open the listing flow?
You don't test the complex auction bidding logic or international shipping calculations here — just the "vital signs."

Smoke testing in CI/CD

In modern pipelines, smoke tests are automated and run on every deployment. They gate whether a build proceeds to further testing or gets rolled back automatically. A failing smoke in CI/CD means the deploy is blocked — no human decision required.

As a tester, you should be able to identify which tests belong in the smoke suite: high value, fast to run, deterministic.

Smoke vs sanity testing

Smoke vs sanity: key differences
DimensionSmoke TestSanity Test
ScopeWide — covers the whole application shallowlyNarrow — focuses on a specific changed area
DepthShallow — happy paths onlyModerate — verifies specific functionality
When runAfter every build deploymentAfter a specific bug fix or change
GoalIs the build worth testing?Did the fix actually work?
Scripted?Usually (stable script, rerun each build)Often ad hoc (targeted at the fix)

What smoke testing doesn't cover

Smoke testing is deliberately shallow. It does not:

  • Test edge cases, error states, or boundary values
  • Validate business rules or complex logic
  • Test performance, security, or accessibility
  • Confirm all features work correctly

Common mistake: treating a passed smoke test as proof the application is ready to release. Smoke tests only confirm the build is stable enough to test further — not that it’s production-ready.

Practice this technique: Try Grad Practice 01 — Typos & wording, Grad Practice 05 — Navigation & footer.

4 Now You Try

Three graded exercises — spot, fix, then build. Write your answer, run it for AI feedback, then compare to the model answer.

🔍 Exercise 1 of 3 — Spot: smoke or not?

A new build of the IRD myIR portal has landed in the test environment. From the checks below, pick the ones that belong in a smoke test and say why each one stays in or out:

(a) Home page loads without errors. (b) GST calculation is correct to the cent for a 15% rate on $1,234.56. (c) Log in with a test RealMe account succeeds. (d) The "File a return" button opens the return flow. (e) Session expires after exactly 30 minutes idle. (f) Browser console shows no JavaScript errors on the dashboard.

Show model answer 
In smoke: a, c, d, f.
Out of smoke: b, e.

Reasoning:
- (a) Home page loads — vital sign, the app is alive. In.
- (c) RealMe login — core authentication; if it fails, nothing else is testable. In.
- (d) "File a return" opens the flow — a critical path is reachable. In (it confirms reachability, not correctness).
- (f) No JS console errors on the dashboard — fast, catches silent breakage affecting everything downstream. In.
- (b) GST correct to the cent — that is business-logic / calculation accuracy. Deep, slow, and not a vital sign. Out (regression/functional).
- (e) Session expires at exactly 30 minutes — a specific timeout value; too detailed and too slow for smoke. Out (functional/security).

The split rule: smoke checks "is the build worth testing?" — wide and shallow, happy paths only. Anything testing correctness of a specific calculation or a precise edge value is deeper than smoke.
🔧 Exercise 2 of 3 — Fix: repair a bloated smoke suite

A grad wrote the "smoke suite" below for a Kiwibank mobile app build. It takes 90 minutes to run and half of it isn’t smoke at all. Rewrite it into a proper smoke suite (target: under 15 minutes) and say what you removed and why.

Bloated "smoke" suite:
1. App opens to the login screen
2. Login with a valid test customer succeeds
3. Every error message across the transfer flow is checked for correct wording
4. Account balances load on the dashboard
5. Interest is calculated correctly for a 12-month term deposit
6. A payment can be made to a saved payee
7. All 14 settings toggles are tested for persistence after logout

Rewrite as a proper smoke suite:

Show model answer 
Keep (the vital signs, ~10 min):
1. App opens to the login screen
2. Login with a valid test customer succeeds
4. Account balances load on the dashboard
6. A payment can be made to a saved payee (critical path completes end-to-end)

Removed:
- 3. Checking every error message wording — that is detailed functional/UX testing. Move to the functional suite.
- 5. Term-deposit interest calculation — business-logic correctness. Move to regression/functional.
- 7. All 14 settings toggles for persistence — feature-by-feature depth, far too slow for smoke. Move to functional/regression.

Why it is now valid: it is wide and shallow, walks only the critical happy paths (open, log in, see balances, complete a payment), and runs well under 15 minutes. It answers "is this build worth deeper testing?" without trying to prove any feature is correct.
🏗️ Exercise 3 of 3 — Build: an automated CI/CD smoke suite

A NZ council rates portal deploys automatically several times a day. Design a smoke suite to run in the CI/CD pipeline that gates whether a deploy is promoted or rolled back. List 5–6 checks, and for each state what a failure should trigger. Remember: CI smoke must be fast, deterministic, and need no human decision.

Show model answer 
Pipeline smoke checks (gate the deploy):
1. Home page returns HTTP 200 and renders — on failure: block promotion, auto-roll back.
2. Health/status endpoint reports database connected — on failure: roll back (the build can’t serve data).
3. Login with a seeded test account returns a valid session — on failure: roll back (auth down = portal unusable).
4. "Pay rates" page loads and the payment form is present — on failure: roll back (critical path unreachable).
5. A scripted test payment reaches the confirmation page — on failure: roll back (end-to-end path broken).
6. No 5xx errors in the app logs during the run — on failure: roll back.

Why safe to automate: each check is a clear pass/fail with no judgement (HTTP status, element present, session returned, confirmation page reached), each runs in seconds, and each is deterministic across runs — so the pipeline can decide promote-or-rollback with zero human input. Deliberately excluded: calculation accuracy, edge-case validation, timeouts — those are too slow or too flaky to gate a deploy.

Self-Check

Click each question to reveal the answer.

Q1: What single question is a smoke test trying to answer?

Is this build worth testing? Smoke is a fast, shallow pass over critical paths to decide whether deeper testing should begin — not whether the application is correct or release-ready.

Q2: A smoke test fails on the login step. What should you do next?

Stop and send the build back immediately. There is no value in deep-testing a build whose critical path is broken — you’d only generate false bugs and waste time. The build is returned to development before further testing.

Q3: How does a smoke test differ from a sanity test?

Smoke is wide and shallow, run after every build to confirm the whole app is stable enough to test. Sanity is narrow and a bit deeper, run after a specific change or bug fix to confirm that one area works. Smoke asks "worth testing?"; sanity asks "did the fix work?".

Q4: Why is checking a calculation’s exact result a poor fit for a smoke test?

Because smoke confirms the build is reachable and stable, not that business logic is correct. Calculation accuracy is deep functional/regression testing — it is slower and tests correctness, which is a different job from the vital-signs check smoke performs.

Q5: What makes a check suitable for an automated CI/CD smoke suite?

It must be high value, fast, and deterministic — a clear pass/fail with no human judgement (HTTP 200, element present, session returned). That lets the pipeline decide promote-or-rollback automatically. Flaky or slow checks, and anything needing interpretation, don’t belong in a gating smoke suite.

Try It — Build a smoke test suite

A new NZ council rates portal has just been deployed to the test environment. You have 20 minutes for a smoke test before the team begins deeper testing. Which of the following tests belong in your smoke suite? Tick all that apply.