Configuration Management for Testing

1 The Hook

A Wellington-based product team spent two days debugging a “critical” defect. The tester had written meticulous reproduction steps. The developer swore the fix was deployed. Yet the bug persisted in staging. After sixteen Slack threads, a painful video call, and one very cold flat white, they discovered the truth: the tester was working from API specification v2.1, while the developers had already built v2.2. The “bug” was a misunderstanding of which version they were all supposed to be testing.

There was no configuration management. No baseline. No version control on test documentation. The team was effectively playing a game of telephone with their own product.

Configuration management is what prevents this chaos. It is not bureaucracy. It is the difference between knowing what you are testing and guessing.

2 The Rule

Configuration management (CM) for testing is the discipline of identifying, controlling, tracking, and auditing all testware and the software under test. Under ISTQB CTFL v4.0, CM supports testing by ensuring:

Identification: Every item — test cases, scripts, data, environments, builds — has a unique identifier
Control: Changes are proposed, reviewed, approved, and implemented through a formal process
Status accounting: You can answer, at any moment, what version is where and who changed it
Auditing: You can prove that the tested version matches the released version

Key concepts:

Baseline: A formally reviewed and agreed-upon version that serves as a reference point
Version control: A system that records changes and allows rollback
Traceability: Links between requirements, tests, defects, and code so you can prove coverage

3 The Analogy

Analogy

Think of a pharmacy in Christchurch. Every bottle on the shelf has a batch number, an expiry date, and a chain of custody from manufacturer to shelf. Before a pharmacist hands anything to a patient, they verify exactly what they are dispensing. They do not guess. They do not assume the latest delivery is the right one. They check.

Configuration management for testing is the same discipline. You never run a test against a build without knowing the commit hash. You never report a defect against a requirement without knowing the version. You never hand over a release candidate without an audit trail.

4 Watch Me Do It

An Auckland-based SaaS startup, KiwiMetrics, is scaling from three developers to twelve. They need a CM plan. Here is how a test lead structures it:

Version-control everything Test cases live in Git alongside code. Test data schemas are scripted. Environment configurations are in Terraform. If it is needed to reproduce a test, it is version-controlled.
Baseline test environments Before each sprint, the staging environment is snapshotted and tagged (e.g., sprint-23-baseline). Testers know they are testing against a known state, not whatever was deployed at 3am on Tuesday.
Manage test data explicitly A shared spreadsheet named “latest” is banned. Test data sets are versioned, anonymised for privacy compliance, and reset between regression cycles using database snapshots.
Build a traceability matrix Each user story links to test cases, which link to automated scripts, which link to defects. During an audit, KiwiMetrics can prove that every requirement was tested and every defect was verified.

Item	CM Approach	Tool Example
Test cases	Version-controlled documents / code	Git, TestRail with versioning
Test scripts	Commit hash tagged to each run	CI pipeline (GitHub Actions)
Test data	Anonymised snapshots, reset between runs	Docker volumes, db restore scripts
Environments	Infrastructure as code, baselined per release	Terraform, Ansible
Requirements	Traceability to tests and defects	Jira, Xray, Zephyr

Failure case: Release chaos without CM

Scenario: A Dunedin fintech team skips configuration management. "We are small. We do not need it." Fast forward three months:

No build labelling: Testers run tests against "production-staging" without knowing it is 5 commits behind main.
No test-case versioning: When a requirement changes, somebody updates the Jira story, but the actual test case exists in three Word documents. Which one is the source of truth?
No test data baseline: A developer tweaks the database schema overnight. Automated tests start failing. Nobody knows why because nobody has a baseline snapshot.
No traceability: A security audit asks, "Which tests cover the encryption requirement?" The lead has no way to answer.

The fix: Start small. Use Git for test cases. Tag every build with a commit hash. Create a one-page traceability map linking requirements to test IDs. Reset the database from a snapshot before each test cycle. These four practices eliminate 80% of the chaos.

5 When / When-not

Formal CM is essential when:

You work in a regulated industry (healthcare, finance, government) where audit trails are mandatory
The team is large enough that “everyone knows” is no longer true
You release frequently and need to reproduce old builds for hotfixes
You are preparing for an external audit or ISO certification

Lightweight CM is enough when:

You are a pre-product startup with two developers and one tester
The product is a prototype with no production users
The cost of formal process would exceed the risk of error

Even in lightweight settings, the core habits — version control, named builds, and a simple traceability map — still apply. You can scale the ceremony, but not the discipline.

Before you implement configuration management, ask:

Do you currently lose track of which version was tested? Are you guessing?
If a compliance auditor asked "prove you tested this requirement," could you?
How often does a test pass locally but fail in CI, and nobody knows why?
Are your test cases stored in multiple versions (Word docs, sheets, tools) with no single source of truth?

6 Common Mistakes

✗ Not version-controlling test cases

Fix: Treat test cases as code. If a requirement changes, the test case should change in the same commit or ticket. You should always be able to see what a test looked like at the time of a past release.

✗ Using “latest” builds without knowing the commit hash

Fix: Every build that enters test should have a unique identifier: commit hash, build number, or tag. “Latest” is not a version. It is a guess.

✗ No traceability — cannot prove coverage

Fix: Link requirements to tests, tests to execution results, and defects to verification. Even a simple spreadsheet is better than nothing. At scale, use a test-management tool.

✗ Test data not reset between runs

Fix: Test data must be in a known state at the start of each cycle. Use database snapshots, seed scripts, or containerised environments. Residual data from a previous run creates flaky, irreproducible results.

When this technique fails

Without configuration management, you end up with flaky tests that pass today and fail tomorrow because nobody knows which version is deployed. Audit trails vanish. You cannot prove that version 1.2.3 was actually tested. Worse, when a production issue arises, you cannot reproduce it because nobody documented what was tested. The cost of CM (documentation, tooling) looks expensive until the cost of NOT having it shows up as rework, customer escalations, and regulatory penalties.

7 Now You Try

Exercise

Scenario: A Christchurch development team is in chaos. Read the symptoms below and identify the configuration management problem and a fix.

A tester reports a login bug. The developer says it was fixed last week. The tester is testing against Friday’s build, but the fix was in Monday’s build. The two builds were both called “staging-latest.”
An auditor asks, “Which test cases cover the GST-calculation requirement?” The test lead opens a folder of Word documents. None are dated. Some have “final” in the filename. Others have “final-final.”
The automated regression suite passes on one machine and fails on another. Both machines connect to the same database. The database was manually edited by a developer three days ago. Nobody documented the change.

Model Answers:

Problem: No unique build identification. Fix: Tag every build with a commit hash and build number. Never use “latest” as a deploy target for testing.
Problem: Test cases are not version-controlled and lack traceability. Fix: Move test cases into a version-controlled system or test-management tool. Link each requirement to its tests in a traceability matrix.
Problem: Test environment and data are not baselined or reset. Fix: Use database snapshots or seed scripts to restore a known state before each test run. Treat environment changes as code-reviewed pull requests.

8 Self-Check

Q1. What are the four main activities of configuration management?

Identification, control, status accounting, and auditing. These ensure every item is known, changes are managed, status is visible, and compliance can be proven.

Q2. What is a baseline in configuration management?

A baseline is a formally reviewed and agreed-upon version of an item or group of items that serves as a reference point for future changes. It is a snapshot you can return to.

Q3. How does traceability support testing?

Traceability links requirements to test cases, test cases to execution results, and defects to verification. It allows you to prove coverage, assess impact of changes, and demonstrate compliance to auditors.

9 Interview Prep — Lead-Level Q&A

At the Test Lead level, interviewers probe your ability to set up reliable systems and lead the team through change.

Q1. Describe a time you implemented version control or configuration management. What was the resistance, and how did you overcome it?

I led a team that was manually managing test cases in shared drives. We started by simply moving them to Git with a naming convention. There was resistance: "Why do we need Git for documents?" I showed them the problem: three versions of the same test case in existence. Then I made migration painless by creating a simple import script. Within two weeks, the team saw the benefit: they could see who changed what and why. Now it is second nature.

Q2. How do you prove that a build released to production was actually tested?

Through traceability. Every build has a unique identifier (commit hash or tag). Linked test cases have run results. Defects that were found are linked to verification tests. At release, I can generate a report: "Build abc123 has 847 tests executed, 0 open defects, coverage 89%. Here is the evidence." This is not bureaucracy; it is confidence.

Q3. A developer says, "We do not have time for all this CM bureaucracy." How do you respond?

I reframe it. Configuration management is not bureaucracy; it is the price of scale. At three people, you can rely on memory. At thirty, you cannot. I propose starting lightweight: Git for test cases, commit hash on every build, a one-page traceability map. These take one day to set up but save weeks when something goes wrong. I then tie it to real pain: "Remember when we spent two days debugging 'the missing feature'? It was missing because we did not know which version we tested."