What Makes a Good Unit Test — From FIRST to Contract Testing

The Hard Problem of “Good” Tests

Writing a unit test is not, in itself, difficult. Countless tests can be written that run and turn green, but not all of them create value. A poorly designed test becomes a liability that breaks on every refactor and slows the team down. Two frameworks have tried to define what a “good” test actually is: the classic FIRST principles, and Vladimir Khorikov’s newer “four pillars.” Using these two as anchors, this article traces the practical design decisions of independence, granularity, and naming, then follows the trail past the boundary of the unit test itself, into integration testing and contract testing.

FIRST — The Runtime Properties of a Test

FIRST is an acronym for five properties a good unit test should satisfy. It was introduced in Robert C. Martin’s Clean Code (2008) as a formulation by Tim Ottinger and Jeff Langr.

Letter	Principle	Meaning
F	Fast	Finishes in milliseconds. Slow tests get run less often and lose their value
I	Independent	Doesn’t depend on the order or result of other tests
R	Repeatable	Produces the same result every time, in any environment
S	Self-Validating	Pass/fail is determined mechanically, with no need to eyeball logs
T	Timely	Written right after the implementation — or, under TDD, before it

Independent and Repeatable in particular are direct countermeasures against flaky tests — tests whose results vary from run to run (see The Fight Against Flaky Tests). Sharing global variables or database state across tests produces a suite whose outcome depends on execution order.

Khorikov’s “Four Pillars” — What Is a Test For?

Where Robert C. Martin’s FIRST focuses on a test’s runtime characteristics, Vladimir Khorikov, in Unit Testing Principles, Practices, and Patterns (Manning, 2020), proposed four pillars a test needs in order to keep generating value over the long term.

1. Protection against regressions — will the test reliably catch a bug that gets introduced?
2. Resistance to refactoring — does the test avoid false positives (failing even though nothing is actually broken) when internal implementation changes without changing behavior?
3. Fast feedback — does it run quickly enough to not disrupt the development rhythm?
4. Maintainability — is the test code itself readable and cheap to write?

The core of Khorikov’s argument is that these four cannot all be maximized at once. Resistance to refactoring and maintainability are, in a sense, binary properties: a test coupled to implementation details breaks; one that isn’t, doesn’t. Protection against regressions and fast feedback, on the other hand, tend to trade off continuously — verifying a broader scope raises regression protection but slows execution.

Khorikov specifically warns that overusing mocks quietly destroys resistance to refactoring. Tests that verify “was this called” or “with what arguments” tend to couple tightly to implementation details — which methods get called, in what order — so changing internal implementation alone turns the test red. Khorikov argues mocks should only be used at “the boundary between the system under test and the outside world” — out-of-process dependencies (see The Rise of Mocks and Test Doubles for the history of mocks themselves).

Test Independence — Three Levels

Overlapping with FIRST’s Independent, independence needs to be secured at three levels.

• State independence: global state or shared database records that one test rewrites must not break another test’s assumptions
• Order independence: results must not change regardless of the order the test runner executes tests in
• Environment independence: results must be identical on a local machine, in CI, and on every developer’s machine

Typical techniques include resetting state before and after each test (e.g. rolling back a transaction), spinning up a fresh in-memory dependency per test, and using dependency injection (DI) to avoid relying on global singletons.

One Test, One Behavior

A good unit test verifies exactly one behavior per test case. What should be avoided is bundling several unrelated behaviors into a single test.

// Bad: crams several unrelated behaviors into one test
test("user registration", () => {
  const user = registerUser("[email protected]", "password123");
  expect(user.email).toBe("[email protected]");   // behavior 1
  expect(sendWelcomeEmail).toHaveBeenCalled();      // behavior 2
  expect(auditLog.length).toBe(1);                  // behavior 3
});

// Good: one test per behavior
test("registration saves the email address", () => { /* ... */ });
test("registration sends a welcome email", () => { /* ... */ });

Following this, a failing test’s name alone tells you what broke. A test with many assertions stacked up, where you have to read the run output to figure out which one failed, is known as “assertion roulette” — an anti-pattern to be avoided.

The AAA Pattern

A widely used convention for structuring the inside of a test is the AAA pattern (Arrange-Act-Assert). Bill Wake is credited with introducing it as “3A” in a 2001 blog post, and it has since become the de facto ordering across xUnit-family frameworks.

def test_withdraw_more_than_balance_raises_error():
    # Arrange
    account = Account(balance=1000)

    # Act & Assert
    with pytest.raises(InsufficientFundsError):
        account.withdraw(2000)

BDD tools express the same structure with the vocabulary Given-When-Then (see The Rise of BDD). The names differ, but the idea of separating setup, execution, and verification is shared.

What Not to Test

Designing a unit test suite also means deciding what not to write. Tests with low value include:

• Trivial code: simple getters/setters, code with no logic
• Implementation details: the internal steps of a private method, the call count or order itself
• Third-party framework internals: whether an ORM or web framework is correct
• Language runtime or standard library correctness: there’s no need to write your own test proving that sort() sorts correctly

Testing these contributes almost nothing to regression protection while only increasing maintenance cost. Khorikov frames this in terms of a test’s ROI (return on investment), arguing that deciding to delete a low-value test is itself part of design. A test written only to raise a coverage percentage can, by this principle, become a liability (coverage’s traps are covered in How Do You Measure Test Quality).

Past the Unit — The Scope Problem of Integration Testing

Few terms are used with as much variance as “integration test.” As microservice architectures spread, this ambiguity of definition became a real practical problem. Martin Fowler, in his bliki entry “IntegrationTest,” proposed sorting it out by scope.

• Broad integration test: spins up every service, database, and external API it depends on — the real thing, or close to it — and verifies against them. Highly faithful, but heavy to set up and prone to instability from its dependencies
• Narrow integration test: isolates just the part of your own service that talks to an external service, replacing the external side with a test double. Fast and stable, but leaves a gap against the real dependency’s actual behavior

Fowler also notes that a “broad integration test” is, in effect, better called a system test or end-to-end test (see The Generational Shift in E2E Testing for the lineage of E2E testing).

Sociable / Solitary — A Different Cut

Separate from scope, Jay Fields, in Working Effectively with Unit Tests, proposed the pair sociable and solitary along the axis of how a test relates to its dependencies. A solitary test replaces every dependency with a test double; a sociable test uses the real dependency as-is. Where “narrow/broad” mostly tracks process boundaries — does it cross a database or external service — “sociable/solitary” tracks whether collaboration between objects within the same process is severed by mocks — a somewhat different axis of granularity.

Testcontainers — Using Real Dependencies via Docker

The problem that “settling for mocks sacrifices fidelity” is what popularized Testcontainers. It began as a Java library developed by Richard North in 2015 and has since been ported to Go, .NET, Python, Node.js, and other languages. Its idea is simple: spin up a real, disposable dependency (PostgreSQL, Kafka, Redis, and so on) in a Docker container at test time, then tear it down once the test finishes. This sidesteps the “slightly different behavior from the real thing” problem that in-memory databases have, at the cost of slower execution from container startup overhead.

The Rise of Contract Testing — Consumer-Driven Contracts

The more services A, B, C, D and their dependencies pile up, the more the cost of maintaining broad integration tests grows exponentially. The answer to this problem is Consumer-Driven Contracts (CDC), put forward by Ian Robinson in a 2006 article on martinfowler.com, “Consumer-Driven Contracts: A Service Evolution Pattern.”

The essence of CDC is this:

1. The API’s consumer explicitly writes down, as a contract, the shape of the request/response it needs
2. That contract is automatically verified in the provider’s CI. As long as the provider satisfies the contract, it is free to change its internal implementation
3. The provider aggregates contracts from multiple consumers and can determine in advance, without ever wiring everything together, which consumers would break if it changed the API

This lets teams verify the fit at each boundary independently, service by service, without ever having to wire the whole dependency chain together and run it.

Pact — The Flagship CDC Implementation

Pact is the leading tool implementing CDC. It grew out of practical work at REA Group, an Australian real-estate listing company, and was developed as a Ruby gem by the consultancy DiUS around 2013. It’s now used as a multi-language contract-testing framework across the JVM, JavaScript, .NET, Go, Python, and more.

The typical workflow looks like this:

1. On the consumer side, a test describes the expected request/response pair and runs against a mock server on the spot
2. That test run automatically generates a “Pact file” — a JSON contract definition
3. The Pact file is published to a Pact Broker (a server for sharing and versioning contracts)
4. On the provider side, CI fetches the contract from the Pact Broker and replays it against the actual provider code to verify it (provider verification)

The order — “the contract is generated automatically from a test the consumer wrote” — is where the Consumer-Driven in the name comes from. The contract isn’t a spec unilaterally dictated by the provider; it’s derived from actual usage.

In the Java/Spring ecosystem, a different approach called Spring Cloud Contract is used instead. Where Pact generates a contract from a test the consumer wrote, Spring Cloud Contract takes a provider-first design: the provider defines the contract (in a Groovy DSL or YAML), and both the consumer-side stub and the provider-side verification test are generated from it.

Service Virtualization — An Alternative

When contract testing is more than the situation calls for, or when an external SaaS is simply outside your CI’s reach, substituting service virtualization or a stub server is a common alternative.

Tool	Notes
WireMock	An HTTP stub server developed by Tom Akehurst in 2011
mountebank	Developed by Brandon Byars. Supports multiple protocols beyond HTTP, including TCP and SMTP
Hoverfly	Strong at capturing and replaying HTTP(S) traffic

The difference from contract testing is that service virtualization only “prepares a canned response,” while contract testing automatically verifies that the canned response actually matches the provider’s real behavior.

Summary

FIRST defines the runtime properties of how a test should behave, while Khorikov’s four pillars define the value properties of what a test exists for — the two are complementary, not contradictory. And outside the unit test itself, Fowler’s narrow/broad axis of scope, together with contract testing’s guarantee of a fit without ever wiring things together, are two answers to the same problem: the exponential growth of combinations to verify once services multiply. The question to ultimately ask is a single one: how much freedom does this test give you to change the code? The next article turns to coverage — a “quantity” metric for tests — and to mutation testing, which supplies the “quality” metric that coverage is missing.

References

• Robert C. Martin, Clean Code (2008, Prentice Hall)
• Vladimir Khorikov, Unit Testing Principles, Practices, and Patterns (2020, Manning)
• Roy Osherove, The Art of Unit Testing, 2nd ed. (2013, Manning)
• Gerard Meszaros, xUnit Test Patterns: Refactoring Test Code (2007, Addison-Wesley)
• Martin Fowler, “IntegrationTest” / “UnitTest” (martinfowler.com/bliki/)
• Jay Fields, Working Effectively with Unit Tests (2014, Leanpub)
• Ian Robinson, “Consumer-Driven Contracts: A Service Evolution Pattern” (martinfowler.com, 2006)
• Pact official documentation (pact.io)
• Testcontainers official documentation (testcontainers.com)