The Lineage of Property-Based Testing — From QuickCheck to Hypothesis

The mocks we saw last time (Article 6) were a technique for verifying specific behavior — whether a particular call happened correctly. Most unit tests work the same way: they write, one at a time, examples of “for this input, this output.” But there’s an entirely different approach: define up front a property that the code should satisfy — “the length of a list doesn’t change after sorting,” “serializing and then deserializing returns the original value” — and have a tool verify that property against a huge number of randomly generated inputs. This is property-based testing (PBT).

The QuickCheck Paper — ICFP 2000

In 2000, at Chalmers University of Technology in Gothenburg, Sweden, Koen Claessen and John Hughes presented a paper titled “QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs” at the ACM SIGPLAN International Conference on Functional Programming (ICFP 2000). This paper is the starting point that spread the concept of property-based testing to the world.

The QuickCheck implementation shown in the paper was, at first, a simple library of only about 300 lines — but the idea was powerful.

-- The property "reversing a list twice returns the original list"
prop_reverseReverse :: [Int] -> Bool
prop_reverseReverse xs = reverse (reverse xs) == xs

Rather than individual input-output examples, the programmer writes a function expressing the property that must hold. QuickCheck then generates a large number of random inputs based on type information and repeats the test until it finds a counterexample.

Generators and Shrinking

Two core concepts established by QuickCheck are what make property-based testing work as a practical technique.

A generator is a mechanism that produces random values appropriate to the type under test (integers, strings, lists, custom data types). Through a type class (Arbitrary), QuickCheck automatically supplies generators for standard types, while letting users define their own for custom types.

Shrinking is the mechanism that, once a counterexample is found, automatically reduces that input to a simpler form. If a failure reproduces with a 47-element list, presenting it as-is is hard for a human to read. QuickCheck keeps shrinking the input while the property remains broken, ultimately presenting “the smallest counterexample that can no longer be simplified.” This combination of “generator plus shrinking” is the architecture shared by every QuickCheck-family tool.

Generated failing input: [3, -17, 0, 256, -1, 42]
                          ↓ shrink
                       [0, -1]
                          ↓ shrink
                       [-1]          ← the smallest possible counterexample

Quviq QuickCheck — Commercialization and Concurrent Systems

In 2006, John Hughes founded Quviq to commercialize QuickCheck’s technology, offering a commercial version of QuickCheck for Erlang. It has been applied to real systems in the telecom and automotive industries to detect complex bugs in concurrent and distributed systems — race conditions, inconsistencies during node failures — and is known for having found many serious bugs in practice.

The Erlang version is also notable for developing stateful property testing: modeling a stateful system (a protocol implementation, a database) as a state machine, and applying a randomly generated sequence of operations to both the model and the real system to check that they stay consistent. Because the randomly generated order of operations often triggers bugs by itself, this approach can find race conditions and timing-dependent bugs that a single-input test would never catch.

Spread Across Languages

QuickCheck’s idea spread rapidly beyond Haskell, and today a “QuickCheck-family” library exists for nearly every major language.

Language	Library	Notes
Haskell	QuickCheck	The original (Claessen & Hughes, 2000)
Erlang	Quviq QuickCheck	Commercial, for concurrent/distributed systems
Scala	ScalaCheck	Developed by Rickard Nilsson
Python	Hypothesis	Developed by David R. MacIver
JavaScript/TypeScript	fast-check	Used alongside Jest and similar tools
Rust	proptest / quickcheck	proptest adopts Hypothesis’s design philosophy
Java	jqwik	Emphasizes integration with JUnit 5
C#/F#	FsCheck	For .NET

What Makes Hypothesis Distinctive

Python’s Hypothesis is not simply a port of QuickCheck — it has undergone several important evolutions of its own. It’s distinguished by an integrated mechanism that stores the history of generated values in a database and preferentially reproduces previously found counterexamples on subsequent test runs, and by the sophistication of its shrinking.

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_reverse_twice_is_identity(xs):
    assert list(reversed(list(reversed(xs)))) == xs

This test doesn’t run against a single example input but against the entire space of “any list of integers”; if a failing input is found, it’s reported after shrinking as a minimal counterexample. Rust’s proptest explicitly acknowledges being inspired by Hypothesis, inheriting its “strategy” vocabulary and its integrated-shrinking design. JavaScript’s fast-check is written in the same spirit.

import fc from "fast-check";

fc.assert(
  fc.property(fc.array(fc.integer()), (xs) => {
    const roundTripped = JSON.parse(JSON.stringify(xs));
    return JSON.stringify(roundTripped) === JSON.stringify(xs);
  })
);

In both examples, no concrete input value ever appears in the test code — this is the biggest difference from an ordinary unit test.

Two Styles of Shrinking

There are broadly two implementation styles for shrinking.

• Type-based shrinking: the style adopted by the original QuickCheck, where “how to shrink” is defined per type (move integers toward 0, remove elements from lists, and so on). Simple to implement, but shrinking rules must be written individually for each type.
• Integrated shrinking: the style adopted by Hypothesis and Rust’s proptest, where the process of generating a value — the history of random-number consumption — is itself the target of shrinking. Even when generator composition is complex, this lets shrinking stay consistent without changing the generation logic.

This design difference bears directly on usability when handling complex custom data types, and is a practical consideration when choosing a library.

Types of Properties — What Should You Write?

The hardest part of property-based testing isn’t learning to use the tool — it’s figuring out what property to write. Commonly used types in practice include:

• Round-trip: verifying that transforming and then transforming back returns the original, as in decode(encode(x)) == x. Common when testing serializers, parsers, and encodings.
• Invariant: a property that must hold before and after an operation — “the element count doesn’t change after sorting,” “pushing then popping a stack returns the same value,” and so on.
• Model-based: prepare a simple but slow, obviously-correct “model implementation” and check its output against the real implementation’s. Often also called differential testing.
• Relation to the oracle problem: even when there’s no available criterion (a test oracle) for judging “what the correct output is,” it’s often still possible to define “a relationship between input and output that must hold.” Property-based testing is positioned as a technique that enables verification even without a strict oracle.

Converging with Fuzzing

The idea of verifying a program with automatically generated input is also deeply related to fuzzing. Where PBT verifies logical correctness — whether a meaningful property holds — fuzzing mainly verifies robustness: does the program crash, or violate memory safety? Coverage-guided fuzzing, which became mainstream from the 2010s onward, measures which code paths were exercised at runtime and preferentially mutates inputs that exercised new code paths, making exploration far more efficient than blind generation.

Tool	Characteristics
AFL (American Fuzzy Lop)	Developed by Michal Zalewski, released in 2013. The pioneering tool that brought coverage-guided fuzzing into everyday practice
libFuzzer	An in-process fuzzing engine built into the LLVM project
OSS-Fuzz	Google’s continuous fuzzing service for open-source projects, announced in December 2016
go-fuzz / go test -fuzz	Developed by Dmitry Vyukov. Natively integrated into the language’s standard toolchain in Go 1.18 (2022)

// Native fuzzing in Go 1.18 and later
func FuzzParseURL(f *testing.F) {
    f.Add("https://example.com/path")
    f.Fuzz(func(t *testing.T, input string) {
        u, err := url.Parse(input)
        if err != nil {
            return // parse failures are acceptable
        }
        if _, err := url.Parse(u.String()); err != nil {
            t.Errorf("re-parse failed for %q: %v", u.String(), err)
        }
    })
}

The two are not mutually exclusive; in recent years, hybrid approaches — structure-aware fuzzing that generates structured inputs while also using coverage feedback — have spread, and the boundary between them is gradually blurring.

Adoption in Practice

Property-based testing is powerful, but its penetration into practice hasn’t reached the level of xUnit-style unit testing. Reasons cited include that formulating a “property” itself demands design skill and is a higher bar than writing examples, and that runtime tends to be longer, requiring seed-pinning and reproducibility management in CI environments. On the other hand, it’s widely recognized as highly effective for domain logic with clear invariants — parsers, serializers, cryptographic code, concurrency — and adoption has expanded, especially in the Python ecosystem, since Hypothesis became popular.

From This Article to the Next

Property-based testing and fuzzing were techniques that increased test coverage by automatically generating the “input side.” Next, we shift that same concern to the “outside” of the system — the perspective of an actual user operating a browser. We trace the generational shift in E2E testing, starting with Selenium in 2004 and continuing through Puppeteer, Cypress, and Playwright.

References

• Koen Claessen, John Hughes, “QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs” (ICFP 2000)
• Koen Claessen, John Hughes, “Testing Monadic Code with QuickCheck” (2002)
• Michal Zalewski, american fuzzy lop (released 2013)
• Google, “Announcing OSS-Fuzz: Continuous Fuzzing for Open Source Software” (Google Testing Blog, December 2016)
• The Go Programming Language, “Go Fuzzing” (go.dev)
• Hypothesis documentation (hypothesis.works)