Prehistory — From Debugging to Testing, Thirty Years Before Systematization

An Era When the Word “Testing” Didn’t Mean Anything Yet

What we call “software testing” today did not exist as an independent discipline from the start. In the 1950s, at the dawn of computing, “debugging” — running a program and fixing it when it didn’t work — and “testing” — confirming that a program worked correctly — were barely distinguished from each other.

This article traces roughly thirty years, from that undifferentiated era to 1979, when Glenford Myers’s book The Art of Software Testing first systematized testing as its own intellectual discipline. It’s the story of a time before anyone had given clear shape to the question at the center of this series’ overview: how do we confirm a program is correct?

The road passes through three main turning points. First, in the 1960s, the growing scale of software becomes a social problem known as the “software crisis.” Next, around 1970, testing gets slotted into the waterfall model of process, even as Dijkstra sharply attacks the limits of that very model. Finally, in 1979, Myers’s book systematizes testing into a coherent body of theory for the first time. Let’s take these one at a time.

1947, and the Moth in the Relay — Where the Word “Bug” Came From

Before we get into the real prehistory, one famous anecdote is worth telling. In 1947, Harvard’s Mark II electromechanical computer malfunctioned, and on inspection, an actual moth was found caught in one of its relay contacts. The team led by Grace Hopper is said to have taped the moth into the logbook and recorded it as the “first actual case of bug being found.”

This anecdote is often credited with popularizing the word “bug” in software. To be precise, though, the word “bug” itself predates this by much longer — it appears in Edison’s letters in the 19th century, already used to mean a technical glitch — so Hopper’s team didn’t coin the term itself. Still, the moth story vividly captures the idea that a malfunction has a concrete cause that can be found and removed, which makes it a fitting way to open the first page of testing history.

The 1950s — When Debugging and Testing Weren’t Distinguished

Programming for early computers (ENIAC, EDSAC, and the like) meant the person who wrote the program checked its behavior in front of the machine, and fixed it on the spot if it didn’t work. That cycle was the whole process.

Neither the role of “tester” nor the independent concept of “testing” existed at the time; making a correctly working program was the goal in itself, and all the trial and error involved was lumped together as “debugging.” Deliverables we take for granted today, like test plans or designed test cases, were not yet common. The correctness of a program could only be confirmed by comparing the developer’s own understanding of the specification, held in their head, against the actual output.

This era is symbolized by a point Charles L. Baker is said to have made in a 1957 book review: he is considered one of the first people to explicitly distinguish “debugging a program” from “testing a program” as separate activities. From this point through the late 1970s, the view of testing as an independent discipline gradually spread.

The 1960s — The Software Crisis and the Seeds of Testing Research

Entering the 1960s, the scale and complexity of software grew rapidly, and development delays, budget overruns, and quality problems became social issues. As hardware performance improved, the ambitions of programs grew with it, and systems whose scale outstripped what individuals or small teams could manage through rules of thumb kept appearing.

This situation was captured in a phrase — the “software crisis” — at the 1968 NATO Software Engineering Conference held in Garmisch, Germany, an event that also served as the occasion for officially naming the field of “software engineering.” The conference proceedings were edited by Peter Naur and Brian Randell into a report that went on to influence the industry considerably. From the standpoint of testing history, it matters that the shortage of quality assurance and testing was repeatedly cited as one cause of the failures that came with growing scale.

The response to this crisis split into two broad directions. One was formal verification, pursued by people like Robert Floyd and C.A.R. Hoare — an approach that tries to mathematically prove a program is correct. The other was the empirical approach of testing: actually running the program to find errors. An early and important piece of work in this latter lineage is the 1963 paper “Systematic Mistake Analysis of Digital Computer Programs” by J. C. Miller and C. J. Maloney. This research, which used graph theory to analyze a program’s structure and systematically design test cases, is regarded as a forerunner to later concepts of path testing and coverage.

A Giant Project’s Struggle — OS/360 and The Mythical Man-Month

A case often cited as emblematic of the software crisis is the development of IBM’s mainframe operating system, OS/360. Fred Brooks, who served as project manager, collected his experience into the 1975 book The Mythical Man-Month.

Although the book is primarily about management, it concretely depicts how testing and integration work in a large-scale system balloons far beyond estimates, and why throwing more people at a schedule that’s already slipping tends to backfire (what’s now known as “Brooks’s law”). The fact that the difficulties of the testing and integration phases became widely shared knowledge is also one reason the industry came to see the need to treat testing as an independent discipline of its own.

Waterfall and the V-Model — Testing Gets Placed at “the End of the Process”

In 1970, Winston Royce presented a development model in his paper “Managing the Development of Large Software Systems,” moving in order through requirements, design, implementation, testing, and operation. This is the diagram that would later come to be called the “waterfall model.”

Ironically, Royce himself emphasized the need for iteration and feedback within that same paper, but later generations simplified and spread it as a single, linear process with testing tacked onto the end. Under this structure, testing tended to be treated as an “acceptance task that confirms, all at once at the very end, whether what was built is correct.”

The “V-model,” which pairs each development phase with a corresponding test phase (unit, integration, system, and acceptance testing) and diagrams them accordingly, is a more explicit rendering of this waterfall-style view of process. That said, it wasn’t until well after this prehistoric period — the mid-1980s — that it spread as a formal standard: West Germany’s defense ministry led the development of the V-Modell starting in 1986, made its use mandatory for defense projects in 1992, and formalized it as the federal government standard “V-Modell 97” in 1997.

Even so, the underlying idea — that there is a tier of testing corresponding to each stage of development — already existed in the waterfall-style view of process around 1970, which is the focus of this article. The structure of placing testing at the very end of the process would later invite the criticism that “testing only happens at the stage where rework is most expensive.” That same concern feeds directly into the modern principle of testing “early and often,” which we’ll cover in Episode 11.

Dijkstra’s Critique — “Testing Can Show the Presence of Bugs, But Never Their Absence”

Against this waterfall-style view of “testing everything, all at once, at the end,” the computer scientist Edsger W. Dijkstra raised a fundamental objection. The line widely attributed to him is this:

“Program testing can be used to show the presence of bugs, but never to show their absence!”

This claim is said to have taken clear shape in the unpublished notes “Notes on Structured Programming” (EWD249), written around 1970, after passing through an earlier variation seen in the proceedings of the 1969 NATO Software Engineering Conference.

Dijkstra’s point is that testing cannot prove exhaustive correctness: no matter how many finite test cases you stack up, you cannot guarantee the absence of errors for inputs you never tested. Building on this limitation, he argued that structured programming and formal correctness arguments deserved more emphasis than testing. This critique remains, to this day, the single most-cited line whenever anyone discusses the fundamental limits of software testing.

At the same time, it was Glenford Myers’s work, discussed next, that acknowledged this limitation while still systematically making the case that “testing is nevertheless useful.” Where Dijkstra pressed a theoretical limit, Myers can be said to have answered from a practical standpoint: even with that limit in place, testing done systematically still delivers plenty of value.

Glenford Myers’s The Art of Software Testing (1979) — Testing’s First Systematization

In 1979, Glenford J. Myers, who worked in software engineering at IBM, published The Art of Software Testing (John Wiley & Sons). This is considered one of the earliest systematic books devoted entirely to software testing.

The most famous definition Myers offers in the book is this:

Testing is the process of executing a program with the intent of finding errors.

This overturned the widely held understanding of the time that “testing is the work of showing a program behaves correctly according to its specification.” In Myers’s view, testing approached with the mindset of “confirming that it works correctly” unconsciously gets soft. Testing is effective precisely when it’s designed and executed with the destructive intent that “there must be a bug somewhere in this program, and I’m going to find it.”

This line of thinking led to a paradoxical evaluation criterion: “a successful test case is one that finds a bug.” This standard represents a shift in thinking — measuring a test’s value not by “I feel reassured because no bug showed up” but by “did it manage to find a bug” — and it remains one of the starting points of test-design thinking today.

Myers went further, devoting a chapter to “the psychology of testing” and pointing out the danger of a program’s own author testing that program. Because the author has trouble noticing the blind spots in code they wrote themselves, he argued for the value of testing, review, walkthroughs, and inspection carried out by an independent third party. He also introduced techniques for designing effective test cases within limited time, such as boundary value analysis and equivalence partitioning. These are practical techniques for answering a very concrete question — how do you narrow down a near-infinite space of input combinations into a manageable set of representative cases within limited time — and they remain part of the basic vocabulary of test design to this day.

Verification and Validation

That same year, 1979, Barry Boehm is credited with formalizing the following contrast in a paper.

Concept	Question	Meaning
Verification	Are we building the product right?	Are we building it correctly, per spec?
Validation	Are we building the right product?	Are we building the thing we should be building in the first place?

Most unit and integration testing falls in the territory of verification — confirming conformance to a specification. User interviews and acceptance testing, on the other hand, lie closer to validation. Passing every test amounts to achieving verification, not validation — something built correctly can still be the “wrong thing,” green tests notwithstanding. This distinction is the first of the four threads mentioned in the series overview, and every subsequent episode returns to it repeatedly.

The Rising-Cost-of-Defects Theory — Why Investment in Testing Got Justified

The claim that “the later a bug is found, the more expensive it is to fix” has long been cited as grounds for justifying investment in testing. In a 1976 paper and his 1981 book Software Engineering Economics, Boehm graphed the claim that defects found after release cost tens to a hundred times more to fix than defects found during requirements — the so-called “rising cost of defects curve.” This claim became, for a long time, a theoretical pillar for the idea that testing should be pushed as early as possible in the process.

However, the empirical grounding for this “curve” has come under doubt in recent years. In his book The Leprechauns of Software Engineering (2015), Laurent Bossavit scrutinized the studies Boehm’s claim was based on, and pointed out problems such as “generalizing from small-scale studies of students” and “citations traced back through secondhand sources with no way to find the original data,” concluding that the curve is “less an empirically supported fact than a graph of Boehm’s opinion.” This doesn’t refute the underlying intuition that catching something earlier is easier to fix, but it did bring a healthy skepticism to the industry about taking specific numbers — like “a production defect costs a hundred times what a requirements-stage defect costs” — at face value. Tracing testing history is also a reminder that even claims that spread as received wisdom can later get re-examined.

The Early Seeds of Test Automation

Testing in this era was, fundamentally, manual work. In batch processing on mainframes, the typical method was preparing test data on cards or tape, running the program, and comparing the output against expected values by eye or by hand. In COBOL and FORTRAN development shops, test cases were managed as paper documents, with staff working through checklists to confirm results manually. Test results came out as paper reports, checked line by line by human eyes — an entirely ordinary sight at the time. Simply re-running the same test took considerable effort, and the very sense of cost around “running a test repeatedly” was vastly different from today.

Even so, the seeds of automation weren’t entirely absent. Research like Miller and Maloney’s (1963), which analyzed a program’s structure as a graph to systematically derive test cases, can be seen as a forerunner of later coverage-measurement tools and test-case-generation tools.

Inside large mainframe vendors, simple mechanisms had also begun to appear for mechanically comparing past output against new output, for regression checking. Still, nothing like today’s xUnit — a common unit testing framework anyone could use — yet existed, and writing test code itself had not become recognized as a standard part of development work.

The Eve of Systematization — Where Things Stood by the End of the 1970s

By the end of the 1970s, the following foundation had been laid.

Aspect	State in the 1950s	State as of 1979
View of testing’s purpose	Undifferentiated from debugging; “it works” was enough	An independent activity pursued with the intent of finding errors
Who performs it	The programmer, checking ad hoc	The value of verification by someone other than the author is discussed
Techniques	None in particular, whatever gets it done	Techniques like boundary value analysis and equivalence partitioning are organized
Where it fits	Work fused with development itself	A discipline explicitly positioned within the development process
Awareness of limits	Barely considered at all	Dijkstra’s critique brings a widely shared awareness of fundamental limits

At this point, testing had finally begun to take shape as “a specialized field with its own theory and technique.” But the tooling to run tests routinely, automatically, and repeatedly still didn’t exist.

Looking back over these thirty years, testing as a practice made steady progress from a state where the very concept barely existed to a specialized field equipped with theory and technique. But that progress remained, strictly speaking, confined to building out the theory. Even a programmer who had learned boundary value analysis from Myers’s book still had to write test cases by hand, and had to keep re-running the same tests over and over at the same manual cost each time. The question “what makes a good test” was starting to get answered; the question “how do you keep running that test routinely, without friction” had not yet been answered by anyone.

It was Kent Beck, in the mid-1990s, who filled that gap — with a small framework he built for Smalltalk called SUnit, and the framework born from it, JUnit. The next episode, Episode 2, takes a close look at this turning point — along with the legendary story of JUnit supposedly being written on an airplane.