The Loop That Measured the Language

The server’s memory problem was tangled in threads, channels, and a store — too many moving parts to reason about a fix. The method said: isolate. The minimal program with the same disease is a single-threaded loop that reads a line, looks at it, and forgets it — wc -l in the language itself, fed millions of lines, with resident memory as the only output that matters. If per-item reclamation is ever going to work, it has to work here first. And the probe earned its keep before the first measurement, in a familiar way: the language had a read-a-line builtin, but only in the interpreter — the sixth member of the interpreter-only family that native programs keep discovering. read_stdin slurps the whole input by design; a native streaming line reader could not be written at all until the C backend learned read_line. One small release later, the measuring could begin.

Two dialects, two wrong numbers

The string-dialect counter — idiomatic code, read_line and str_len — showed exactly the expected disease: resident memory tracking input size, about thirty permanent bytes per line, a quarter-gigabyte to count eight million lines that wc counts in two and a half megabytes. Every line string lands in the program-lifetime region and stays. That number was the baseline the eventual fix would be judged against. The control experiment was supposed to flatter the language: the byte dialect — tcp_read into one reused buffer, counting newlines with mem_get_u8, no strings at all — should have run in constant memory. It ran in two gigabytes. The frugal version was ten times worse than the wasteful one, and no story anyone believed about the language could explain it. This is what measurement is for: a number that contradicts everyone’s model is a bug’s home address.

Every call was paying rent

Bisecting the anomaly led somewhere no one had looked, because it had been hiding inside the most ordinary code shape in the language. A multi-parameter function in an ML is a chain of single-argument functions, and the C backend compiled every call level by level: count buf i n acc became “call count with buf, get a closure back, feed it i, get a closure back…” — and each step allocated a closure environment in the program-lifetime region, through the region’s thread lock, on every call. A four-argument function in a loop over three hundred million bytes allocated three environments per byte, forever. Every hot loop in every dogfood had been paying this rent invisibly; the byte-dialect probe just paid it three hundred million times in a row. The fix is the classic one — for each top-level curried function, also emit a direct N-ary C function, and compile exactly-saturated calls straight to it, pinning the interpreter’s left-to-right argument order with temporaries because C’s is unspecified. Partial applications keep the closure chain; nothing observable changes. What changes is the meter: no environments, no lock traffic, and self-recursion becomes a genuine C tail call.

Leaner than wc, and an honest split

With uncurrying in place the byte-dialect counter processed three hundred megabytes in a tenth of a second at one and a half megabytes of memory, constant across input size — below wc itself. The number is pleasing; the diagnosis mattered more. The string dialect re-measured at essentially the same quarter-gigabyte as before, which was the point: the two problems had been cleanly separated. One was an accident — a compilation strategy silently taxing every saturated call, now abolished for good. The other was the real design debt: idiomatic string code still had no way to give per-line memory back, because the values themselves, not the calling convention, were permanent. The probe had done what probes do — turned “the server leaks somehow” into two precise statements, one already fixed and one now unavoidable. The next episodes take on the unavoidable one, but not before a detour: while the memory work was being staged, a generic data structure quietly demonstrated that the interpreter and the compiler didn’t always agree on what a program means.