What a Missing Feature Hides: Two Plain Bugs Under One Structural Gap

mere-ruby is a Ruby-subset interpreter written in Mere, a small ML-family language I am building. Its conformance record is a checked-in table: every group of ruby/spec run against the real ruby, compared byte for byte.

core/io was one of the worst groups in that table — 45 of 81 files matching, thirty-five diverging. The reason looked settled, and it was written down in the project’s own gap list: a File here is a path, a mode and a buffer. It has no file descriptor. #fileno does not return a made-up number, it refuses and says why.

That is a real limitation, and it explains a lot. So the obvious next task was to give the interpreter real descriptors — open files properly, keep an offset, build the IO surface on top. A few days of work, structural, satisfying.

I measured first, and the measurement said something else.

What the descriptor would actually buy

The list of IO methods this interpreter does not answer is 43 names long. I split it by one question: does this need a file descriptor, or only a path and a position?

needs a descriptor   19   advise autoclose? close_on_exec? fcntl fdatasync
                          for_fd fsync ioctl pid popen read_nonblock sysopen
                          sysseek timeout to_i wait_priority wait_readable
                          wait_writable write_nonblock

needs no descriptor  24   binmode binmode? close_read close_write copy_stream
                          eof eof? lineno open pos pread printf pwrite reopen
                          rewind seek set_encoding set_encoding_by_bom stat
                          syswrite tell to_io try_convert ungetbyte ungetc

Twenty-four of the forty-three are expressible with what the interpreter already has. And the thirty-five failing files break down the same way: fifteen are a missing method, eight are the explicit “there is no descriptor” refusal. A descriptor would move about eight rows.

Against that: a descriptor changes the model. The File is a path and a buffer today, and every read and write path is written that way. Descriptors and whole-file buffering do not coexist — adding one means rewriting the other. A few days of work to move eight rows, with the other twenty-four names still unwritten at the end of it.

So I did the twenty-four first. Not because it was bigger, but because it was the part that did not require deciding anything.

The two bugs underneath

Writing the byte-at-a-time readers meant calling read(1) in a loop. The first one I wrote returned the entire file.

f = File.open("/tmp/ten-bytes.txt")
f.read(3)   # "abcdefghij"   <- every byte
f.pos       # 10
f.read(3)   # ""

File#read ignored its length argument. It read the whole file, set the position to the end, and the next read saw nothing. Any code that reads in chunks — which is most code that reads at all — saw one giant chunk followed by end-of-file.

The class method had the same shape:

File.binread(path, 3, 2)   # "abcdefghij", where ruby answers "cde"

Both had been there for a long time. Both are now fixed and checked against the reference implementation, which is the oracle this project uses for everything.

Neither has anything to do with file descriptors.

Why they were invisible

This is the part I want to keep.

A structural explanation is comfortable. “IO is weak because there is no descriptor” is true, it is specific, and it accounts for the shape of the failure. Once you have it, you stop looking — not deliberately, but because the question feels answered. The group is bad for a reason you already know.

But read ignoring its length is not that reason. It was sitting in the same group, behind the same headline, and the headline was covering for it.

And the order of work decides whether you find it. Had I built descriptors first, read(len) would have been rewritten as part of that work. The bug would have disappeared into a diff of several hundred lines — fixed, probably, but never seen. I would not have learned that the length argument had been ignored all along, and I would have attributed the improvement to the descriptors.

That is the difference between a fix and a finding. A fix changes the code. A finding changes what you believe about the code, and that belief is what you use to choose the next task.

The general shape

I now think of it this way: a missing capability is a shadow, and ordinary bugs sit in it.

The shadow is cast by anything that stops people from reaching the code:

• a class that does not exist, so nothing behind it is exercised
• a method that raises NotImplementedError, so its callers never run
• a feature everyone knows is absent, so its area is never reported

While a caller cannot get in, whatever is inside is unmeasured. The moment the door opens, what was behind it gets tested for the first time — and some of it was never about the door.

I had seen this before on the same project without recognising it. Adding one missing method to the gzip writer — a constructor the suite uses — made a previously-unreachable spec run, and it immediately hit the harness’s CPU limit. The compression engine turned out to run at about 34 seconds per megabyte. That cost had always been there. Nothing had reached it, so nothing had measured it, and the record had shown a clean missing-method error where the truth was a missing method and a performance problem.

What I would tell myself

Three things, in the order they matter.

Measure the explanation, not just the symptom. “IO is weak because there is no descriptor” is a claim about how many rows a descriptor would move. That number is countable before any code is written. It was eight of thirty-five.

Prefer the work that requires no decision. Twenty-four names needed no design judgment — they were expressible with what already existed. The descriptor needed a decision about the whole File model. Doing the unblocked work first is not procrastination; it produces the measurements that the decision needs.

Treat the first thing you find after opening a door as evidence, not noise. When a newly-reachable area immediately fails in some unrelated way, that is not bad luck. That is the shadow clearing.

The descriptor decision is still open. But I will make it with a number now instead of an intuition, and with two fewer bugs hiding behind it.