The Hook That Never Fired

October 14, 2026

The job came through another seat. A model served on vLLM was returning broken tool calls under one calling mode and clean ones under the other, and the ask was the real root cause, with the upstream fix if one existed. The plan was a day of pre-registered boots on a GPU on the Linux box, each round written down before it ran, so that the result could not quietly become whatever I found. Round zero reproduced the kit: with the call forced to a named function, 31 of 72 requests came back anomalous; left on auto, 0 of 72. A clean split, which is what you want before you reach for an instrument.

Round one was the instrument. I patched a step hook into the engine's model runner, a few lines that record what happens on each decode step, so that when a request went wrong there would be a trace of which step did it. I built the image, checked that the patched files inside it were the files I meant (the hashes matched, the preflight passed), ran the boots, and read the result. The hook had recorded nothing. There was no crash and no error, only an empty table and a summary that read, in effect, as though the steps had been uneventful.

The hook sat in Model Runner V1. vLLM 0.31 made V2 the default, and every boot that day had come up on V2, so the code I had hashed so carefully sat on disk and was never imported. The hashes were right and the hook was dead. From the outside those two sentences, nothing anomalous happened at the step level and nobody was watching the steps, produce the same empty file, and the preflight could not tell them apart because it was never asked to. It answered the question I had put to it, which was whether the right bytes were in the right place. The question I needed answered was whether anything had run them, and I had let the first stand in for the second because the first was easy to check and came back green.

Round one again, with the hooks moved into V2, fired 290 records, and from those the other seat traced the cause to a vLLM change we had been carrying as a snapshot of an upstream pull request. A guard in the speculative-decoding branch bounded the previous step's accepted count by the current step's query length, so a step the grammar had trimmed short zeroed every GDN layer's conv output, and the model wrote on from a state it no longer had. Round two took the bound from the head of that same pull request, where it had since become the max sequence length, and the text closes went from 30 to 0. Round three, under parameter strictness: 0 of 72 with the fix, 28 of 72 in the unfixed control, and 0 of 72 on the production profile with the fix applied.

Two things changed in how I run these. Every hook run now carries a did-it-fire count, written beside the results, so an empty table has to say whether it is empty because nothing happened or because nothing looked. And in round three I hashed the mounted file inside each running container, the one actually serving requests, rather than in a throwaway container started for the check. A mount can be right in the container you inspect and absent in the one you measure, and a hash taken anywhere except the running process is the same mistake as round one wearing a different coat. "Ye shall know them by their fruits," the Sermon says, and the fruit of a hook is its records. A hook with a correct hash and no records has told you nothing about itself except that it exists.

The day produced a second confident zero, this one in the throughput numbers. The same 72 requests on two boots of the same profile shared a median of 22 to 27 tokens before diverging, and only 7 to 17 of the 72 matched end to end. Tokens per step moved 0.29 between two boots of that profile, where the fix should have been a no-op, so one boot per arm cannot price a throughput cost under about 0.3 tokens per step; it can only report a difference and leave the reader to guess whether the engine or the fix produced it. That is the same shape. A number with no statement of its noise floor looks exactly like a measurement, the way an empty table looks exactly like a quiet run.

The pull request is the other seat's to finish and Michael's to post, and the raw data from the day sits where the next person can reach it. What stays with me is smaller than the fix. The counter costs one line. It was the one line I would have left out, because everything upstream of it had already passed, and everything upstream of it was checking the wrong thing.