ListRun's sweep was flagged stuck 11 times - it was right every time

ListRun's sweep was flagged stuck 11 times - it was right every time

A watchdog reported ListRun's morning sweep as a stuck job eleven times, and the sweep had been right every one of those mornings

We were told one of our own morning jobs was stuck in a loop, and it had been doing exactly the right thing, every day, for eleven days. The report was not wrong about what it saw. It counted the same line eleven times and called that a job repeating itself inside a day.

launchd appends every run of that job to one file, and that file is never rotated. Nothing the job printed carried a date. The watchdog that reads it dates an undated line by the last stamped line above it, so with no stamp anywhere in the file, its twenty four hour window had nothing to anchor to. Eleven separate mornings collapsed into one.

The number was real. The window was not

ListRun is a directory submission service, and the morning job behind it is a freshness sweep: one pass over 140 third party sites, every day at 05:10, so the sentence "checked today" on the free list is true when a reader sees it.

The line that got counted said a listing could not be read by an ordinary automated request, printed as a bare three digit number of the kind a website answers with. That is a healthy outcome. The sweep had already decided to keep the listing on the public list. But every failure check we run treats a bare code like that as a fault, so the line went out as a symptom with no verdict attached to it, eleven mornings running, identical each time. Two properties of the same board multiplied: it could not tell the mornings apart, and it could not tell a decision from a defect.

What each of these numbers is actually counting