Patch Notes #123 — This Is Not a Drill (It Was a Drill)

Last Saturday morning, every phone in Hawaii received: “BALLISTIC MISSILE THREAT INBOUND TO HAWAII. SEEK IMMEDIATE SHELTER. THIS IS NOT A DRILL.” For THIRTY-EIGHT MINUTES, it stood. People put children in storm drains. Said goodbye. And the cause, when it emerged, was the most professionally humbling artifact of the year: an employee running a shift-change drill selected the wrong item from a dropdown menu — the real-alert option living adjacent to the test option, same list, similar labels, with a confirmation dialog that (like all confirmation dialogs) had been trained into muscle-memory clickthrough. No cancel template existed; one had to be COMPOSED, mid-crisis, over 38 minutes.

This blog has a whole shelf for this: Joyent’s datacenter reboot (#032), the S3 typo (#102). The operator is never the bug; the dropdown is the bug. Test and production sharing a menu is the bug. The missing undo is the CATASTROPHIC bug. Every engineer I know read that story and went hunting through their own admin panels for dropdowns where “drill” sits one arrow-key from “apocalypse.” We found two. Fixed by Wednesday.

Elsewhere: crypto’s biblical correction (#110, #121) has begun in earnest — Bitcoin roughly halved from December’s peak, altcoins worse, my barber gone quiet. The date, previously unknowable, is looking a lot like “the top was the futures launch.”

TIL: forcing functions in UI — physical interlocks, typed confirmations (“type the environment name to proceed”), and separated blast radii. Guardrails over discipline (#032), now with a missile-alert case study for the training deck.