The SRE Book Era (Jul 2015 – Sep 2016) Google published Site Reliability Engineering in 2016 and gave the industry a shared vocabulary: SLOs, error budgets, toil, blameless postmortems. At the same time, the era’s biggest outages kept showing the same pattern. Systems broke not from the original fault, but from how they tried to recover.
The incidents that defined the period AWS DynamoDB, September 20, 2015. The classic cascading-failure postmortem (aws.amazon.com/message/5467D2). A network blip made storage servers all re-request their membership data at once. The metadata service was already near capacity from a new feature and couldn’t keep up, and retries made it worse. The outage spread from DynamoDB into EC2, SQS, and CloudWatch in us-east-1. The fixes (more headroom, longer timeouts, smarter retries) read like a distributed-systems checklist. Salesforce NA14, May 2016. A database failure plus a failed failover left a major instance degraded for almost a day, with some data lost for good. It put “your SaaS vendor’s DR plan is your DR plan” into procurement conversations. Southwest (July 2016) and Delta (August 2016). Back-to-back airline meltdowns from single points of failure (a failed router, a datacenter power problem) cancelled thousands of flights. Boards started asking about technical debt. Telstra, 2016. A run of national mobile outages in Australia, one caused by taking a single node offline the wrong way, made the telco postmortem press release routine. What the postmortems reveal Retry storms got a name and a set of standard fixes. The DynamoDB writeup made this failure pattern mainstream well before the academic term “metastable failure” existed. Exponential backoff, jitter, circuit breakers, and load shedding moved out of Netflix blog posts and into default library behavior.
...