The 2021 Facebook outage

On 4 October 2021, at around 15:40 UTC, Facebook, Instagram, WhatsApp, Messenger and Oculus all stopped working at the same moment. They stayed that way for roughly six hours.

What made it remarkable was not the duration. It was the mechanism: Meta did not go down in the ordinary sense of servers failing. It vanished. From the perspective of every other network on the internet, Meta's addresses ceased to exist.

BGP, briefly

The internet is not one network. It is around a hundred thousand independent networks that agree to carry each other's traffic, and the protocol they use to tell each other what they can reach is the Border Gateway Protocol.

A network announces, in effect: "send me anything destined for these address ranges." Neighbouring networks believe it, pass the announcement on, and within minutes the whole internet knows where to send that traffic.

The corollary is the important part. If a network stops announcing its ranges, those addresses stop being reachable from anywhere. Not slow. Not erroring. There is no longer a path.

What happened

According to Meta's own published account, engineers were performing routine maintenance intended to assess the capacity of the global backbone. A command was issued that was meant to evaluate the availability of backbone capacity. It unintentionally took down all of the connections in the backbone network.

An audit tool existed specifically to catch commands like this. It had a bug, and it did not stop the command.

With the backbone gone, Meta's data centres could not reach each other. And then the second-order effect that turned a bad situation into a catastrophic one: Meta's DNS servers are designed to withdraw their BGP announcements if they lose their own connection to the data centres, on the reasonable theory that a nameserver which cannot reach anything should stop advertising itself.

Every one of them lost that connection simultaneously. Every one of them withdrew. And in doing so, Meta's authoritative DNS disappeared from the internet.

Why that was worse than servers failing

Once the authoritative nameservers for facebook.com were unreachable, no resolver anywhere on Earth could answer a query for any Facebook hostname. Not with a stale answer, not with an error from the site - With NXDOMAIN, the definitive "this domain does not exist".

The servers were running the whole time. The data centres had power. The applications were healthy. And there was no way for anyone to find out, because finding out requires DNS.

The failure also produced a global side effect. Every device with a Facebook, Instagram or WhatsApp app installed began retrying, and each retry was a DNS query for a name that could not be resolved. Public resolvers reported enormous spikes in traffic, and other sites were measurably slowed by the load - An outage at one company degrading the resolution of everybody else's.

Why it took six hours

Because the network that failed was the network needed to fix it.

Meta's internal tools - The systems engineers use to access and reconfigure infrastructure remotely - Ran on the same backbone. When it went, remote access went with it.

Physical access was also affected. The badge readers and internal directory systems that control access to Meta's data centres were on the same network. Reporting at the time described staff unable to enter buildings, and Meta's own account confirms that recovery required sending engineers physically to data centres, and that those facilities are deliberately hard to get into, with hardware and routers designed to be difficult to modify even with physical access.

Bringing the backbone back was not simply a matter of reversing the command either. Restoring power and connectivity to everything at once risked a surge that could damage equipment, so recovery was staged deliberately.

What it cost

Meta's share price fell sharply on the day, wiping tens of billions from its market value. Revenue loss for the outage itself was estimated in the region of $60–100 million.

The wider cost is harder to count and was probably larger. In much of the world WhatsApp is the primary way people contact businesses and government services. Six hours without it meant small businesses could not take orders, families could not reach each other across borders, and in several countries essential coordination simply stopped. "Log in with Facebook" broke across thousands of unrelated sites.

What changed

The technical lessons were widely adopted, and they are the reason this outage is still discussed.

Out-of-band access. Recovery access must not depend on the network being recovered. Separate management networks, cellular backup and console servers became standard where they had previously been optional.

Physical access must not depend on the corporate network. Badge systems with offline fallback are now a specific item on disaster-recovery checklists at large operators.

DNS withdrawal logic needs a floor. The self-withdrawal behaviour was individually sensible and collectively catastrophic. Designs now generally keep some authoritative capacity announcing even under partial failure.

Audit tools are load-bearing. A safety system with a bug is worse than no safety system, because people rely on it.

The broader point

The outage is a clean demonstration of something this site returns to repeatedly: At scale, the dangerous failures are not hardware failures. They are changes made deliberately, executing correctly, in a system whose dependencies nobody had fully mapped.

Meta employs some of the best infrastructure engineers in the world and has redundancy at every layer. It still spent six hours locked out of its own buildings because a command did exactly what it said and a safety check did not run. That is the normal shape of a large outage, not an aberration.