← All posts
2026-07-28

Two clouds went down in 24 hours — and neither outage was in the code

On July 23, Microsoft Azure's West US region went dark for nearly five hours. Teams, SharePoint, OneDrive, Copilot Chat, Azure Kubernetes Service, and a dozen other services stopped responding. Nineteen downstream providers reported cascading failures. Thousands of users hit Downdetector. Microsoft's explanation: a bug in their maintenance automation software removed IP routes from more network devices than intended.

The next day — July 24 — AWS's US-West-2 region in Oregon lost connectivity for 80 minutes. Apple Pay, Reddit, Hulu, DoorDash, and PlayStation Network went offline. AWS traced it to a networking hardware failure at the boundary between their data centers and the public internet. It was their third reliability incident in eleven weeks.

Two of the three biggest cloud providers. Two outages in 24 hours. And in neither case was the problem in anyone's application code.

The automation that was supposed to prevent this

The Azure failure is the more instructive one. Microsoft's maintenance process has safety checks built in: before any maintenance work begins, the system verifies that at least one of two redundant network paths is healthy. Only then does it proceed. That's a reasonable safeguard — on paper.

The bug was in the step before the safety check. Microsoft's maintenance request conversion software — the tool that translates a human maintenance request into machine-readable instructions — incorrectly scoped the work. It marked additional network devices as part of the maintenance event. The system then faithfully executed the request as written, pulling IP routes from devices that should never have been touched.

The safety net was intact. The input to the safety net was wrong. The automation did exactly what it was told to do — and what it was told to do was destructive.

This is a pattern that keeps showing up. The Cloudflare outages earlier this year — three in four months — were all traced to internal configuration changes, not external attacks or software bugs. A BGP misconfiguration in February. A WAF testing tool accidentally disabled in December. A bot management file generation bug in November. Each one was the infrastructure team trying to improve the system and breaking it in the process.

Your tests passed. Your platform disappeared.

Here's what makes this class of failure different from the ones we usually write about. When Amazon's site went down in March because of AI-generated code, you could point to a missing code review. When Shopify failed on Cyber Monday, you could point to a missing load test. Those are failures in the application layer — failures that better testing of the application would have caught.

Azure and AWS didn't fail in the application layer. Every app running on those platforms had passing tests, green CI pipelines, and clean deployments. The code was fine. The platform underneath it stopped existing.

You cannot unit-test your way out of this. You cannot integration-test your way out of it either, because the integration environment is the one that disappeared. The only test that catches a platform-level failure is one that runs from the outside — a real HTTP request, to a real URL, checking for a real response — on a schedule frequent enough that "the entire West US region is unreachable" gets caught before your users report it.

The third time in eleven weeks

The AWS story is quieter but arguably more concerning. July 24 was the third US-West-2 incident since May. Three distinct failures in the same region in under three months. Each had a different root cause. Each affected a different set of downstream services. The common thread is that the region itself — not any application running in it — was the point of failure.

AWS attributed the July outage to networking hardware between Oregon and Seattle. Traffic that both started and ended inside the region kept working. Anything crossing the region boundary timed out. The failure lasted 80 minutes, with routes flapping again during recovery as new paths settled.

Eighty minutes doesn't sound like much until you calculate what Apple Pay, DoorDash, and Reddit lose per minute of downtime. Or until you realize that your application's health check was running inside the region and reporting healthy the entire time the outside world couldn't reach it.

This is the blind spot in almost every monitoring setup. Health checks run where the application runs — inside the cloud. If the problem is between the cloud and the internet, internal checks stay green while real users get timeouts. Your dashboard says everything is fine. Your revenue says otherwise.

What's actually different now

Cloud outages aren't new. What's changed is the frequency and the cause. The 2024 CrowdStrike incident was an outlier — a catastrophic update from a security vendor. But the 2026 pattern is different. These are the cloud providers themselves, breaking their own infrastructure through routine maintenance and configuration changes. Not attacks. Not bugs in application code. Operational changes to the platform, by the people who run the platform, with consequences nobody predicted.

The H1 2026 cloud reliability report from IncidentHub tracked cascading failures across providers at a rate that would have been alarming two years ago. Azure's July 23 outage alone cascaded to 19 confirmed downstream incidents across 7 providers. That's one maintenance bug, one region, nearly 5 hours, 19 service failures.

And the tooling most teams rely on to catch outages — cloud-native monitoring, provider status pages, internal health checks — is designed to detect application failures, not platform failures. Your APM tool runs on the same cloud that just went down. Your status page subscription emails route through the same network that's unreachable. By the time the provider confirms the incident on their status page, your users have already found out the hard way.

The boring test that catches it

The test that would have caught both of these — for any team running on Azure West US or AWS US-West-2 — is embarrassingly simple:

  1. Hit your real URL from outside the cloud. Not a synthetic health check running in the same region. A real HTTP request, from the public internet, to the URL your customers use.
  2. Check that the response is what you expect. Not just a 200 status code. Load the page. Verify the content renders. Confirm the login works. An infrastructure failure can return a 200 from a CDN cache while the actual application is unreachable.
  3. Run it every few minutes. A test that runs hourly would have missed the 80-minute AWS outage entirely. A test running every 5 minutes catches it within 5 minutes — early enough to activate your failover, notify your support team, or at minimum put up a status page before the reports start flooding in.

This isn't sophisticated. It's a recorded user flow — open the app, log in, check the dashboard — replayed on a cron from a location that isn't inside your cloud provider. When it fails, you get an alert with a screenshot of exactly what your users are seeing. When Azure's West US routes get wiped by a maintenance bug, you know within minutes, not when the status page updates an hour later.

The real monitoring gap

The testing industry spent 2026 building AI agents that generate tests, heal selectors, and plan test suites autonomously. Those tools are solving a real problem — maintaining large test suites is genuinely tedious. But they're solving it at the wrong layer.

The outages that actually hurt — the ones that make the news, trigger Senate hearings, and cost millions per hour — aren't failures in application code. They're infrastructure collapses, maintenance mishaps, and configuration errors in the platforms everything else depends on. And the only way to detect those is to test from the outside, against the real system, on a schedule that doesn't depend on the system being tested.

Two clouds went down last week. Every app on both platforms had tests. None of those tests caught it, because they were all running inside the thing that broke.