Diagram showing Azure global network outage with failures in North America, Europe, and Asia

Microsoft 365 Outage: Azure Maintenance Bug Wiped IP Routes, Taking Down Teams for Hours



Microsoft’s automated network maintenance system contained a bug that caused it to remove IP routes from far more devices than intended during a routine maintenance event on July 23 — stranding a West US Azure datacenter from Microsoft’s global wide-area network and knocking out Teams, SharePoint, OneDrive, Copilot Chat, and more than a dozen other cloud services for enterprise users across North America for nearly five hours according to Microsoft’s preliminary Post Incident Review.

The disruption, which started at 10:44 a.m. ET and was not fully resolved until 3:41 p.m. ET, generated 2,403 user reports on Downdetector by 11:11 a.m. — 83 times the platform’s typical baseline of 29 reports, BleepingComputer reported. IncidentHub, which monitors cloud service status pages, detected cascading outages in at least 19 downstream SaaS platforms whose infrastructure routes through Azure’s West US region — and 16 of those 19 services still had open incidents after Microsoft declared full recovery, IncidentHub’s analysis found.

How a Routine Task Corrupted Its Own Safety Net

Microsoft’s preliminary Post Incident Review (PIR) identifies the failure’s origin with unusual specificity: the bug was not in the network hardware itself, but in the software layer that translates human maintenance requests into machine-readable instructions according to Microsoft’s PIR.

Here is how the failure chain worked. Engineers issued a routine maintenance request to isolate specific network paths in the West US region — a standard procedure designed to be low-risk. Microsoft’s system converts those requests into instructions the infrastructure can act on and runs a safety check: it verifies that at least one of two redundant paths will remain healthy before the maintenance begins. That safety check existed precisely to prevent this kind of disruption.

But the request conversion system contained a bug identified in Microsoft’s PIR. It incorrectly marked additional network devices as part of the maintenance scope — devices that were not supposed to be touched. Because the safety check evaluated the buggy conversion’s output rather than the engineers’ original intent, it did not catch the error. The check passed. The maintenance proceeded. IP routes were removed from more devices than anyone had authorized, severing the connection between a West US datacenter and Microsoft’s wide-area network (WAN).

A wide-area network is the private backbone connecting Microsoft’s globally distributed datacenters. All user traffic flowing into or out of an Azure region must traverse that backbone. When the WAN link between the West US datacenter and the backbone was cut, the datacenter became an island: services running entirely inside it kept working normally, but any traffic that needed to enter or leave hit a dead end.

The failure initially appeared to Microsoft’s engineers as "large-scale route churn" in the WAN — routers rapidly removing and re-advertising routes, a symptom that can indicate a misconfigured or buggy automation system executing a change repeatedly per Microsoft’s PIR. Tracing that churn back to a specific datacenter in the West US region took more than three hours.

This pattern — a safety check that validates the conversion layer’s output rather than the original human intent — is the architectural gap Microsoft’s forthcoming final PIR will need to address. A bug that corrupts the conversion can produce a correctly-executed but incorrectly-scoped change that defeats all safety checks, leaving the safety net with a structural blind spot.

What Failed and When: A Five-Hour Cascade

The blast radius was wide. Microsoft confirmed disruptions across more than a dozen products and services under incident ID MO1437424:

SharePoint Online returned a generic "Something went wrong" error when users tried to open or share files — accounting for 78% of Downdetector complaints. Microsoft Teams’ chat degraded significantly, with images failing to load. OneDrive access became intermittent. The Microsoft 365 Admin Center — the tool IT administrators needed to investigate and respond — loaded slowly or not at all. Power Automate workflows failed to execute. Copilot Chat experienced delays and failures across queries. Microsoft Loop pages would not open.

The outage also reached deep into Azure infrastructure: connectivity failures and elevated latency spread across Azure App Service, Application Gateway, Azure AI Search, Azure API Management, Azure Cosmos DB, Azure Databricks, Azure Firewall, Azure Kubernetes Service, Azure Monitor, Azure Virtual Desktop, ExpressRoute, Log Analytics, Microsoft Graph, Microsoft Sentinel, Power BI Embedded, Virtual WAN, and VPN Gateway per Microsoft’s PIR.

Microsoft Defender customers experienced delays receiving responses from Defender Experts during the window, and some threat investigation workflows triggered through Threat Explorer and Advanced Hunting failed entirely — meaning security operations teams lost visibility during a period when they needed it most, BleepingComputer reported.

The timeline of the engineering response, drawn from Microsoft’s own PIR, reflects how long it takes to diagnose a novel failure mode even with an experienced team actively working it:

At 10:44 a.m. ET, device maintenance began and the first customer impact started. By 10:45 a.m., network and incident response engineers were reviewing traffic anomalies and packet loss. From 11:00 a.m. to noon ET, engineers validated the scope and blast radius while investigating the large-scale WAN route churn. From noon to 1:45 p.m. ET, the investigation narrowed to abnormal routing behavior tied to improper route advertisement, which engineers then correlated with the recent maintenance activity. Microsoft initiated a rollback of the maintenance change at 1:45 p.m. ET. The rollback completed at 2:26 p.m. ET. Full recovery across all affected services was confirmed at 3:41 p.m. ET, as documented in Microsoft’s preliminary PIR.

An initial attempt to reroute traffic through alternate network paths provided partial relief but was not sufficient to restore service for most affected users. Mid-outage, Microsoft issued an advisory urging enterprise customers to review their business continuity and disaster recovery plans — an unusual step that signaled to IT teams that mitigation was not imminent.

Downstream Services Felt It Longer Than Microsoft Did

When Microsoft declared full recovery at 3:41 p.m. ET, the disruption was not over for organizations whose SaaS vendors run infrastructure in Azure’s West US region.

IncidentHub, which monitors the status pages of hundreds of cloud services, tracked cascading outages in at least 19 downstream services that acknowledged impact tied to the Azure incident. Sixteen of those 19 still had active incidents after Microsoft’s recovery announcement per IncidentHub’s analysis. Acknowledgement lag among downstream vendors varied from a few minutes to more than three hours after Azure’s impact began.

Named examples illustrate the breadth of the cascade. Datadog reported that Azure metrics might be delayed due to a partial outage in the Azure Monitor API. MongoDB Cloud saw cluster connectivity failures for clusters hosted in Azure West US, with nodes appearing incorrectly offline and cluster modifications delayed. Wiz reported portal and API degradation across multiple US data centers. Ellucian Cloud’s Student Financial Success products — serving educational institutions — went offline until Azure recovered. MeridianLink opened five separate incidents, each covering a different affected product component, IncidentHub documented.

The recovery lag pattern follows from how distributed systems restart after a large infrastructure event: once Azure’s network was healthy again, downstream services faced their own reconnection queues, backed-up API calls, and state reconciliation before declaring themselves recovered.

For enterprise IT teams, this lag reveals a risk that SLA reviews often miss. An organization’s own Microsoft 365 service may be covered by the 99.9% uptime guarantee for core services like Exchange Online, SharePoint Online, and Teams. But the SaaS tools that integrate with those services — workflow automation platforms, data connectors, security tools — carry their own dependency chains and their own recovery timelines, and none of those downstream SLAs account for the blast radius of an upstream cloud failure.

Part of a Documented 2026 Reliability Pattern

The July 23 event did not arrive in a vacuum. Microsoft 365 delivered only 99.526% uptime in Q1 2026 — the lowest quarterly figure since analysts began tracking it in 2013, and a figure that consumed nearly the entire year’s SLA budget in a single quarter, as TechTimes previously reported.

IncidentHub founder Hrishikesh Barua, in his analysis of the July 23 outage, placed it in a broader pattern: the 2025-2026 automation and change-management failures share a common thread — safety checks existed, but a bug somewhere in the workflow defeated them, as Barua observed.

That characterization fits. The February 2026 Cloudflare BYOIP outage involved automation software mistakenly withdrawing customer BGP routes — safety mechanisms worked as designed but could not prevent self-inflicted configuration errors in the automation layer. The May 2026 Azure OpenAI outage involved a retry amplification bug in an upstream API change that overwhelmed a shared load balancer — safety mechanisms existed but the failure propagated through a gap in how internal workloads were protected. The July 23 Microsoft 365 outage adds a third variant: a conversion layer bug that over-scoped a maintenance change before safety checks could evaluate the true intended scope, as Microsoft’s PIR confirms.

Each event is distinct in its technical details. Together they form a recognizable class: large-scale infrastructure outages caused not by hardware failure, external attack, or deliberate human error — but by automation systems that behave differently than their designers intended, in ways that defeat the safety mechanisms built to catch exactly that kind of error.

What Microsoft Has Committed to Do Next

Microsoft confirmed it is conducting a full internal review focused on the safety checks and automated processes used to execute maintenance requests, and that it will publish a final Post Incident Review after completing that investigation — typically within 14 days of the event, placing the expected deadline around early August 2026 per Microsoft’s preliminary PIR.

In the PIR, Microsoft explicitly flagged that the review will focus on "safety checks, automated maintenance request change process, and more." That language tracks the structural gap identified above: the safety check’s inability to catch a conversion-layer bug. Whether the company addresses that gap architecturally — by validating human intent independently of the conversion layer’s output — or operationally through additional approval gates remains to be seen.

What Enterprise IT Teams Can Do Before the Final PIR Arrives

The July 23 outage offers three concrete signals for enterprise IT and cloud architecture teams, each of which applies regardless of the final PIR’s findings.

First: Microsoft’s mid-outage advisory to review business continuity plans was not routine language. It was a signal that rerouting — the standard first-response mitigation — was not going to resolve the situation quickly. Organizations that had documented fallback communication channels (an alternative messaging platform, an out-of-band email system, a backup file-sharing path) experienced less disruption than those that did not.

Second: Microsoft explicitly recommended multi-region deployment strategies in the PIR itself, noting that for mission-critical workloads, customers should consider architectures that maintain availability during events affecting a single region per Microsoft’s PIR. A West US outage that has no impact on a workload with active failover in East US is a different category of risk from one that takes down a single-region deployment.

Third: The 99.9% SLA for Microsoft 365 core services promises uptime, not data protection and not compensation calibrated to business impact. The SLA credit structure — 25% of monthly fees if uptime falls below 99.9% — provides a fraction of the actual cost of an extended outage to most enterprise deployments. Organizations that have not revisited their risk acceptance posture in light of Microsoft’s 2026 reliability figures should do so before the next event, not after.


Frequently Asked Questions

What caused the Microsoft 365 outage on July 23, 2026?

A bug in Microsoft’s automated network maintenance request conversion system caused it to mark more network devices as in-scope for a routine maintenance task than engineers intended. That caused IP routes to be removed from those extra devices, severing the connection between a West US Azure datacenter and Microsoft’s wide-area network backbone. Because all user traffic must traverse that backbone to enter or exit the region, the datacenter became unreachable from the internet for nearly five hours.

Why didn’t Microsoft’s safety checks catch the bug before it caused the outage?

Microsoft’s maintenance process includes a safety check that verifies at least one of two redundant network paths will stay healthy before maintenance begins. The problem is that this check evaluated the output of the request conversion layer — which already contained the bug. Because the check ran on the buggy instructions rather than on the engineers’ original intent, it saw a valid-looking request and passed. The safety net had a structural blind spot: it could not catch a bug that occurred upstream of where the check ran. Microsoft’s forthcoming final Post Incident Review is expected to address this gap.

How can I check whether Microsoft 365 services are experiencing an outage?

The fastest authoritative source is the Microsoft 365 Service Health dashboard at admin.microsoft.com. Enterprise administrators can also configure Microsoft Service Health alerts to route to Slack, email, Microsoft Teams (via a separate channel), webhooks, or PagerDuty — so notifications reach the operations team even when Teams itself is unavailable. Downdetector provides independent user-report tracking as a secondary signal.

Should organizations move away from Microsoft 365 after this outage?

The more useful question is whether your organization’s architecture assumes Microsoft 365 will always be available. The July 23 event and the three-month reliability pattern leading up to it — including the 99.526% Q1 2026 uptime figure — suggest that single-provider cloud concentration creates outage risk that SLA credits do not fully offset. Multi-region deployments within Azure reduce regional failure impact; maintaining alternative communication and file-sharing tools provides a fallback when the primary platform is down. Neither requires abandoning Microsoft 365 — they require treating its availability as probabilistic rather than guaranteed.

Original Post>

Enjoyed this article? Sign up for our newsletter to receive regular insights and stay connected.

Leave a Reply