What Broke GitHub on August 17 and How Retries Made the Incident Worse

GitHub’s incident on 17 August 2026 ran from 13:28 to 21:15 UTC, seven hours and forty-seven minutes. At peak, web and API traffic saw error rates of around 20%, while archive and raw-content downloads reached roughly 50%. SAML and OIDC authentication, SCIM and Team Sync were affected alongside Git operations, Actions, Pages, Issues, Pull Requests and Copilot.

GitHub has now published its root cause analysis, and the most interesting part isn’t what failed but what happened after.

The initial failure was contained. The retries made it worse.

GitHub says the immediate cause was network saturation on load balancers in its Central US datacenter during a new peak in traffic, which traced back to an Istio sidecar pod that reached its concurrency limit and failed to autoscale correctly. The autoscaling failed because its scaling policy was watching the capacity of the host service, rather than the sidecar itself. In short, the thing meant to add capacity wasn’t watching the part that had actually run out of it.

That initial failure continued to cascade until four HAProxy nodes exhausted their flow limits, degrading GitHub’s gateway authentication path and causing the widespread authentication latency and failures that made up most of the incident’s visible impact; however the capacity problem was only half the story. GitHub says it was worsened by optimistic retry logic that overloaded its own internal load balancers, and pausing HAProxy on the affected nodes is what produced what GitHub calls an “immediate broad recovery.” The system was already struggling to process requests, so instead of retries helping it catch up, it actually added more work to infrastructure that already couldn’t keep up with what it had.

The same failure pattern appeared again during recovery

GitHub moved some of the affected traffic from Central US to Northern Virginia while the original network problem was being investigated, and at first that worked. Then delayed responses from a single internal endpoint triggered a separate, latent retry bug in VS Code, which amplified traffic to the Copilot Token Service by roughly 10x and delayed its recovery specifically. Whilst normal traffic to that service runs at 7,000–9,000 requests per second; during the incident it reached 70,000–100,000, as a failed token operation generated multiple extra requests that themselves entered a retry loop. GitHub stabilized it the same way it stabilized the first failure: reduce the retry logic, block the responses that were triggering more retries, then ramp traffic back up gradually rather than all at once.

Two unrelated systems, two separate retry mechanisms, the same failure pattern, on the same day.

Scraping traffic made recovery harder too

GitHub also says a number of scraping attacks against codeload endpoints complicated its recovery. Whatever the intent behind that traffic, its effect on an already constrained system was the same as the retry storms: more load arriving at the exact point the system was least able to absorb it. Retry storms and unexpected external traffic are different problems that expose the same weakness, that systems built for normal conditions don’t necessarily behave the same way once capacity is already under pressure.

What this is actually about

None of this is unique to GitHub. Any distributed system that retries failed requests can make a bad situation worse if those retries arrive faster than the recovering service can absorb them, and GitHub’s own follow-up work reflects exactly that: correcting the autoscaling policy, auditing Istio concurrency and scaling limits, reviewing retry and backoff behaviour across gateways and clients, fixing the specific VS Code behaviour that amplified Copilot traffic, and improving load-balancer capacity monitoring and regional failover safeguards.

There’s a wider lesson here for anyone depending on a service like this one. Partial degradation is much harder to spot than a clean outage. A service can keep responding while error rates climb, one endpoint can recover while another stays degraded, authentication can fail intermittently while everything else looks healthy. That’s why monitoring a dependency is worth building around the specific behaviour your own systems actually rely on, rather than a single up-or-down signal, whether that’s the response an API is supposed to return, the health of an authentication path, or response times where latency is what actually matters to you.

External monitoring won’t tell you that an Istio sidecar has hit its concurrency limit, or that a retry storm is building inside a vendor’s own infrastructure. That needs the vendor’s own telemetry, and GitHub’s report is a good example of what that looks like once it’s published. What external monitoring can tell you is whether the dependency is behaving the way you expect it to, from outside that infrastructure, while the vendor is still working out why. Both views matter during an incident. They’re just answering different questions.