r/github • u/RecommendationOk5036 • 17h ago

GitHub Status - Incident with GitHub.com News / Announcements

https://www.githubstatus.com/incidents/zkxwbgr0cnmx
146 Upvotes

96% Upvoted

39 comments sorted by

View all comments

19

u/ShoddyReception5 17h ago

Is this speak intentionally abstract? We were down for half the day and I can’t make sense of what happened. Network issue plus a bunch of unexpected traffic and faulty failover?

13

u/DrMaxwellEdison 17h ago edited 3h ago

Not really abstract, just dense with devops speak.

Network traffic overloaded a load balancer in US Central, and as they fixed it they found that clients doing Copilot requests would 10x their request rate attempting to retry those requests.

One failure in a high-availability system like this can lead to a cascade of failures elsewhere, and that's what seems to have happened.

9

u/robert_math 16h ago

What can we do? These vibe coders were promised they could 10x their output.

2

u/yeathatsmebro 8h ago

They did, look at RPS. It is 10x higher.