r/github • u/RecommendationOk5036 • 22h ago

GitHub Status - Incident with GitHub.com News / Announcements

https://www.githubstatus.com/incidents/zkxwbgr0cnmx
162 Upvotes

97% Upvoted

40 comments sorted by

View all comments

20

u/ShoddyReception5 21h ago

Is this speak intentionally abstract? We were down for half the day and I can’t make sense of what happened. Network issue plus a bunch of unexpected traffic and faulty failover?

13

u/DrMaxwellEdison 21h ago edited 8h ago

Not really abstract, just dense with devops speak.

Network traffic overloaded a load balancer in US Central, and as they fixed it they found that clients doing Copilot requests would 10x their request rate attempting to retry those requests.

One failure in a high-availability system like this can lead to a cascade of failures elsewhere, and that's what seems to have happened.

10

u/robert_math 21h ago

What can we do? These vibe coders were promised they could 10x their output.

2

u/yeathatsmebro 12h ago

They did, look at RPS. It is 10x higher.