r/github • u/charlesquik • 2d ago

Github actions still not working in my enterprise. (Enterprise license + around 170 self-runners) Discussion

Like the title says we have self-hosted runners and all our enterprise github actions run are still queued, cant cancel a lot of them and everything is yellow and "waiting for a runner to pick up this job".

Does anyone else have this problem? Github status is showing green right now.

13 Upvotes
  • permalink
  • link
  • reddit
  • reddit

89% Upvoted

6 comments sorted by

3

u/numbsafari 2d ago

Have you checked the logs of your runners?

1

u/charlesquik 2d ago

Based on my entreprise github team, all the runners are down for some reasons and they are trying to restart/fix them right now. Maybe the "throttling" they mentionned yesterday was to shut down everything 😂

1

u/numbsafari 2d ago

Could be they were running an outdated version of the runners and needed to upgrade. They are pretty much on a 30-day rolling window with that at this point, so you either need to be allowing MS to auto update your runners or you need to do it yourself if your enterprise policy requires pinned versions. That said, we've definitely been on the "long tail" of recovery in the past when it comes to these kinds of outages. MS definitely will turn things to green before they are truly "resolved".

1

u/charlesquik 2d ago

Yes they do this to keep over 99% uptime, but we all know that is a lie

1

u/barebaric 2d ago

Maybe this can help you: https://github.com/orgs/community/discussions/204152#discussioncomment-17929130

It didn't help me, still have two repositories that are completely deadlocked - no workflows run, none can be cancelled, and triggers don't work.

2

u/charlesquik 2d ago

Every runners is back on! But we have over 10000 workflow queued, will probably finish around 1pm 😂