Funny because I spent a couple of years on self-hosted Gitea, including doing some work on Gitea itself to add features we wanted. Then I changed projects and thought "well that was a lot of work, let's just do what everyone does and use GitHub". Now twiddling my thumbs this morning...wondering if I need to resurrect the Gitea deployment.
> Update - We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available
It seems vastly higher on every region and VPN I've tested, so I think this is a case of the "technicalies".
It is technically 20%, because a bunch of the requests that happen on page load do succeed. Just not the few crucial ones that are required for the page to load correctly - those return a 500 the vast supermajority of the time.
Feel free to disregard this because ultimately its a tangent that doesn't matter; but supermajority is specifically a political term referring to the required threshold to advance legislation when simple majority isn't procedurally enough. It doesn't mean simply a greater share of.a majority
For supermajority: the construction means “beyond majority” or “above majority”, it is implemented as a higher percentage in most systems, and there is a utility in being able to refer to high ratio majority.
So people have repurposed a technical word to a general word, of roughly the same meaning.
The error rate is 100% on the service that allows you to create PRs in the web site. Actual remote git service is working just fine. I might try the API to make a PR, see if that's working.
Update: yes that worked. gh CLI using the API was able to make a PR while the web interface was inoperable.
Second update: defeat from the jaws of victory -- the page that displays open PRs is also down.
Third update: CI isn't working anyway, so the CLI PR creation workaround doesn't achieve much.
Well I was dumped out of Github Copilot as this issue started, and haven't been able to get back in. Seems like a slightly strange thing to be connected, but feasible.
I find it quite amusing what they seem to consider "degraded performance" to mean -- I guess if you really squint, your service not working at all is actually just very large latency spikes (until the service recovers) and thus can be labelled as "services experiencing degraded performance for a small portion of our customers".
Is github down? Does the day end in y?
No, es lunes.
Damn. Wonder why it's not working then
We're moving to self hosted Gitea. It's never been easier to do devops with the help of LLMs. I cannot believe how flaky GitHub is.
https://git.tinygrad.win/tinygrad/tinygrad
Funny because I spent a couple of years on self-hosted Gitea, including doing some work on Gitea itself to add features we wanted. Then I changed projects and thought "well that was a lot of work, let's just do what everyone does and use GitHub". Now twiddling my thumbs this morning...wondering if I need to resurrect the Gitea deployment.
> Update - We are seeing an approximate 20% error rate across numerous experiences including Pull Requests, Issues, and others. Investigations are currently under way and we will be posting updates as they become available
brutal
20%, do we just make numbers up?
It seems vastly higher on every region and VPN I've tested, so I think this is a case of the "technicalies".
It is technically 20%, because a bunch of the requests that happen on page load do succeed. Just not the few crucial ones that are required for the page to load correctly - those return a 500 the vast supermajority of the time.
I can't get a single PR or issue to work. Maybe it's "20% globally, 100% anywhere people are awake".
Feel free to disregard this because ultimately its a tangent that doesn't matter; but supermajority is specifically a political term referring to the required threshold to advance legislation when simple majority isn't procedurally enough. It doesn't mean simply a greater share of.a majority
This is how language evolves.
For supermajority: the construction means “beyond majority” or “above majority”, it is implemented as a higher percentage in most systems, and there is a utility in being able to refer to high ratio majority.
So people have repurposed a technical word to a general word, of roughly the same meaning.
20% of requests on a GUI page also makes the whole thing unusable.
Depends if this is coming from engineering or management
The error rate is 100% on the service that allows you to create PRs in the web site. Actual remote git service is working just fine. I might try the API to make a PR, see if that's working.
Update: yes that worked. gh CLI using the API was able to make a PR while the web interface was inoperable.
Second update: defeat from the jaws of victory -- the page that displays open PRs is also down.
Third update: CI isn't working anyway, so the CLI PR creation workaround doesn't achieve much.
incident was just created - https://www.githubstatus.com/incidents/zkxwbgr0cnmx
Well I was dumped out of Github Copilot as this issue started, and haven't been able to get back in. Seems like a slightly strange thing to be connected, but feasible.
Yes it is https://downdetector.com/status/github/
“Can’t get latest commit”
Ffs GitHub, get your act together.
Seems to affect web as well. Currently working on a PR and now it says "Merge status cannot be loaded"
https://www.githubstatus.com/
It's all green which means it's working. Ignore the unicorn error on every page.
edit: After 5-10 minutes they got around to opening an incident and (apparently manually?) marking the API as having 'degraded performance'.
I find it quite amusing what they seem to consider "degraded performance" to mean -- I guess if you really squint, your service not working at all is actually just very large latency spikes (until the service recovers) and thus can be labelled as "services experiencing degraded performance for a small portion of our customers".
Even better "degraded performance" doesn't seem to count against their uptime stats. Issues is labeled as "degraded" with 100% uptime.
Not sure if related, but claude seems to be down for me too.
Same here. Just as I was going to push out a release
Clones, repo browsing etc are all working.
/pull/:id/changes is not working, but /pull/:id.diff _is_. It seems parts of the API is also ok? some gh commands I tried worked.
Files changed works on the mobile app somehow. I guess a particular api version or something is down.
Anyway if someone needs files changed and merge status, check the mobile app. Past actions are also loading on that.
[Android, India]
Some more discussion: https://news.ycombinator.com/item?id=49330684
Been doing this wrong for years, thanks
reminder: git is distributed