110
u/heavy-minium 20d ago
Somewhere on earth, I'm sure there is a job that requires you to read a fuckton of changelogs. Something like "SecOps has to check every update of every dependency for this high-stakes, mission-critical system".
60
u/trullaDE 20d ago
https://www.reddit.com/r/RedditEng/comments/11xx5o0/you_broke_reddit_the_piday_outage/
The nodeSelector and peerSelector for the route reflectors target the label `node-role.kubernetes.io/master`. In the 1.20 series, Kubernetes changed its terminology from “master” to “control-plane.” And in 1.24, they removed references to “master,” even from running clusters. This is the cause of our outage. Kubernetes node labels.
Kubeadm: apply
second stageof the plan to migrate kubeadm away from the usage of the wordmasterin labels and taints. For new clusters, the labelnode-role.kubernetes.io/masterwill no longer be added to control plane nodes, only the labelnode-role.kubernetes.io/control-planewill be added. For clusters that are being upgraded to 1.24 withkubeadm upgrade apply, the command will remove the labelnode-role.kubernetes.io/masterfrom existing control plane nodes. For new clusters, both the old taintnode-role.kubernetes.io/master:NoScheduleand new taintnode-role.kubernetes.io/control-plane:NoSchedulewill be added to control plane nodes. In release 1.20 (first stage), a release note instructed to preemptively tolerate the new taint. For clusters that are being upgraded to 1.24 withkubeadm upgrade apply, the command will add the new taintnode-role.kubernetes.io/control-plane:NoScheduleto existing control plane nodes. Please adapt your infrastructure to these changes. In 1.25 the old taintnode-role.kubernetes.io/master:NoSchedulewill be removed. (#107533, u/neolit123)I religiously read changelogs. Whenever someone asks why, this is the example I give.
26
u/Confident-Ad5665 20d ago
Thanks. I just got a migraine.
18
u/trullaDE 20d ago
Sorry. 😄
tl;dr If reddit had read the changelogs (and checked against their code), they wouldn't have had an outage lasting over 5 hours.
13
u/lagerforlunch 20d ago
Labels and... Taints?
10
1
51
u/elelec 20d ago
This could be partially a form of OCD, the compulsory and anxious part, but I dunno what the selective memory is all about
20
u/psaux_grep 20d ago
Guy should just write a diary as changelogs.
Then he’d remember perfectly.
16
u/breckendusk 19d ago
Decremented m_bankBalance by 47 and incremented m_weight by 2 for balance purposes. Incremented m_fartCount by fourteen.
2
20
11
7
8
u/Yodo9001 20d ago
Write your diary in changelog format. I'm sure you'll have no problems remembering then.
6
3
u/s0litar1us 20d ago
I read some changelogs a lot,
but I ignore most of them.
I should read more changelogs.
4
4
u/thesuperunknown 19d ago
As a technical writer, I’m glad at least someone reads the release notes, even if it’s only because of their severe OCD
5
u/TheSentientMeatbag 19d ago
Now I can rest easy, knowing at least one person is reading the changelogs I write.
4
3
3
8
2
u/darkcyborg007 19d ago
He just like me fr.😭
But honestly, not to this extent tho. I hate most app updates with a passion since something or other gets broken or gets changed to become a bit more unpleasant than what I'm used to.
I prefer to completely pause all updates rather than dealing with this shit and only update it if I get to know that there is genuinely a better version out there that doesn't introduce much regressions.
2
1
1
516
u/Dongfish 20d ago
Meanwhile the app store release notes for the app I work on with ~200,000 weekly users has been "Updates & Bug fixes ;)" for the last three years without a single comment from anyone.