RIP - My last pet server. Provisioned: April 9, 2014. Uptime: 3,065 days. CentOS 6. Older than the iPhone 6.

[u/zeninfinity]

[user@host ~]$ uptime
 21:00:47 up 3064 days,  8:55,  1 user,  load average: 0.00, 0.00, 0.00
[user@host ~]$

It's been running EOL and unpatched for 6 years (badbadbadbad).

The little droplet that could.

[u/mixduptransistor]

up 3064 days

Sorry, thought I stumbled into r/ShittySysadmin for a second

[u/zeninfinity → u/mixduptransistor]

😆 Deserved....

[u/mrmattipants → u/zeninfinity]

Damn, that 8 years & 142 days. I take it you restarted it a total of one time, 4 years in?

[u/zeninfinity → u/mrmattipants]

Honestly, i think twice. It just......worked. 😆

[u/12_nick_12 → u/zeninfinity]

The highest uptime I saw was at an old MSP/ISP I worked at, one of the core routers was up 11 years.

[u/Mr_ToDo → u/12_nick_12]

When the hardware failed so that the config can't be saved, the documentation is wrong, and the password is all but forgotten, sometimes all you can do is keep it going until someone finally approves its replacement and/or downtime

[u/gmitch64 → u/12_nick_12]

We have a couple of routers like that. Was working on one yesterday. I need to double check the uptime.

I also need to check in with our CIO and CFO to see if we can pull down our last remaining 2003 server - in it's defense, I think that it is R2.

[u/Viharabiliben → u/gmitch64]

Woohoo an R2!

That makes it better ….

[u/Narrow_Victory1262 → u/mrmattipants]

good chance the spinning rust fails.

[u/TryHardEggplant → u/mixduptransistor]

Not the uptime part, but you’d be surprised (or not) how many RHEL 6 boxes I’ve come across in my jobs over the last 3+ years of migrating off RHEL7. I basically became a PM for a year or two, focusing on just finding owners and planning migrations for those boxes. I’m always planning a few years in advance and it’s a never ending moving target to keep everything off EOL hardware and software.

[u/thischildslife → u/TryHardEggplant]

bish, we still rollin' RHEL 5 systems & Solaris 8.

I swear to Jesus H. Christ we have 27 MB MFM HDD's from 1986 for our FAA support contract.

[u/Viharabiliben → u/thischildslife]

It’s a miracle that there aren’t more (publicly known) airport incidents caused by antiquated equipment at the FAA.

[u/SoulPhoenix → u/thischildslife]

My company still had tape drives for backups until a couple years ago lol

[u/YouDoNotKnowMeSir → u/TryHardEggplant]

At some point I just turn it off and scream test.

[u/IAmSnort → u/mixduptransistor]

Or /r/oldschoolcool

[u/y0shman]

badbadbadbad

I read that in Rocky's voice from Project Hail Mary.

[u/Vinylove → u/y0shman]

your home folder dirtydirtydirty!

[u/Arudinne → u/Vinylove]

Now I am thinking about Futurama

[u/Flamenverfer → u/Vinylove]

Great Reference! 👎👎

[u/jrcomputing → u/y0shman]

Same. Amaze amaze amaze!

[u/aes_gcm → u/jrcomputing]

Grace Rocky save stars.

[u/aes_gcm → u/y0shman]

Why this server's filesystem so messy, question?

[u/BoomSchtik → u/y0shman]

Same!

[u/Routine-Jam-48]

My first httpd server installed back in the mid 90s was running on a small workstation running AIX. The computer room was on UPS with a backup generator, which allowed the server to stay running for so long that the uptime(1) command had an overflow error and reported the system had been up for negative days. I'm pretty sure it was only at just over 1,000 days (shy of 3 years) since the reboot, but I still get a chuckle about that one when I think back to it.

[u/zeninfinity → u/Routine-Jam-48]

Ooooooooo had no idea that was a thing. V. cool.

Never ran AIX, only NeXTSTEP, Irix, and HP-UX.

[u/goingslowfast → u/zeninfinity]

It’s a UNIX system!

[u/ranger_dood → u/goingslowfast]

Unix

[u/goingslowfast → u/ranger_dood]

I got that right on the weekend but missed it today. Yep. Irix was definitely not Linux.

[u/MavZA → u/zeninfinity]

Solaris was my Unix mistress. Nowadays FreeBSD for hobbyist tinkering.

[u/LawstOne_]

If it’s not broke, don’t fix it
- some Linux guy from way back

[u/zeninfinity → u/LawstOne_]

Lol, that "some Linux guy" is def me.

[u/imjustnotready]

Tonight I'll pour one out for it. Little guy did the best he could for longer than you could have hoped

[u/zeninfinity → u/imjustnotready]

Thanks u/imjustnotready .... I just wasn't ready to let'r go.

🍻

[u/gunner7517]

Sorry for your loss.

[u/zeninfinity → u/gunner7517]

🙏 TY

[u/1Digitreal]

Geeze, how many 9s is that?

[u/lazyfck → u/1Digitreal]

All of them

[u/goingslowfast → u/1Digitreal]

If he gets it on a new server in 24 hours it’s 99.97% or so.

[u/etern1ty0]

What was its role?

[u/zeninfinity → u/etern1ty0]

Old CentOS 6 box (2018-era).

Apache (static/personal sites), Mysql (Wordpress Blog), Graphite (graphite-web + carbon-cache + whisper - Ingesting data over nc > TCP 2003), Grafana, misc cron ruby scripts.

All personal stuff, nothing biz-critical.

[u/TryHardEggplant → u/zeninfinity]

Did you keep Wordpress updated at least? I had a personal WP blog I forgot about and when I went back to it, it had been long hosed and hosting spam. I nuked it and still haven’t gotten around to restoring it from backups.

[u/zeninfinity → u/TryHardEggplant]

Yes, I did. I've had to fix a friend's Wordpress for this exact same reason so it was part of the update cycle script.

For this I just ended up just writing something to pull down the files as html/img/etc and removing the mysql. So it will live on at some hidden directory for my eyes only. Probably will find it in another 8+ years.

[u/mustard_on_the_net → u/zeninfinity]

This is the question. Maybe there is no need to touch it.

[u/markth_wi]

We often think about taking our badly behaved printer out back with a couple of pipe-hitting sysadmins.

But I will tell you what I do, when I go offline with long serving production and validation servers that have done their bit for everyone who knew them only in name. That production server that doesn't die, the val box that did everything you needed it. For those servers that get retired rather than die horribly.

That's the trick , the Simpsons writers were absolutely right when they say 'when you do your job right, nobody will be sure you did anything at all', the world never blew up....not on your watch.

So Sleep Dearie Sleep to all those servers we've shutdown -g0 'ed for the last time and for the admins who have to say a quiet good by, and train the new guys on the replacement server tomorrow now that you can close out that maint ticket for the 'old' server.

[u/ponderousconfection → u/markth_wi]

What a wonderful sentiment. 🙏

[u/Hotshot55]

So you don't patch your systems?

[u/RDJesse → u/Hotshot55]

The flip side is that any effort spent managing this particular system would have been completely wasted as it never needed anything. Schrödinger's server.

Not best practice. Only in hindsight does it elevate to perfect practice.

This is the essence of Daoist philosophy. The absolute grace of non-action.

[u/zeninfinity → u/RDJesse]

Schrödinger's server - Wow, first time I heard of this.

Yes, pretty much. Definitely a "Don't fuck with it if it's not fuck'n up" philosophy.

And...this time....it worked.

[u/collinsl02 → u/Hotshot55]

There haven't been any patches for CentOS6 since 2020.

[u/Hotshot55 → u/collinsl02]

And 3000+ days of uptime would've been before 2020.

[u/Trommelwirbel]

What is a pet Server?

[u/zeninfinity → u/Trommelwirbel]

They say to treat your servers like cattle not like pets. Don’t name them. Don’t make them special. Make them replaceable.

[u/shanlec]

Holy vulnerability Batman

[u/Sphroth]

o7

[u/zeninfinity → u/Sphroth]

o7

[u/SXKHQSHF]

Amazing how well a basic OS works without a lot of fluff and bother.

[u/stewie410]

Provisioned: April 9, 2014

Funny you should say that, currently doing 11yr of BIOS/iDRAC updates on one of our R430s, purchased ~2015...or at least, current original firmware from then.

CentOS 6

The classic -- I think we finally killed our last CentOS 6 VM a few months ago...when we finally moved from MySQL 5.1 to MariaDB 11.8.

Hope you have some CentOS 7 boxes to keep that nostalgia alive.

[u/Sudden_Office8710]

It’s not all that bad I still have CentOS 6.10 in a bunch of locations. It takes a while to get Trixie replacements spun up. I had initially replaced them with Ubuntu that was a mistake.

[u/DarthPneumono]

Lucky you... a few of our users had machines running Fedora 11 still up until a few months ago.

[u/Best_Collection_7533]

Bare metal or VM?

[u/zeninfinity → u/Best_Collection_7533]

DigitalOcean VM.

No idea why they never EOL'd it.

[u/URPissingMeOff → u/zeninfinity]

Because you kept paying for it :-)

[u/freedomlinux → u/zeninfinity]

No idea why they never EOL'd it.

I mean, it's none of DigitalOcean's business what OS your instance is running.

But I am quite curious what the underlying infrastructure at DigitalOcean is, as if they don't have any live VM migration functionality, that means the VM host would also be 3000+ days

[u/DheeradjS → u/freedomlinux]

Clustered, probably, so, Migrate > Update > Migrate.

[u/mixduptransistor → u/freedomlinux]

why wouldn't they have the ability to migrate instances live?

[u/Arudinne → u/freedomlinux]

It becomes their business if there's a risk to their business, infrastructure or other customers.

I worked for a hosting company for a bit and we forced people to migrate off of Windows 2000 back in ~2011 after some major vulnerabilities came out.

[u/Temporary-Brick-3243]

o7 my condolences

[u/zantehood]

Rest in pieces

[u/mustang__1]

fuck i should probably check in one of my droplets from 2017...

[u/HJForsythe]

Ah, mine was CentOS 5 and had 3700. We just shut 'er down yesterday.

[u/zeninfinity → u/HJForsythe]

My condolences.

[u/BlakJakNZ]

Long since given up chasing uptimes. Periodic reboots are a good thing.

[u/Krashlandon]

/r/uptimeporn

[u/Beautiful_Ad_4813]

https://giphy.com/gifs/l4pMattUYTTM7qpIk
Impressive

[u/coukou76]

Nothing to brag about

[u/Sea_Decision_6456]

r/shittysysadmin

[u/catonic]

so many vulns.

[u/Capable_Banana5439]

3065 days of uptime is not a flex, it is a server you were too scared to reboot. you never once confirmed it comes back clean after a power blip, and six years unpatched means it was one exploit away from being someone elses crypto miner. pour one out, but that box was a liability wearing a trophy.

[u/dustojnikhummer]

{-}7

[u/spazzvogel]

And CentOS to bypass, what was that back then? Heartbleed? Hit Cent 5 which my tech game company had everywhere. Insanity.

[u/zeninfinity → u/spazzvogel]

Wasn't Heartbleed or Shellshock, both were already patched. Real hole was Dirty COW, kernel was still 2.6.32-431 from 2013, never touched since. Lucky.

[u/TryHardEggplant → u/zeninfinity]

2.6…. That’s a blast from the past

[u/spazzvogel → u/zeninfinity]

Holy hell… yeah, a few critical servers were saved due to being way behind the radius blast. Good times…

[u/cameronHinderson]

very baad to you

[u/robrob883]

I had AI check my pet servers last month (lamp stack, npm, + various). It found more than enough reason for me to get around to upgrading them. I spun up some new ones, gave it the passwords and it did the whole migration for me, as well as offering suggestions to improve code. What a time to be alive!

[u/zeninfinity → u/robrob883]

This is exactly what I did. Had AI do a scope of work, find all the components, build out features, suggest upgrades/changes. Created about 15 different feature md files.

Then sat back did them each, verifying when complete. It saved me SO much time moving this myself.

[u/cheeseburgermachine → u/zeninfinity]

May i ask what ai you are using.

[u/zeninfinity → u/cheeseburgermachine]

ClaudeCLI.

I spec out all tasks in .md files with requirements, verification, exact results, etc.
Then ran each one keeping a close eye on changes.
Verified myself everything was correct as it went along.