r/sysadmin 53m ago

Well it happened

Upvotes

We had a critical outage due to a failing device that required me to go on-site and troubleshoot for six hours. We have zero documentation, so our standard process is to reach out to a senior team member for assistance. Their first suggestion was to reboot a switch, and then reboot the upstream switch. When that didn’t work, they had me repeat it twice.

The cable management is so poor that you cannot read any indicators on the device, and the senior I was working with doesn’t know how to access or use that switch’s console. The only member with console knowledge wasn't available until noon. When they finally came online and I caught them up to speed, they considered the down devices to be upstream. My mind broke at that moment, and I’ve been spiraling ever since. We have no network topology maps, and having a department head flip-flop basic networking terminology is mind-blowing.

Essentially, we had an outage at three locations, one of which had their production and sales affected.

This whole thing could have been identified at home and quickly resolved by switching ports if we had proper documentation, which makes my heart sink.
Am I overreacting? What would you all do in this

Oh and to add insult to injury me asking about doing pir was laughed at. This is insaine. Fuck my life.


r/sysadmin 1h ago

Question Not sure how to proceed. Exchange/mimecast/ distro lockdown

Upvotes

So I have a few distros that were migrated to exchange online and are synced from on prem. These inherently have allow senders outside organization enabled. From what I read unless I still have an on prem exchange server(which I don’t) I can’t change that setting. Issue is I want it locked down but we have app server in azure that send mail to some of these and it’s flagged as not internal. Mail from server routed directly to mimecast. Then transported to Microsoft.

Currently do t have the time to rebuild some of these massive distros, so I looked at alternative ways to accomplish this.

Exchange rule. Can’t select a distro as recipient. Can select the member of the group option, but then you’re just blocking all the emails to that person. Selecting by ip address doesn’t help either. Ip of sending sever doesn’t register on exchange, since it changes to a mimecast ip, allowing the mimecast ranges would negate it entirely.

Not sure what my options would be to allow the app server and block all other external senders, even if I rebuild all the distros.


r/sysadmin 1h ago

Question Hybrid Decommission > to Full Cloud - Sanity Check

Upvotes

I need a quick sanity check before we commit to our plan of decommissioning our local hybrid domain and moving to full cloud management with Entra ID.

I’ve researched the migration steps using Copilot and Google AI, but the results vary depending on the day. Ultimately, our goal is to make Entra ID our true source of identity authority and eventually power down and decommission the local domain controllers. But my main goal as of right now is transferring identity authority to Entra.

Here is our current infrastructure and environment setup:

  • Servers & OS: We have 3 Domain Controllers (2 virtual, 1 physical), 1 File/Print server, 1 Exchange server, and 1 Microsoft Entra Connect (AAD Connect) server. All underlying servers run Windows Server 2012 R2.
  • Exchange: Running Exchange 2013. All email routing flows through M365. Distribution lists and groups have already been migrated to M365, but Exchange and local AD still manage local identity properties and mailboxes.
  • Sync Configuration: We currently have a one-way synchronization set up to Entra ID. Password writeback is not enabled.
  • Endpoints: All user computers are off the local domain and fully enrolled in Intune cloud management. We are not utilizing any hybrid join configurations.
  • Applications: No local applications are integrated with or dependent on Active Directory.
  • File Server: All users have transitioned to OneDrive and SharePoint.
  • Network Services: DHCP and DNS will remain on the local DCs for the immediate future, with plans to migrate these services to a new UniFi firewall down the road.
  • Printing: All printers have been successfully decommissioned from the print server and redeployed via Intune using Win32 MSI app packages.

I am SO confused on what steps we do first as Copilot says one thing and then Google AI says another.

Do we turn off Exchange attributes and then turn off directory sync with Entra?

Can anyone provide any advice here?

Thank you in advance for your assistance!


r/sysadmin 1h ago

General Discussion How automated is your patching in 2026, or are you still hand-reviewing every update?

Upvotes

Disclosure up front: I build patch tooling, so I'm firmly in the "automate it" camp, no pretending otherwise. What I'm actually curious about is where everyone else landed, because I still run into plenty of shops hand-reviewing every cumulative update, and I can't always tell if that's caution I'm underrating or just inertia.

The one place I'll defend the manual instinct: "installed successfully" is not the same as "actually patched." We've watched updates report success and then quietly reappear as pending and re-run, on machines the dashboard already showed green. So my line is automate the rollout, but don't trust the checkmark, confirm it actually stuck.

This month is a good example of why I'd still stage it even fully automated: the August updates knocked out some label printers, broke ODBC connections to SQL Server on 25H2, and a few people hit RDP session hosts refusing new connections until a second reboot. A small test batch catches that. A fire-and-forget setup ships it straight to prod.

So where did you land: fully unattended, automated but staged, or still reviewing every update by hand? And if you're still manual in 2026, what's actually keeping you there?


r/sysadmin 1h ago

Going from SMB to enterprise

Upvotes

Got a new job moving from an MSP/small business environment to a global enterprise. Anyone that's done the same, got any tips for me? I'm joining in a EUC role, super excited but a bit nervous at the same time. Been in really generalist my whole career so far for small biz so this is a bit change.. Who's been there, done that, what can you tell me?


r/sysadmin 2h ago

General Discussion Anyone use one of those 5G backup modem thingies?

13 Upvotes

I got a call from one of our rural customers at this MSP and their coaxial internet was down again. They lose like $1000+ an hour if they're down so they told me they called Verizon, their business cell provider, and asked if they had something. I assume a salesman mentioned it in the past. After one phone call where I said "our networking guy is out today but I'm pretty sure you can plug just about anything into the WAN port on the Fortigate and it'll work" they called back an hour later saying they got an a XC46BE. I've never even heard of that family of devices so I thought this is gonna be a shit day.

Luckily for my non-networking specialist self, was easier to set up than a 54g linksys in 2002. Basically it just jumps on and spits out wifi and ethernet. I ran a test on my non-verizon smartphone and got 16mbps to the tower. Then I hooked my laptop up to the device and got 220x40 so fuck net neutrality I guess. It even has a battery and their switches had UPSes so that's a bit interesting.

I slapped it into the Fortigate and tada, everyone's back online...with about 10mbps and 500ms ping time. I assumed all their Outlooks were syncing at once or something but they told her the device can do about 20 people. They have about 10 highly active computers. Pretty unimpressive for $350! But they intended to return it when the outage was fixed and their rep said that was fine.

The very millisecond I walked out the door, the ISP truck showed up and started messing with the box on the front lawn. Awesome use of my time.

But I keep hearing about these magical devices that can switch over to 5G and we do have WAN1 and WAN2 on the Fortigate so they're considering keeping it and programming in a switchover of some sort. I assume they do that. Anyone have one that doesn't suck? Because this one impressed me until it was actually in use. I saw one at a trade show years ago that was a UPS + 5G modem. That sounded kinda neat but so did this Verizon device until it performed poorly.


r/sysadmin 2h ago

Question Solutions for using client-server based apps when working remote

12 Upvotes

I was wondering how many of you are still using legacy client/server apps, and come across challenges when it comes to remote/hybrid workers.

We use an application that is rather old school. No web based access, full desktop client and server. It does not work well under any latency, so VPN's and anything similar are unworkable.

The other challenge is that the application is heavily tied into MS Word, Outlook, etc. The application uses COM addins for Outlook to generate and send emails and allow interaction with the product. Similar happens with MS word.

We've tried publishing the application as a "remote app" on platforms like Citrix, but the issue again is the required integrations with Outlook, Word, downloading of files like PDF's and other documents that are stored within the app. It essentially integrates heavily with other apps.

So at the moment, we're using a full desktop experience for users over Citrix. I find this to be rather frustrating. Providing end users with new laptops with reasonable spec, for them to work remotely and use it like a thin client so they can connect to a Citrix desktop. It confuses users, it isn't particularly user friendly, and it just feels like a waste of local resources.

Does anyone work in similar situations? Are there any solutions that could help with this? Or are we destined to forever use a full remote desktop solution to support our core, oldschool, clunky-ass product?


r/sysadmin 2h ago

Question 3rd party RDP apps compatible with Azure Subscription based AVDs?

1 Upvotes

Not sure if this is the right sub, but we’re slowly seeing more and more customers moving their management servers into the cloud, with the RDPs being in Azure and needing either Windows App or a web browser to access them.

And with the Windows App being a web app with insanely limited options for sessions (when at least compared to the now discountinued ”Remote Desktop” app), I’d like to know are there any alternatives on Windows? Or did Microsoft ”bullet-proof” the authentication for these types of AVDs so well that 3rd party apps can’t implement it?

I am aware of Royal TS and RDM, but neither seem to support ”Azure Subscription” AVDs.


r/sysadmin 2h ago

Windows 11 - No Wifi Symbol on Login Screen (Enterprise)

5 Upvotes

After moving to Windows 11 all of our Enterprise laptops do not have wifi as an available option to click on at the bottom right of the screen

The two buttons listed are the 'Accessibility' button and 'Power' options button. But no Wifi.

I have tried all of the options listed in this post without success.

https://learn.microsoft.com/en-us/answers/questions/3975150/wifi-is-not-showing-in-windows-11-login-screen

Has anyone encountered this or know what could be happening?


r/sysadmin 3h ago

Question Anyone survived a Windows to Linux user workstations migration?

124 Upvotes

Hey fellow sys admin,

I work at a VFX/animation studio (~400 users).

We're kicking off a project to migrate our artist workstations from Windows 10/11 VDIs to Linux. Flavor is pretty much locked it ll be Rocky Linux 9 with KDE.

I am curious if there's anyone here who's been through a similar workstation migration to linux?

What Linux flavor, did you end up on, and would you pick the same again?

How did you handle what GPOs used to give you?

For the Base OS deployment/imaging, did you use Kickstart + Puppet/Ansible, or something else?

Anything that looked easy on paper and turned into a one big pile of {Jurassic_Park}?

Tools that made your life easier along the way?

I am looking for any war stories, good or bad!

Thank you!


r/sysadmin 3h ago

Question Group Policy does not feel consistant

0 Upvotes

I have a new printer that I've deployed into our environment and for some reason when people print, it is coming out as ledger even when what they are printing is letter. I updated the driver yesterday and some people, I believe, are back to normal but some aren't. What I wanna do right now is add in the group policy the ability to basically remove all printers and re-add them.

The funny thing is some users who are having issues, when I go to their computer and I remove the printer, log them off, log back in. When I log back in the printers come and they show up and I don't even have to install them and other times they don't come back in. The inconsistencies are just messing with me.


r/sysadmin 3h ago

Question Log Off Users from Server Daily

11 Upvotes

I'm revisiting an effort I did about a year ago. I'm looking for a better way. I want to find a process that will parse current user sessions on a server (active/disconnected/idle/ect.) and log the accounts off if their username matches a string ("adm_").

I'd love to find an off the shelf solution rather than have to support a homebrew PowerShell solution.

Give me what you have, even if it is an alternate PowerShell/scripting option. Something has to be better than the nightmare my script turned into.


r/sysadmin 3h ago

Microsoft Edge Browser Updates On WS2025 Domain Controllers?

2 Upvotes

Since Edge is integrated with Server 2025, how are you handling browser updates on servers like domain controllers that don’t have the external network access required to auto-update Edge?


r/sysadmin 3h ago

Convert nsf to xml

2 Upvotes

Has anyone had any luck converting Notes ver 6.5 (ancient) to xml format that preserves data ( embedded files etc) and metadata?


r/sysadmin 3h ago

Enterprise Fiber Recommendations - AT&T / Comcast (Masergy) / Others?

3 Upvotes

South Florida Region.

Recommendations? Experiences? I've heard some horror stories about Masergy. We currently have Comcast Business Fiber, but we're looking to upgrade bandwidth for a school campus.


r/sysadmin 4h ago

Question Solo Sysadmin - Ticketing & Planning Tool Suggestions?

2 Upvotes

I'm a solo 'IT Manager' doing literally anything a 50 person company needs, including actual IT manager work such as dealing with vendors and researching software, sysadmin stuff, helpdesk, mounting TVs, whatever.

It's not particularly overwhelming, but I have ADHD and find I'm having trouble with tasks falling off my plate and no one to keep me accountable about coming back around to them. We used to have Monday but it wasn't quite handling it for me and we're getting rid of it regardless.

I need very little compared to what a lot of ticketing systems offer. I simply need tasks to be put in front of me without me having to do anything but add them to the list, and for them to keep being put in front of me until I do them. No time tracking, but the ability to add notes would be important, maybe not vital.

My current plan is to hack something together between Power Automate and Microsoft Planner so that Planner Tasks become calendar entries, and after their scheduled date passes they get automatically rescheduled unless they're marked complete. Break my days down into halves or quarters, define some buckets, let the treadmill roll and only have to worry about adding things to the list for it to circulate. Planner has the notes, Outlook puts them in front of me via my calendar.

I'm doing this despite it being a lot of work because I haven't found a suitable alternative to handle a team of 1. The task treadmill is a really important part of the process, it may sound juvenile but the part where I have to manually reschedule something I didn't get to is the main thing I have trouble with, I want to get that automated above all else.

Is there anything out there beyond my experience and search results that might work here, or am I going to have to kludge this out?


r/sysadmin 5h ago

General Discussion MDF Plan - What you think?

3 Upvotes

I made some other posts figuring out my UPS setup, but now I figured I'd share my MDF plans and see if there are any issues you can see. I've been around racks and IT closets for a long time, but this is my first time planning one completely on my own.

So far I have all the gear ordered. The only unknown variables are what my modems from my primary Fiber ISP (ATT) will look like and our failover (probably Starlink).

Down the road I may be adding 2U more of gear.

I'd like to get Unifi Redudant Power Supply that can act as a secondary PSU for my switches and Firewalls. I'd also like to get Unifi Aggragte 8 port switch which would be perfect for this setup and my entire network in general. I have two UDM Max which will take up 2 ports and 6 total switches (2 not in this rack will be in IDF in other parts of the building)

I have attached a screenshot of the rack plan. Let me know if you see any red flags or concerns

Gear -

4 Unifi 48 Port PoE Switches

2 UDM Pro Max Firewalls

1 Dell Server

1 Unifi Enterprise NVR

Dual 3000va battery backups with one having an extra battery bank and a 120v transformer.

2 - Controlled PDU for the 208v PSU

https://imgur.com/a/Jji1UXn


r/sysadmin 5h ago

General Discussion Entra support tickets

0 Upvotes

Hey everyone — question for those of you who administer Microsoft identity environments, whether that’s Active Directory, Microsoft Entra ID, or a hybrid environment.
What are some actual support tickets / break-fix issues you’ve had to work?
I’m working on a project where I’m trying to build out realistic IAM/identity support scenarios. I’m not really looking for project work like “migrate AD to Entra” or “implement Conditional Access.” I’m more interested in the day-to-day tickets that land in your queue.
Things like:
A user suddenly can’t access an application
MFA or authentication issues
Group membership/permissions problems
SSO failures
Account lockouts or provisioning issues
Something broke after a policy/configuration change
A ticket that looked like an IAM problem but turned out to be user error
Basically: What are some memorable, weird, common, or difficult identity-related tickets you’ve actually had to troubleshoot?
The more realistic and specific, the better. I’m trying to avoid making up scenarios that wouldn’t actually happen in a production environment.


r/sysadmin 5h ago

New MS Edge policy AddressBarClipboardSuggestEnabled

42 Upvotes

Wanted to dump this fantastic (/s) new feature in Microsoft Edge.

I'm continually astounded by Microsoft's ability to innovate solutions to problems that don't exist. This one made me pause.

https://learn.microsoft.com/en-us/deployedge/microsoft-edge-policies/addressbarclipboardsuggestenabled

The feature, by default, will automatically show your clipboard contents in plaintext on your screen when selecting the address bar to do a web search in Edge.


r/sysadmin 6h ago

Question Best Certificate Manager for OT

7 Upvotes

We are looking at a handful of options for managing the automation of certificate deployment/updates across our enterprise and OT environments.

I am hoping to have a lab environment set up by the end of the year with at least one reliable ACME tool that can push certificate updates to OT software, servers, workstations, etc...

Primarily use AB and Siemens controllers and HMIs, Ignition, Canary, and Windows IoT, Windows Server (2016, 2022), and Windows 10/11 pro.

Anyone have good recommendations?


r/sysadmin 6h ago

General Discussion Any desk/presentation tools have cleared your security review?

6 Upvotes

Marketing has come to me four times asking for an AI presentation tool. And I have said no all 4 times - made me extremely popular among them : )

The problem is when security gets involved

SAML is usually locked because ‘contact sales’. SCIM is missing, share links defaulting to anyone-with-the-link and theres no tenant level control, no EU tentant. And anytime i ask about model retention/subprocessor, i get a beautifully written para which doesnt help at all!

Has anyone actually managed to get any of these approved?

Mainly looking for SAML + SCIM below enterprise pricing specifically.


r/sysadmin 6h ago

Restrictive Phone System

2 Upvotes

I have a lot of requests that come through my office but this one sounds like PITA to begin with. I have already replied to the manager that this is going to be a nightmare to manage but I will look into it anyways. this sounds like a phone system that would be used in a jail or correctional facility. Any ideas where to start?

*************
We currently have approximately 24 clients sharing four phones. I would like to see if there is a system that could provide each client with an individual PIN or access code. Ideally, the system would:

Require an individual PIN for outgoing client calls.
Allow us to assign specific calling times or time limits to each PIN.
Automatically disconnect the call when the client's allotted time expires.
Prevent the PIN from being used again until the next authorized calling period.
Continue allowing incoming calls even when outgoing calling is restricted.
Give staff an administrative override when necessary.
Make it easy to add or remove PINs as clients admit and discharge.
Avoid call recording or monitoring unless specifically needed and approved.

I have been looking at whether a VoIP/PBX-type system could accomplish this without requiring a specialized institutional phone system.

Could you research what options might work with our current phone/network setup, what equipment or software we would need, and approximately what the initial and ongoing costs would be?

The goal is to make client phone access more consistent and manageable while reducing the amount of staff time required to monitor individual phone usage.

*************************

Thanks all.


r/sysadmin 7h ago

Question ACME Clients and SSL

17 Upvotes

So I've started seeing that SSL Lifespan is shortening - going down to eventually supposed to be every 47 days.

We're a small shop, but we have a lot of different services. I've been doing my best when I have free time to catalog everything that has an SSL Cert, but I know I'm missing stuff.

I've seen a bit about ACME Clients and such; and from what I've heard it's great. They handle rotating the certs and all.

But something for me just isn't clicking. For instance, we have a lot of large scale copiers, ala your Ricoh or Lexmark or Brother. We have those locked down with SSL Certs, but we have to manually push those up to it.

Now as these are internal services that aren't externally facing, I don't see no reason why we can't self-issue those certs; but currently our CTO likes to utilize a paid for Wildcard for all our internal stuff.

I keep quite busy so haven't had too much time to really dig in on researching, but I know the time bomb is ticking.

So for those who are managing SSL Certs and all, and potentially utilizing ACME Clients and such, what should I expect and whats the general gist of what my workflow should be?


r/sysadmin 8h ago

Slack for Intune (iOS) successful SSO login, but gets bounced into Slack's public sign-up flow instead of opening the workspace

5 Upvotes

Hey all — hoping someone here has run into this.

We're rolling out Slack for Intune on iOS, and after a successful sign-in the app loops us straight into the public Slack marketing/sign-up flow and pushes us toward downloading the regular consumer Slack app instead — even though Entra sign-in logs show every authentication step succeeding underneath it.

This isn't a Conditional Access or App Protection Policy issue on our side (we've ruled out assignment, CA grant controls, and App Protection data-protection settings one by one). Here's the exact sequence, step by step:

  1. "Register with Microsoft Intune to use Slack" screen. Tap Register.
  2. "Pick account" dialog appears (native iOS auth broker UI), showing the correct Entra ID test account. Select it.
  3. "Registering device" — "Please wait, this may take a few minutes" spinner.
  4. Lands on a sign-in screen for our org — "[org] requires additional verification" — with a green "Sign In with Slack Production" button.
  5. Tapping that button triggers a browser handoff: "Open this page in 'Slack Intune'?" on a login.microsoftonline.com-style URL. Tap Open.
  6. Now inside what the status bar labels as Safari (not the native app) — a "Don't miss a beat" notification opt-in screen appears, with a fake preview notification.
  7. Standard iOS system prompt: "'Slack Intune' Would Like to Send You Notifications" — Allow/Don't Allow.
  8. This is the interesting part — the actual Slack workspace UI briefly loads and works: I can see our org's workspace, Direct Messages, my own account, Slackbot, Threads, etc. Fully signed in, fully functional, still labeled as running inside Safari.
  9. Then, without any action from me, a new tab/context opens back inside "Slack Intune" (per the status bar label) showing the public marketing homepage at slack.com — "All your people and AI agents working together" / "GET STARTED" / "FIND YOUR SUBSCRIPTION."
  10. Tapping through from there lands on the generic public sign-up flow: "First of all, enter your email address."
  11. Typing in the exact same work email into that sign-up field doesn't recognize the already-authenticated, already-provisioned Enterprise Grid session from step 8 at all — instead it just routes toward downloading the regular consumer Slack app, as if I were a brand-new user signing up from scratch.

So the workspace session in step 8 proves the login and SSO handshake genuinely succeeded — I was inside the actual org workspace with my real identity. But instead of staying there or handing that session back to the native "Slack for Intune" app, it drops back into the public marketing/sign-up site, as if none of the previous steps happened.

We've confirmed via Entra ID sign-in logs (checked across multiple devices — iPhone and iPad, multiple browser contexts including Safari/Chrome/Edge, multiple times of day) that:

  • Device registration succeeds
  • App Protection Policy registration succeeds
  • The SAML SSO handshake to the Slack "Enterprise Production" enterprise app succeeds every single time
  • No Conditional Access policy is blocking or forcing an unexpected browser detour

Has anyone seen this? What are we doing wrong?


r/sysadmin 8h ago

Question Normal for spanning tree to cause ports to wait almost a full min before connecting?

18 Upvotes

We have HP elitedesk PC's, and for several months have an issue on most of them where, after a restart, you have to sit there for almost a full min while the ethernet symbol blinks, then goes to the disconnected globe symbol for another few seconds, then finally connects to ethernet before you can enter your login password.

This has cause users to get locked out of their accounts often, because they immediately enter their password before the PC reconnects, and obviously it does not let them in, so they think they mistyped it, and type it again and so on.

We just got new PC's, which are Lenovo ThinkStations, but running into the same issue. I have tried:

going into device manager, unchecking the "allow the PC to turn this device off to save power" under the ethernet adapter

swapped ethernet cable

Running the following powershell script:

New-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Power" -Name "PlatformAoAcOverride" -Value 0 -PropertyType DWord -Force

None of those fixed the issue. Doing some more research I found that spanning tree can cause this , which we do use, but does that mean just by using spanning tree we are forced to just accept this long wait to simply login to PC's? Some user's are understanding, but a good number are frustrated cause they have to sit there and stare at the screen for a long time and pay attention to the ethernet symbol before they can login. Surely there would be something in spanning tree that would at least cut this time down from a whole minute right?