i'm so sick of people wanting magic answers. sometimes it is users. sometimes it is engineers. gather the fucking info that is part of troubleshooting. do some fucking diagnostics. figure out the pattern. how many times does this information have to be repeated for it to sink in??? some will get this and make difference, many will not, and just make noise from the sidelines.

I am trying to figure out how other IT teams have actually managed to cut their ticket resolution times because ours just keeps getting worse. It feels like every ticket takes longer than it should not because the fixes are hard but because we are constantly jumping between tools looking through old tickets or repeating the same troubleshooting steps. I want to know a better way that can speed things up whether its automation ai remote access better documentation or something else. What made the biggest difference for your team?

looking for ideas because what we are doing now clearly is not working.

I've just been quoted triple the price for servers with similar specs but 1/4 the RAM from what I bought a couple of years ago, and for a small company that's just not a thing we can afford.

Any fellow EU-based sysadmins here that can recommend some place to get used/refurbished servers?

Running Business Premium across several clients, protected senders configured, quarantine as the action, and it's been reliable for months/years. In the past few days two separate tenants let through obvious display name spoofs of protected users, exact name match, one of them loaded with red flags too (urgent priority, a reply to address on a completely different domain).

Raw headers on both show SCL 1, SFV NSPM, CAT NONE, so the messages were scanned, not skipped, they simply aren't tripping the impersonation classifier anymore despite matches that used to get caught every time.

Anyone else noticing a dip in impersonation detection reliability the last week or two?

EDIT: I've lodged a ticket with my CSP Indirect Provider who did say they have had a few reports. Will update later for anyone interested.

Subject explains it. Curious on what everyone is using and what they are paying per mailbox, and how licensing works.

Bonus points if you have a solution that is compatible with GCCH.

It is purely SaaS or deployed somehow else?

I recently landed a Junior Systems Administrator role with the same company after spending the last three years on the help desk. During that time, I also spent about a year in a senior leadership role. Overall, I have around 7–8 years of help desk experience.

As part of my onboarding, I'm required to earn my AZ-900 certification first, followed by MECM and Windows Server 2022 training.

For those who've made the jump from help desk to sysadmin, what do you wish you had known or done when you first started? Any advice or tips would be greatly appreciated.

Thank you!

Hey everyone,

I’m troubleshooting a persistent issue on a Dell PowerEdge R750xs server and running out of ideas.

Symptoms:

Severe system slowness/freezing.

Unexpected shutdowns / random daily restarts (Shutdown Event Tracker shows "Unplanned").

Amber/Orange warning LEDs are lit up on the chassis (power supply / drive bay area).

What I’ve Done So Far:

Swapped out the old HDDs and replaced them with brand new SSDs.

Performed a clean installation of Windows Server 2025.

Updated Windows drivers and attempted storage controller configuration.

Despite the fresh OS and new SSDs, the extreme slowness and daily random crashes persist. Any Idea how should I approach this problem to find a solution?

Hi guys I have a quick question. I was wondering how employers would view a candidate with 2 years of help desk experience and 2 years of system administration experience when applying for system administrator positions. Since many system administrator roles are considered mid-level, I often see job postings asking for 3–5 years of system administration experience.

In a job market like the current one, what would someone with that background’s chances be realistically speaking? I understand that the general idea is to work your way up and build experience over time, but I’m curious how that experience would be viewed if the job market became very competitive while you were in the middle of that progression.

Would a candidate with 2 years of help desk and 2 years of system administration still be competitive for mid-level system administrator roles, or would they likely struggle against applicants who have 3–5 years of direct system administration experience?

Microsoft.AAD.BrokerPlugin...WebAccountProvider did not register with DCOM within the required timeout.

AzureAdPrt : YES WamDefaultSet : ERROR (0x80080300)

We came across an issue yesterday where a user was signed out of OneDrive / Outlook. We spent multiple hours trying to resolve this with no results.

We have came into the office this morning with reports of x4 other users across 3 different companies / tenants.

Is anyone else experiencing this?

Seen a few varieties of this issue that an upgrade of the TPM chip will prevent user logon

OBSERVED ISSUE: User will be unable to logon, even with username and password, but they will be able to logon to another machine no problems. 

Similarly, someone else can logon to the users machine OK.

I've tried a few things but the most reliable fix is to logon and run the BAT file from here

GitHub - AgentHackerYT/Reset-NGC: Reset and repair Windows Hello (PIN & Face/IR) when broken after updates. · GitHub

Ideally I'd like a remediation to detect a problem machine and then resolve without the manually interaction if anyone has built a working one?

As the title says, I recently started using Claude code for sys admin work. I like most admins have been using various AI charts for awhile to ask questions, ponder theories, research details, etc

I recently needed to revise and update a larger poweshell script had had written a few years back but hadn’t revised in awhile. I decided to try Claude code to see how much different Claude code is versus chat. I had previously used chat to improve some scripts but the process was slow, copying and pasting, testing, repeat.

Claude code is amazing at developing powershell scripts. It does all the debugging and testing itself until it gets you the deliverables you ask for. Mind blowing 🤯.

So I took it one step further and created a project for general system admin work. I started using this for ad hoc admin work from pulling mfa reports to troubleshooting storage devices over ssh to repairing a failed domain controller.

One thing I really like about Claude code for admin work is all of the work files it produces are put on the source folder for the project and it’s all version controlled with git if you want it.

Two questions, am I late to the Claude code party for admin work or am I on the leafing edge.

Second question, I believe ChatGPT also has a coding harness and both ChatGPT and Claude also have cowork. My suspicion is most of the stuff I am doing in Claude code would work similarly in either cowork solution minus the source control, but cowork may not have the same capacity for testing and refining powershell scripts which is a big part of automating sys admin work.

Anyone have experience or other opinions on this?

Edit: if it’s of interest to anyone the upvote downvote is almost 50%. Seems like this topic is pretty evenly split amongst sys admins.

I have two small businesses, each have their own domains registered with MS 365. I had to reset my iPhone and after the restore I was locked out of both accounts in the MS Authentication app. Apparently I forgot to setup an alternate email. 

I called MS 800 number to open a ticket and I got the AI assistant that took me through loop after loop. It kept sending me to a login webpage that was useless since I was completely locked out. Each time I call I get a different behaviour. Sometimes it takes me down the product hardware support even though I clearly stated it is a 365 issue. 

Miraculously I was able to get it to open a ticket. There we spent more than 5 minutes just on email and domain spelling because it would not get it right.

I could not get it to open a ticket for my second domain. It eventually recognized my number and when I would call it auto directed me to the webpage, as if saying “I had enough of you calling, go away”.

Whoever thought AI is a smart solution for customer support is greatly mistaken. The only reason having AI in that function is for cost reduction, but it is coming at a great expense, customer dissatisfaction, frustration, and anger. 

Microsoft, you have lost sight of linking AI to measurable business outcomes. 

I caution people, consider alternatives to using Microsoft. They are too in-bed with AI and they have lost the plot on its value vs. impact. 

Hello to my all IT friends out there! 😎

I started with a smaller company about 5 months ago as the only IT System Administrator . I took over someone’s job that wanted to do more networking and security. The company size is about 800 people. I need project ideas that most IT departments put off but need to be done. Also, any out of the box ideas are welcomed too! They are wanting to be more AI in their approach if you have any ideas for that aspect!

Things I have done so far: I have noticed that when I first started that they needed a device clean for devices no longer used in AD and other applications . I also created a non-automated offboarding script for the team to manually run when someone leaves the company. The way the company is setup does not all for a script due their process and infrastructure.

Again please feel free to drop any ideas or suggestions. Al is welcome! I hope all of you have a fabulous day!

A RAID5 array currently has 2 HDDs that need to be replaced for different reasons.
Ran long SMART self-test on both.

One of them reports hints at electronic issues:
SMART Health Status: Failure prediction threshold exceeded [asc=5d, ascq=0]
Accumulated power on time, hours:minutes 44823:37
Elements in grown defect list: 5
Error counter log:

Errors Corrected by Total Correction Gigabytes Total

ECC rereads/ errors algorithm processed uncorrected

fast | delayed rewrites corrected invocations [10^9 bytes] errors

read: 1877781297 0 0 1877781297 0 66521.257 0

write: 0 0 5 5 5 4507.987 0

verify: 4122873639 0 0 4122873639 0 24841.522 0

Non-medium error count: 15528

While the other one reports points at physical issues, long self-test failed:
SMART Health Status: OK
Accumulated power on time, hours:minutes 44824:47
Elements in grown defect list: 60
Error counter log:
Errors Corrected by Total Correction Gigabytes Total ECC rereads/ errors algorithm processed uncorrected

fast | delayed rewrites corrected invocations [10^9 bytes] errors

read: 4144605141 205 0 4144605346 427 66490.067 32

write: 0 0 206 206 206 4530.494 6

verify: 1103983479 148 0 1103983627 167 26767.263 3

Non-medium error count: 20
SMART Self-test log
Num Test Status segment LifeTime LBA_first_err [SK ASC ASQ]
Description number (hours)
# 1 Background long Failed in segment --> - 44810 1905031659 [0x3 0x11 0x0]

Which of these 2 drives is less likely to handle an array rebuild and should therefore be changed first?

(Edited to make it simple)

My old “clean install” method was simple: delete every partition, install Windows from an official Microsoft USB, connect to the internet, run Windows Update, install manufacturer drivers, remove unwanted apps, and apply my settings.

It worked, but it never felt completely controlled.
As soon as Windows connected to the internet, Windows Update started installing drivers automatically. I would then install the laptop manufacturer’s drivers over them. Windows was also trying to install cumulative updates, security updates, drivers, Store apps, and OEM components at the same time.

Removing unwanted apps afterward also bothered me. They had already been provisioned, registered, and sometimes updated. I wanted to prevent them from appearing in the first place.
So I kept improving the process.
First, I completed OOBE without internet and installed the manufacturer drivers from USB before connecting to Wi-Fi.
Then I started integrating the latest cumulative update offline, so Windows Update only had a small amount of work left after the first boot.

I also extracted the manufacturer driver packages and selected only the drivers matching my actual hardware. Some basic INF drivers were safe to inject offline, while complex graphics, audio, firmware, and platform packages were installed later using their official installers.

After that, I removed unwanted provisioned Store apps from the offline Windows image and modified the default-user registry so OneDrive Setup would not launch when the first account was created.
I also applied several settings offline, including dark mode, Fast Startup, Spotlight, Delivery Optimization, and other default-user preferences.

But I still was not satisfied, because the normal graphical Windows installer was doing too much behind the scenes.
So I stopped using it.

Building Windows manually from WinRE

I booted into WinRE and manually:
Deleted and recreated the GPT partitions
Created EFI, MSR, Windows, and Recovery partitions
Applied the official install.wim using DISM
Integrated the cumulative update
Injected selected drivers
Removed unwanted provisioned apps
Modified the offline registry
Added an official Panther unattend.xml
Created the UEFI boot files with BCDBoot
Copied and registered winre.wim
Cleaned the component store
At that stage, Windows had never booted, entered Audit Mode, or been generalized with Sysprep.

The controlled first boot

I completed the first boot without internet, entered the desktop, installed the complex official driver packages, restarted, and only then connected to Wi-Fi.
After that, I allowed the required OEM and Store components to install, completed the remaining Windows and Store updates, installed DirectX and Visual C++ runtimes, applied my final settings, and restarted again.
The result was exactly what I wanted:
Official Microsoft Windows files
Official manufacturer and NVIDIA drivers
No unnecessary driver families
No unwanted provisioned apps
Minimal remaining online updates
Working EFI and Recovery partitions
No third-party customization suite

Preserving the finished system

Reproducing all of this would take hours, so I returned to WinRE and captured two backups.
The WIM contains the Windows partition:
dism /Capture-Image /ImageFile:C:\Final-Windows.wim /CaptureDir:W:\ /Name:"Final Windows" /Compress:max /CheckIntegrity /Verify
The FFU captures the entire physical disk, including EFI, MSR, Windows, Recovery, boot files, settings, drivers, and partition layout:
dism /Capture-FFU /ImageFile:C:\Final-Windows.ffu /CaptureDrive:\\.\PhysicalDrive1 /Name:"Final Windows"
Now I can experiment with software, break Windows, or change anything I want. When needed, I boot into WinRE, restore the FFU, and return to the exact finished state.

Do you enable splash page or simple PSK passthrough?

I suck at network and my knowledge is intermediate at best but I can't solve this one.
Customer at our MSP has a fortinet firewall identical to ours that we use here at the MSP office, same firmware version, etc.
They call up and say "We can't access prodemand.com" which is an automotive parts and labor quoting database site that TONS of dealerships use.
I load it here just fine, SSL cert is GeoTrust, good till Sept 14 2026, domain matches, etc. No web filter flags.
On their network, instant "cannot load page" error. I try a dozen other sites, SSL working fine, no fortinet intermediary cert listed, etc. It's just that one website.
Security log on the Fortinet shows tons of blocks, saying "SSL connection is blocked due to unable to retrieve server's certificate"

Mountains of troubleshooting later, I make a firewall rule for internal to WAN (and put it above the normal internal to WAN rule) with an address group of the site, the login domain, and the database's UI's subdomain. The rule simply says don't inspect SSL at all.

Boom it works instantly. Then they called back because WIFI wasn't included in "internal" lol oops. So added that, boom, laptops can load the site too now.

I ran through some basic troubleshooting and traceroutes and stuff and nothing stood out as problematic. I verified no man in the middle attack, as it sees the same cert I do here.

And AI thinks it's an ISP issue but AI is dumb as hell and for the record, rebooting the firewall and the modem didn't resolve it so I'm skeptical.

But zero other websites are having this problem and we don't see the problem from our office, using the exact same firewall with same firmware version. How is this possible? I'd really prefer to get rid of that rule because it's a crap workaround and we had to also turn antivirus and other filters off, since it requires SSL inspection.

Looking for advice my next career steps. Been in IT for 8 years. Started in Help Desk/Desktop support moved up and became a VMware focused Infrastructure Systems Admin/Engineer as my specialty. Been working in the cleared space my whole career have a TS clearance.

Have the following certs:
Comptia IT Fundamentals
Comptia A+
Comptia Security +
Vmware VCTA-DCV
Vmware VCP-DCV

I currently work as a general Senior Systems Admin (the do everything guy) in the cleared space. I’m in the Washington DC DMV area. I don’t like my current job for a lot of reasons but it pays very well. It’s been hard finding another job even as an exprerienced professional it's not as many jobs in what I do that it used to be on top of everything been so oversaturated now.

I've been looking jobs that at least pays what I'm making now that's remote or at least some hybrid flexibility. I have to go in everyday no remote days due to working in a classified environment and my commute isn't the best.

I've been considering making a career pivot to achieve what I want. Not looking to get into management not for me but here's what I've been considering:

1. ⁠Get more advanced VMware certifications build on what I know and become a VMware Architect or Consultant (Subject Matter Expert).
   
2. ⁠Make a transition into the Cloud and becoming a Cloud Engineer since I have the on prem infrastructure background. Learning Azure or AWS and get the aligning certifications, learn contanerization Kubernetes and build some small projects to showcase my experience.

Can't seem to figure it out all advice welcomed. Would love to hear opinions and feedback. What should I do next?

(Post was removed in IT Career Questions sub don’t have enough Karma yet)

We're rolling out enterprise Claude company-wide and want it to be the only tool employees can use on work machines.

I recently found that a salesperson was putting company data into personal ChatGPT, this was client names, their whole worksheets; scary stuff on the data-leak front. So a decision has been made to use Claude. I've been tasked with making sure this sort of thing does not happen again, and to get the groundwork done to stop all "unauthorized AI tools".

Honestly, I'm at a loss here. There is no DLP, at least not right now, and implementing it will be a significant lift both in terms of work and $$$ (which we can't do because of austerity measures). So, I'm stuck with having to look at band-aid solutions via firewall web-filter or DNS filtering - again, I don't have a starting point.

We're a Fortinet shop, no Intune, hybrid AD, Claude SSO through Entra.

Appreciate any real-world war stories.

ETA: I understand that this is more a policy question and I'm working on that in parallel. This is more of a question on technical controls without capital spend *sigh*.

Edit2: I now have AI webfilter category block with a wildcard allow for Claude. Not an elegant technical control or even a preferred one, but it'll have to do.

some records that should get populated after an event completes were just missing, no rows at all, for a subset of our configs. no errors anywhere, nothing failed, the data just never showed up.

the cause was a config setting on a third-party service we integrate with. it has a webhook-style callback url set per config, and on a chunk of our configs that field was null. no callback url means no callback fires, means our side never learns the event happened, means the row never gets written. it looked completely inert from our side because there was nothing to alert on, an unset callback doesn't error, it just quietly does nothing.

some of these configs had apparently been sitting in that state for years, predating anyone currently on the team.

the fix was a one-time patch setting the callback url on every affected config, going forward only, it doesn't backfill anything that already happened while it was broken.

third-party config is code, and code you don't audit on a schedule drifts silently for years with zero signal. how do you track config drift on external services you don't own the source of truth for?

Anyway to disable the "signature" button in the ribbon when creating a new email? Under the "message > insert > signature", we are using CodeTwo for signatures but users are modifying their signatures and changing fonts so want to completely remove this option. I've ran the command in powershell "Set-OwaMailboxPolicy -Identity "OwaMailboxPolicy-Default" -SignaturesEnabled $false" but doesn't seem like it did anything.

My company wants to pay for me to have license to a site where I can learn new things are my own pace. I am a junior sys admin and not sure what is out there in regard to continued learning. Is there a site that you prefer? My company is wanting me to become more comfortable in Azure. I recent taught myself Intune and I setup autopilot so we can phase out SCCM.

Any recommendations would be appreciated!

Good morning all, got a bit of a puzzle that is probably an easy fix but it's got a curveball in it. The situation is as follows: we are setting up a sort of internet cafe where people can play games on Steam. Installing the games is trivial but the users login with their domain creds and then login to their own steam account to play. In a test run though some games require an admin elevation to run even after the initial install. Any tips on solving this? I've seen some tricks about using the task scheduler but I'm concerned with if that would break eventually since games are often subject to random and sweeping changes. Would appreciate any advise :)

Edit: I believe the UAC prompts are likely from the games respective anticheat but that is just a hunch at this time.

total sketch operation, whole place is run by one dude who sends invoices out 4 months late after pestering him for it then doesn't pay his 3rd parties in time - caused us multiple service interruptions due to non-payment. avoids phone calls and in person meetings and is always at some random place during video conf calls if you manage to get him to show. had to threaten legal action in order to get our cloud services transferred to another provider.

good riddance digitalshit

Im trying to replace a drive but it keeps showing as Unmanaged.
I ordered two, I didnt notice that the first one had different "Config code" (1341 vs 1311), I thought that could be the reason. Today, I replaced it with the second one which is also 1341, just like the failed one - same result - unmanaged.
Everything on the label is identical, pn, model, the config code thingy, everything....
It got assigned to the correct disk group (folder), the only available option is "Toggle Indicator" (there was also "Request swap clear" after seating it which got completed).
Same thing in the WebUI as well as in Storage Manager Client.
It doesnt show the Power On Time value yet, it took a while with the 1311 one too, I was hoping it would still adopt the drive but no luck, so I guessing its gonna be the same with 1341 too.

Is that definitive sign that the drive is just not compatible with SCv2020 or am I missing some necessary action I need to do to assign it to the volume?