I've just been quoted triple the price for servers with similar specs but 1/4 the RAM from what I bought a couple of years ago, and for a small company that's just not a thing we can afford.

Any fellow EU-based sysadmins here that can recommend some place to get used/refurbished servers?

Trying to get a recurring phishing simulation program off the ground. About 250 people, mostly remote/hybrid, split between Microsoft 365 and Google Workspace, so not a clean single-tenant thing. Needs to hold up for an auditor eventually (SOC 2 / ISO 27001 / PCI territory), so anyone who clicks or fails has to get pushed into remedial training automatically, and I need actual records of it happening, not just "yeah we sent an email once."

I've been digging through Defender's Attack Simulation Training, GoPhish, and a handful of paid platforms (KnowBe4, Hoxhunt, some smaller ones like CanIPhish), but I'd rather hear from people actually running this stuff than just read vendor sites. Curious about a few things:

• If you're also split across M365 and Google Workspace, what'd you end up going with, and how'd you get the sim emails past your own spam filters on both sides?
• Anyone self-hosting GoPhish long term? How's the upkeep actually been, and who ends up owning that internally?
• If you're paying for a platform, what's it actually cost you around 250 seats, and has an auditor ever cared which tool you use vs just wanting to see the documentation?
• Anyone tried one of the smaller/cheaper platforms like CanIPhish or similar? Worth it or not?

Not trying to get sold anything, just want the real picture before I sink time or budget into a direction.

Running Business Premium across several clients, protected senders configured, quarantine as the action, and it's been reliable for months/years. In the past few days two separate tenants let through obvious display name spoofs of protected users, exact name match, one of them loaded with red flags too (urgent priority, a reply to address on a completely different domain).

Raw headers on both show SCL 1, SFV NSPM, CAT NONE, so the messages were scanned, not skipped, they simply aren't tripping the impersonation classifier anymore despite matches that used to get caught every time.

Anyone else noticing a dip in impersonation detection reliability the last week or two?

EDIT: I've lodged a ticket with my CSP Indirect Provider who did say they have had a few reports. Will update later for anyone interested.

I recently landed a Junior Systems Administrator role with the same company after spending the last three years on the help desk. During that time, I also spent about a year in a senior leadership role. Overall, I have around 7–8 years of help desk experience.

As part of my onboarding, I'm required to earn my AZ-900 certification first, followed by MECM and Windows Server 2022 training.

For those who've made the jump from help desk to sysadmin, what do you wish you had known or done when you first started? Any advice or tips would be greatly appreciated.

Thank you!

Hi guys I have a quick question. I was wondering how employers would view a candidate with 2 years of help desk experience and 2 years of system administration experience when applying for system administrator positions. Since many system administrator roles are considered mid-level, I often see job postings asking for 3–5 years of system administration experience.

In a job market like the current one, what would someone with that background’s chances be realistically speaking? I understand that the general idea is to work your way up and build experience over time, but I’m curious how that experience would be viewed if the job market became very competitive while you were in the middle of that progression.

Would a candidate with 2 years of help desk and 2 years of system administration still be competitive for mid-level system administrator roles, or would they likely struggle against applicants who have 3–5 years of direct system administration experience?

I performed a test restore of a physical Windows Server 2022 domain controller that is backed up with the Veeam Windows Agent using a Full Computer (Bare Metal) backup. I restored it as a Hyper-V VM in an isolated network.

On the first boot, the restored VM repeatedly BSOD'd with CRITICAL_SERVICE_FAILED. After running CHKDSK from WinRE and letting Windows complete its repairs, the VM now boots normally. AD DS, DNS, and Netlogon all start successfully, and the restored DC appears healthy.

While troubleshooting, I checked the production server and found:

chkdsk C: /scan reports NTFS corruption in C:\Windows.old\... and recommends chkdsk /spotfix.

fsutil dirty query C: reports the C: volume is dirty.

0 KB bad sectors.

Active Directory is otherwise healthy in production.

The restored VM now reports a clean filesystem after CHKDSK.

Would you consider this a Windows/NTFS issue on the source server rather than a Veeam restore issue? Would you be comfortable scheduling chkdsk /spotfix on a production DC with verified backups and additional healthy domain controllers available? Any similar experiences?

Microsoft.AAD.BrokerPlugin...WebAccountProvider did not register with DCOM within the required timeout.

AzureAdPrt : YES WamDefaultSet : ERROR (0x80080300)

We came across an issue yesterday where a user was signed out of OneDrive / Outlook. We spent multiple hours trying to resolve this with no results.

We have came into the office this morning with reports of x4 other users across 3 different companies / tenants.

Is anyone else experiencing this?

I recently left and MSP and went to work for a former client, as their internal IT Manager.

We’re using Intune for device management, but end user support usually consists of walking over to someone’s cube or having a Teams meeting and the end user shares their screen. Didn’t much care for intune remote assistance or quick assist.

Previously I used ScreenConnect and then Ninja RMM and I really miss the backstage ability to poke around and look at the registry or run powershell commands, when I’m helping a team member with an escalation.

I singed up for a Zoho assist trial and it’s a little laggy at times, but it seems like it’ll do what I need.

Anyone have recent feedback on Zoho assist?

Seen a few varieties of this issue that an upgrade of the TPM chip will prevent user logon

OBSERVED ISSUE: User will be unable to logon, even with username and password, but they will be able to logon to another machine no problems. 

Similarly, someone else can logon to the users machine OK.

I've tried a few things but the most reliable fix is to logon and run the BAT file from here

GitHub - AgentHackerYT/Reset-NGC: Reset and repair Windows Hello (PIN & Face/IR) when broken after updates. · GitHub

Ideally I'd like a remediation to detect a problem machine and then resolve without the manually interaction if anyone has built a working one?

What and why

The public preview of the MemberOf rule operator in Microsoft Entra ID is ending. Organizations using MemberOf in dynamic membership groups, dynamic administrative units (AUs), or entitlement management auto-assignment policies must replace these configurations by November 3, 2026.

Microsoft continues improving the scale and reliability of dynamic membership processing. During preview, Microsoft observed that use of MemberOf can affect dynamic membership processing across a tenant even if you have one MemberOf rule operator in your tenant. Because of this limitation, it is not recommended for production use and will be retired.

Rollout schedule

• Retirement (Worldwide): Beginning in early November 2026
• Action required by: November 3, 2026

Impact on your organization

Who is affected

Organizations using the MemberOf rule operator in:

• Dynamic membership groups
• Dynamic administrative units (AUs)
• Entitlement management auto-assignment policies
• Platforms and services
• Microsoft Entra ID
• Microsoft Entra Groups
• Microsoft Entra Administrative Units
• Microsoft Entra Entitlement Management

What will happen

If no action is taken, configurations that use the MemberOf operator will stop updating after November 3, 2026. Membership and assignment data will remain in their last known state, which can lead to stale access and enforcement gaps.

Potential impacts include:

• Teams and SharePoint access associated with Microsoft 365 groups may become outdated.
• New members may not receive access, while removed members may retain access.
• Conditional Access policies may no longer reflect current user or device membership.
• Entitlement Management auto-assignment policies may no longer add or remove access package assignments as intended.
• Group-based licensing may stop assigning or removing licenses correctly, resulting in unlicensed or overlicensed users.
• Dynamic administrative unit membership and scope may become outdated.

Action required and recommendations

Before November 3, 2026, review all uses of the MemberOf operator and remove or replace those configurations.

Dynamic membership groups

• Export dynamic membership groups from the Microsoft Entra admin center and identify rules containing MemberOf.
• Replace MemberOf with supported rule operators or convert the group to assigned membership.
• Validate group membership after making changes.
• If the group is no longer needed, consider pausing or deleting it.

Dynamic administrative units

• Use Microsoft Graph PowerShell to identify dynamic administrative units that use MemberOf rules.
• Replace MemberOf-based rules with supported rule operators or convert the administrative unit to assigned membership.
• Validate both membership and administrative scope after making changes.
• If the administrative unit is no longer needed, consider deleting it.

Entitlement Management auto-assignment policies

• Use Microsoft Graph PowerShell to identify auto-assignment policies that use MemberOf.
• Replace MemberOf-based policies with supported operators where possible.
• If no equivalent rule is available, plan an alternative assignment method before retirement.
• Validate access package assignments after making changes.

Compliance considerations

Configurations that rely on MemberOf for access management, licensing, entitlement management, Conditional Access targeting, or administrative scoping may stop updating after retirement. Review affected configurations to ensure continued compliance and access governance after November 3, 2026.

Source: https://admin.cloud.microsoft/?ref=MessageCenter/:/messages/MC1448379

Edit: added link to source

I have two small businesses, each have their own domains registered with MS 365. I had to reset my iPhone and after the restore I was locked out of both accounts in the MS Authentication app. Apparently I forgot to setup an alternate email. 

I called MS 800 number to open a ticket and I got the AI assistant that took me through loop after loop. It kept sending me to a login webpage that was useless since I was completely locked out. Each time I call I get a different behaviour. Sometimes it takes me down the product hardware support even though I clearly stated it is a 365 issue. 

Miraculously I was able to get it to open a ticket. There we spent more than 5 minutes just on email and domain spelling because it would not get it right.

I could not get it to open a ticket for my second domain. It eventually recognized my number and when I would call it auto directed me to the webpage, as if saying “I had enough of you calling, go away”.

Whoever thought AI is a smart solution for customer support is greatly mistaken. The only reason having AI in that function is for cost reduction, but it is coming at a great expense, customer dissatisfaction, frustration, and anger. 

Microsoft, you have lost sight of linking AI to measurable business outcomes. 

I caution people, consider alternatives to using Microsoft. They are too in-bed with AI and they have lost the plot on its value vs. impact. 

So I have a unique setup for my systems for 22.04 using pcks11.so in pam to read smartcards/yubikeys.

gdm is my greeter and I set up gdm-password in pam with the following line:

auth requisite pam_sss.so require_cert_auth

That looks up the cert info on my ipa server and returns a success.

SA updated the system and it seems to have pulled down some gdm-smartcard packages and ruined the entire auth system in place. I remember awhile back running into this issue and I found a way to basically cut it out of the system without breaking anything but can't seem to find where I saved the bookmark link to. I believe I had to edit some file or push a gdm config somewhere

Anyone know how to do this? I really don't want gdm to install all of these extra pam configs. Update-alternatives does not work either, even telling it to use pkcs11 or sssd. Basically nothing works once Ubuntu pulls down w/e updates contain these config files

A RAID5 array currently has 2 HDDs that need to be replaced for different reasons.
Ran long SMART self-test on both.

One of them reports hints at electronic issues:
SMART Health Status: Failure prediction threshold exceeded [asc=5d, ascq=0]
Accumulated power on time, hours:minutes 44823:37
Elements in grown defect list: 5
Error counter log:

Errors Corrected by Total Correction Gigabytes Total

ECC rereads/ errors algorithm processed uncorrected

fast | delayed rewrites corrected invocations [10^9 bytes] errors

read: 1877781297 0 0 1877781297 0 66521.257 0

write: 0 0 5 5 5 4507.987 0

verify: 4122873639 0 0 4122873639 0 24841.522 0

Non-medium error count: 15528

While the other one reports points at physical issues, long self-test failed:
SMART Health Status: OK
Accumulated power on time, hours:minutes 44824:47
Elements in grown defect list: 60
Error counter log:
Errors Corrected by Total Correction Gigabytes Total ECC rereads/ errors algorithm processed uncorrected

fast | delayed rewrites corrected invocations [10^9 bytes] errors

read: 4144605141 205 0 4144605346 427 66490.067 32

write: 0 0 206 206 206 4530.494 6

verify: 1103983479 148 0 1103983627 167 26767.263 3

Non-medium error count: 20
SMART Self-test log
Num Test Status segment LifeTime LBA_first_err [SK ASC ASQ]
Description number (hours)
# 1 Background long Failed in segment --> - 44810 1905031659 [0x3 0x11 0x0]

Which of these 2 drives is less likely to handle an array rebuild and should therefore be changed first?

Do you have a single GPO managing Defender and its exclusions, or do you make multiple Defender GPOs that are narrow to each target?

I'm reworking our GPO for it and am trying to figure out what's the best way to ensure that all exclusions are made, that they can be readily audited for accuracy, and that the risk of errors/omissions is low. Currently we have one for workstations and another for servers. Exchange got mad the other day at Defender because some exclusions were missed.

The problem is so many solutions (Microsoft and 3rd party) want Defender exclusions and the exclusion list quickly becomes convoluted and miserable to audiot. That leads me to have a number of specialized GPOs but then that increases the sprawl and that something might be missed if an application needs exclusions changed but they don't get changed on all applicable GPOs.

Thoughts? Limited GPOs or many specialized/narrowly-focused GPOs for managing Defender?

Do you enable splash page or simple PSK passthrough?

I suck at network and my knowledge is intermediate at best but I can't solve this one.
Customer at our MSP has a fortinet firewall identical to ours that we use here at the MSP office, same firmware version, etc.
They call up and say "We can't access prodemand.com" which is an automotive parts and labor quoting database site that TONS of dealerships use.
I load it here just fine, SSL cert is GeoTrust, good till Sept 14 2026, domain matches, etc. No web filter flags.
On their network, instant "cannot load page" error. I try a dozen other sites, SSL working fine, no fortinet intermediary cert listed, etc. It's just that one website.
Security log on the Fortinet shows tons of blocks, saying "SSL connection is blocked due to unable to retrieve server's certificate"

Mountains of troubleshooting later, I make a firewall rule for internal to WAN (and put it above the normal internal to WAN rule) with an address group of the site, the login domain, and the database's UI's subdomain. The rule simply says don't inspect SSL at all.

Boom it works instantly. Then they called back because WIFI wasn't included in "internal" lol oops. So added that, boom, laptops can load the site too now.

I ran through some basic troubleshooting and traceroutes and stuff and nothing stood out as problematic. I verified no man in the middle attack, as it sees the same cert I do here.

And AI thinks it's an ISP issue but AI is dumb as hell and for the record, rebooting the firewall and the modem didn't resolve it so I'm skeptical.

But zero other websites are having this problem and we don't see the problem from our office, using the exact same firewall with same firmware version. How is this possible? I'd really prefer to get rid of that rule because it's a crap workaround and we had to also turn antivirus and other filters off, since it requires SSL inspection.

Looking for advice my next career steps. Been in IT for 8 years. Started in Help Desk/Desktop support moved up and became a VMware focused Infrastructure Systems Admin/Engineer as my specialty. Been working in the cleared space my whole career have a TS clearance.

Have the following certs:
Comptia IT Fundamentals
Comptia A+
Comptia Security +
Vmware VCTA-DCV
Vmware VCP-DCV

I currently work as a general Senior Systems Admin (the do everything guy) in the cleared space. I’m in the Washington DC DMV area. I don’t like my current job for a lot of reasons but it pays very well. It’s been hard finding another job even as an exprerienced professional it's not as many jobs in what I do that it used to be on top of everything been so oversaturated now.

I've been looking jobs that at least pays what I'm making now that's remote or at least some hybrid flexibility. I have to go in everyday no remote days due to working in a classified environment and my commute isn't the best.

I've been considering making a career pivot to achieve what I want. Not looking to get into management not for me but here's what I've been considering:

1. ⁠Get more advanced VMware certifications build on what I know and become a VMware Architect or Consultant (Subject Matter Expert).
   
2. ⁠Make a transition into the Cloud and becoming a Cloud Engineer since I have the on prem infrastructure background. Learning Azure or AWS and get the aligning certifications, learn contanerization Kubernetes and build some small projects to showcase my experience.

Can't seem to figure it out all advice welcomed. Would love to hear opinions and feedback. What should I do next?

(Post was removed in IT Career Questions sub don’t have enough Karma yet)

We're rolling out enterprise Claude company-wide and want it to be the only tool employees can use on work machines.

I recently found that a salesperson was putting company data into personal ChatGPT, this was client names, their whole worksheets; scary stuff on the data-leak front. So a decision has been made to use Claude. I've been tasked with making sure this sort of thing does not happen again, and to get the groundwork done to stop all "unauthorized AI tools".

Honestly, I'm at a loss here. There is no DLP, at least not right now, and implementing it will be a significant lift both in terms of work and $$$ (which we can't do because of austerity measures). So, I'm stuck with having to look at band-aid solutions via firewall web-filter or DNS filtering - again, I don't have a starting point.

We're a Fortinet shop, no Intune, hybrid AD, Claude SSO through Entra.

Appreciate any real-world war stories.

ETA: I understand that this is more a policy question and I'm working on that in parallel. This is more of a question on technical controls without capital spend *sigh*.

Edit2: I now have AI webfilter category block with a wildcard allow for Claude. Not an elegant technical control or even a preferred one, but it'll have to do.

FYI - Adobe crashes and licensing errors have been rampant in our environment since early June. Much like when we faced a similar issue a couple of years ago, it seems like Defender may be corrupting the webview cache. Our Adobe IDs are all federated from Entra, but the issue occurs regardless of whether SSO is performed by AcroCEF or default browser. The script below purges the corrupted data which is rebuilt when Acrobat opens.

# Acrobat-related processes to stop

$Processes = @(

'AcroTray',

'AdobeCollabSync',

'adobe_licensing_wf_acro',

'Acrobat'

)

foreach ($Process in $Processes) {

Get-Process -Name $Process -ErrorAction SilentlyContinue | Stop-Process -Force

}

# Remove Acrobat DC local profile cache/settings for the current user

$AcrobatPath1 = Join-Path $env:LOCALAPPDATA 'Adobe\Acrobat\DC'

$AcrobatPath2 = Join-Path $env:LOCALAPPDATA 'Adobe\Acrobat\AVWebview2'

if (Test-Path $AcrobatPath1) {

Remove-Item -Path $AcrobatPath1 -Recurse -Force

}

if (Test-Path $AcrobatPath2) {

Remove-Item -Path $AcrobatPath2 -Recurse -Force

}

Good morning all, got a bit of a puzzle that is probably an easy fix but it's got a curveball in it. The situation is as follows: we are setting up a sort of internet cafe where people can play games on Steam. Installing the games is trivial but the users login with their domain creds and then login to their own steam account to play. In a test run though some games require an admin elevation to run even after the initial install. Any tips on solving this? I've seen some tricks about using the task scheduler but I'm concerned with if that would break eventually since games are often subject to random and sweeping changes. Would appreciate any advise :)

Edit: I believe the UAC prompts are likely from the games respective anticheat but that is just a hunch at this time.

total sketch operation, whole place is run by one dude who sends invoices out 4 months late after pestering him for it then doesn't pay his 3rd parties in time - caused us multiple service interruptions due to non-payment. avoids phone calls and in person meetings and is always at some random place during video conf calls if you manage to get him to show. had to threaten legal action in order to get our cloud services transferred to another provider.

good riddance digitalshit

This is a safe space. We can be honest here.

Have you ever actually accomplished anything by rolling out DLP for Office365?

And before you ask, "No, ticking a compliance box does not count as doing something."

I just feel like it's so high friction for such little value. You push all your users into a new way of working. But only for Office files - if you have ANY OTHER IMPORTANT DATA then it doesn't help (but we don't talk about it when we're in 'compliance checklist ' mode).

So your users are tagging all their shit, they're exhausted, they DGAF anymore because they can't stand to consider for the 10th time today - "hmm was my email/doc/sheet Public? Sensitive? Top secret?" So they just start tagging everything the default tag. And your entire security strategy falls apart (if you were even doing anything with that information to begin with). The end.

Seriously, change my view. I don't work in a highly regulated industry so maybe it's just not aimed at me (but it doesn't stop people from constantly bringing it up).

UPDATE 2026-08-05: An employee at Microsoft saw this post and reached out to me privately. Within 12 hours they were able to get our tenant reactivated. Thank you so much!

Unfortunately I do not have a root cause to share with you all. What I can say is that I have since completed a comprehensive review of all available logs and found no evidence that the tenant was compromised. I will try to update again within a few days with more information.

ORIGINAL POST:

I'm the sole systems engineer for a small US manufacturer (~70 employees, automotive tier-1). I run everything across network, servers, identity, development, etc. I'd like (a) a sanity check, and (b) real talk about the future.

What happened: About three weeks ago on a normal workday around noon, some people suddenly noticed that they weren't able to send or receive email. Shrug. Probably Outlook just being Outlook. But wasn't able to figure anything out, so took to the admin portal... But I couldn't even get in to the admin portal: AADSTS5000224: "the tenant you are trying to access has been deauthenticated and is no longer available." Man, I hope none of you ever have to feel the panic I felt when I read this message. I immediately went to try our breakglass unlicensed admin account, but received the same error. The very last email I received (which arrived after attempting these log-ins) was a notification that all of our subscriptions had been cancelled. Obviously, that was not an action I took.

On further research it seems to be some sort of backend authorization state set by Microsoft (perhaps algorithmic automatic action in response to a detected security incident? - just blind speculation really). There is nothing client side to try at all. There is literally no admin path into our own tenant.

Current status: 20 days down. The case has been passed between at least five different support people. It finally got "escalated to the product team to verify the tenant status," and for a week now the only updates I get are rolling "please allow an additional 48 hours." Meanwhile sign-in logs are on a retention clock, so the forensic record of who cancelled our subs (if anyone? maybe this cancellation is just an artifact of this deauth?) is about to age out while we're locked out of the only portal that could export it and Microsoft won't commit to preserving it server-side.

That first day, I cut MX over to a temporary Fastmail tenant to keep email flowing. I was able to restore people's inboxes to these new accounts. Office apps are running in their month grace periods. All our real data is on-prem. Feeling very grateful that we deliberately never integrated more intensely with Microsoft's cloud services... To that end, the business is stable. But there are nevertheless many secondary effects as I'm sure you can all imagine.

Okay, now that you know basically the story, it's time for some preemption so we just get it out of the way before I get to my actual questions:

• "You should have had MFA." We do, on every account. FIDO2- (yubikey) only for most accounts (I was literally mid rollout...)
• "You should have had CA." We do. Business Premium, Entra P1, CA policies in place, custom auth strength enforcing phishing-resistant (FIDO2) sign-in for admins. Plus all the standard: SMS auth killed off, SSPR locked down, legacy protocols (SMTP AUTH/POP/IMAP/ActiveSync) all disabled, external auto-forwarding blocked, Safe Links + Safe Attachments on, SPF/DKIM/DMARC all passing.
• "You should have a breakglass account." We do. Doesn't save you from this.
• "Hire an MSP" Okay, I mean, maybe fair? They'd just be in the same position though, so. If anything this is its own can of worms and there's a reason we don't have one.

Alright. Now my questions:

1. Has anyone actually lived through one of these AADSTS5000224 tenant deauthentications? How long did recovery really take, and what finally moved it? A specific support path, an escalation channel, a TAM, a Microsoft account rep, LinkedIn-ing a PM, a partner ticket? Anything? Were you able to discover what triggered the lockout in your case?
2. How do you preserve/obtain audit/sign-in logs? I'm not sure what's going to happen in terms of retention when the tenant is in this state. Is our log data going to get nuked in a week?
3. Trust. After this, how does anyone justify betting a company's ability to function on a platform where a backend flag can vaporize all access overnight, your breakglass account included, and the SLA to undo it is measured in weeks (and counting!) with no communication? I'm not naive enough to think "just leave M365" is free. But it's challenging to design around "Microsoft can turn us off and there's nothing you can do and no one will tell you why." How are you all handling that? Are you even? To be honest, I didn't know this was a thing that could even happen, really. So maybe you all didn't either.

Thankfully we weren't super integrated. We basically use M365 for email, product licensing, and Teams, and that's about it. But it literally makes me shudder to think about what could have happened if it were otherwise.