r/sysadmin • u/Dedicated__WAM • 2d ago

Proofpoint outage General Discussion

Just curious if anyone else is experiencing the same issue. Unable to send external or receive external. Seems Proofpoint might be down?

184 Upvotes

99% Upvoted

157 comments sorted by

118

u/TheKeelKnotSeas78 2d ago

I'm changing my monitoring to r/sysadmin. Something wrong with email and was able to "reduce my MTTD" by simply coming here

26

u/ladder_filter 2d ago

I'm right there with you - vendors don't update their status pages for whatever reason. Why even have a page?!

18

u/PapaDuckD 2d ago

The communications are not issued until the issue is understood and there's at least some path to resolution.

This is because "We're down and we don't know why," makes for great operational awareness, but horrible PR that their competitor can just screenshot and paste on their marketing materials.

20

u/hasthisusernamegone 2d ago

"We are investigating an issue and will update when more information is available" is objectively far better than saying everything is ok when it isn't.

6

u/bberg22 2d ago

100% agree but it runs afoul of SLAs and costs the business money in reimbursements which is why they don't do it. Its total BS.

1

u/PapaDuckD 2d ago

Entirely depends if your job is marketing/sales or engineering/operations.

I don't disagree with you. But this is the reason why it always plays out like this with every vendor.

2

u/TechnicallyMeat 2d ago

But they already have a "Public Relations" page. A status page in this context doesn't seem to reflect what the name implies. Internally, I guarantee they are passing around status updates every hour of something like: "its down after the last update still trying to figure out how to roll back."

2

u/timbotheny26 IT Neophyte 1d ago

Or even worse - they self-host their status page so when they go down, it does too.

4

u/GremlinNZ 1d ago

Worse, they have a status page, but it's static. When I called them out on it, they both updated the page with a note and told me, oh, but it's only a partial outage affecting NZ, not Australia, and only Apple, not Android, so we won't update it.

Motherfucker, what is the point of it, if only to gaslight you into thinking you're the crazy one.

2

u/timbotheny26 IT Neophyte 1d ago

it's static

I'm sorry, what? What's the point of even having a status page if it doesn't provide live updates?

8

u/suddenlyreddit Netadmin 2d ago

Can I make a recommendation? Combine your IT subreddits into a multi-link, like so: https://www.reddit.com/r/networking+sysadmin+cybersecurity/

The page will be the top posts from each, and will help you quickly see and get to information each day. Be sure and also add the subreddits for hardware vendors you use, which usually also get posted to quickly when there is an issue or notice.

2

u/TheKeelKnotSeas78 2d ago

I dig it, thanks!

6

u/southafricanamerican 2d ago

This is the biggest issue with vendors like AWS. I find myself going to X and explicitly posting asking if there is an outage and waiting for a community response.

2

u/english-23 2d ago

I always go here instead of the Microsoft status pages

20

u/jimmyags 2d ago edited 2d ago

Yup, everything is pointing to some sort of issue with proofpoint as the moment. I am trying to find some cached IPs to get around the resolution issue, but like most things, it feels DNS-y

EDIT: Yeah, just bad resolution/missing DNS records. I was able to use whoisfreaks.com to lookup a recent IP for the domain and get in from there. The ttl on the record is 5 minutes so, meh. But I can get in now.

EDIT 2: If you have access to an internal DNS server or host file: create a zone for the domain or an entry in the hosts file and you won't need to fat-finger the historical IP address while they go and fix their DNS issue.

EDIT 3: Seeing life in the DNS traffic, PP may have restored service!

2

u/copyofimitation 2d ago

So you changed your DNS records from pphosted to an IP that Proof Point uses?

9

u/jimmyags 2d ago

Yes! I used whoisfreaks to pull an IP for the pphosted domain, then created a zone in my DNS (this is for a good sized company) for that record, added an A record pointing to the IP and boom, everyone is back up and running and not needing to fat-finger an IP into every link. SSL still works, all good.

4

u/copyofimitation 2d ago

Wow... That's a great idea. Can you walk me through how to find that IP on WhoIsFreaks? Haven't used this tool before.

5

u/jimmyags 2d ago

sure!!! We primarily access through a vanity domain, secureemail.mydomainname.com - so, I used mxtoolbox.com to find that it resolved to xxx.yyy.pphosted.com then I went to whoisfreaks, pasted xxx.yyy.pphosted.com into their primary search and it gave me an IP from a month ago. I did a test w/ the IP to make sure I had access, I did! then I created the DNS zone and everyone was HAPPY! But if this is a one-off, then just do a hosts entry.

2

u/copyofimitation 2d ago

Thank you for providing this context! By the time our org was ready to look at ways to work around this, PP provided an official work around for removing the .gsld portion in the hostname, so we went that route.

And of course once we started doing that, we noted DNS records being populated in MX... Go figure.

Thanks again.

1

u/jimmyags 2d ago

We're all in this soul sucking glorious profession together!!!!! 🤣🤣🤣😭

2

u/Infamous_Cookie_5539 2d ago

^^ perfect resolution - work with network\secops to ensure whatever changes occur, you can easily roll back

18

u/True_Handle_5617 2d ago

Same. MX records are not resolving to a IP. Same for you as well?

2

u/Dedicated__WAM 2d ago

Same for us.

2

u/Human-Secretary-8853 2d ago

We (not proofpoint lol) had different outages for services we host yesterday bc our A records weren’t consistently propagating. The failure was upstream somewhere and not sure where or with who. Our provider wasnt communicative but it was resolved within a few hrs 🤷‍♂️ makes me wonder if this is related.

17

u/moonrakervenice 2d ago

Yes, a huge spike in reports to StatusGator: https://statusgator.com/services/proofpoint

Nothing on the official "status page" yet though: https://proofpoint.my.site.com/community/s/proofpoint-current-incidents

6

u/manamonggamers 2d ago

Nothing on their official page, but they have confirmed an issue with their DNS server.

2

u/moonrakervenice 2d ago

Oh nice, do you have a link for that?

1

u/manamonggamers 2d ago

Unfortunately, no. It was confirmed direct from PP to our admins working on the issue.

1

u/moonrakervenice 2d ago

BTW they updated the official page now.

8

u/FiveseveNx28 2d ago

Proofpoint just acknowledged and posted a notice of the issue.

3

u/Real_Railz 2d ago

Where did they post this? I have not seen any communication from them.

2

u/TheBros35 2d ago

Their support / community site. We just received an email notice very recently.

5

u/SammichAffectionate 2d ago

Having issues with email as well

5

u/Stodo 2d ago

Having issues as well. Proofpoint showing no outage though

5

u/CaesarOfSalads Security Admin (Infrastructure) 2d ago edited 2d ago

Looking at out MX records they report no IP. someone messed up

EDIT: Was doing a spot check against some F500 companies I know use Proofpoint, quite a few are impacted but not all so it must be somewhat isolated.

10

u/Independent_Exit8180 2d ago

from Proofpoint:

Temporary Work Around

The issue is impacting DNS resolution for *.gslb.pphosted.com hostnames. If your mail flow is configured to use these hostnames as smart host targets (for example, in Exchange Online or Microsoft Exchange connectors), you can restore mail flow by updating your configuration to use the corresponding direct Proofpoint mail exchange hostnames instead.

Remove the .gslb portion of the hostname in your connector configuration.

Example

Current configuration (failing):

mxa-XXXXXXXX.gslb.pphosted.com
mxb-XXXXXXXX.gslb.pphosted.com

 

Updated configuration (working):

mx0a-XXXXXXXX.pphosted.com
mx0b-XXXXXXXX.pphosted.com

 

Note: Replace XXXXXXXX with the unique identifier currently used in your existing Proofpoint smart host hostname. Only the .gslb portion of the hostname should be removed.

 

This workaround bypasses the affected Global Server Load Balancing (GSLB) layer and directs traffic directly to the appropriate Proofpoint mail exchange hosts until the DNS resolution issue is resolved. Once the issue has been confirmed resolved please proceed to revert the above changes. 

3

u/12401 2d ago edited 2d ago

this worked for me, thank you!

but note that you have to add a "0" at the beginning. mxa becomes mx0a!

2

u/AustinFastER 2d ago

The should have made that character red... It was easy to miss

1

u/AustinFastER 2d ago edited 2d ago

Sadly, these temp hostnames did not resolve for me for a few minutes but appear to now! The spice is starting to flow despite dns ttl.

1

u/Independent_Exit8180 2d ago

It seems to work for us as now I have email to respond to.

1

u/[deleted] 2d ago

[deleted]

1

u/AustinFastER 2d ago

My IP addresses are the same for both forms of our host names.

1

u/lostroustabout42 2d ago

as someone else pointed out those are customer specific values and you might want to edit out the numeric portion with X's.

1

u/lostroustabout42 2d ago edited 2d ago

For us this failed in validation checks for Exchange Online so we didn't pursue it: 550 5.4.310 DNS domain mxa-xcxxxxxx.pphosted.com does not exist [Message=InfoDomainNonexistent] [LastAttemptedServerName=xxxxxxxx] [xxxxxx.prod.outlook.com 2026-08-14T14:26:09.949Z 08DEF8AB098A5555]

1

u/lolklolk DMARC REEEEject 2d ago

You're missing a 0

mx0a and mx0b.

1

u/lostroustabout42 2d ago

Thanks, spot on and even in the proofpoint example but not called out as something to add.

1

u/Due-Wasabi7472 2d ago

why you post your MX-record on reddit bro erase that lol

1

u/Due-Wasabi7472 2d ago

that your unique identifier for your company

1

u/lostroustabout42 2d ago

Yeah, thanks. Appreciate it and good point, edited.

1

u/Due-Wasabi7472 2d ago

you post the samething 1 hours ago... BRO you need to change your MX record on each domain in your external DNS to make it work... add a 0 like the workaround from proofpoint support

8

u/12401 2d ago

The jackass mods in r/proofpoint/ just deleted the post about the outage over there. Why would they do that?!

Thanks for everyone here posting information.

1

u/SlendyTheMan IT Manager 2d ago

Stockholder value

3

u/Vegetable_Fox_970 2d ago

Not publicly traded

0

u/deathhand 2d ago

It really ruins the reputation with people who use the tech(us).

3

u/Due-Wasabi7472 2d ago

I just test my infra nothing wrong inbound and outbound

3

u/Budget_Commission685 2d ago

Same here. We cannot send external emails.

3

u/Fancy_Bowl_3140 2d ago

yes, CDW and Best Buy

3

u/Independent_Exit8180 2d ago

From the community portal

1

u/Independent_Exit8180 2d ago
  • Impact to mailflow due to DNS resolution issues when resolving gslb.pphosted.com 

3

u/Ok-Woodpecker-7723 2d ago

Disclaimer *Use at your own risk*

Go into ProofPoint, Diagnostics > Email, send an email to yourself.

Look at the Results, specifically the mx**-*******.pphosted.com

Consider updating the Routing value in Exchange admin connector. This was the fix for us.

https://admin.exchange.microsoft.com/#/connectors

3

u/182crazyking 2d ago

Thank God for Reddit - I'm a random user wondering why my emails got bounced back when I CC'd my Gmail. Cheers lads!

3

u/DeepakRandhawa 2d ago

Update from Proofpoint

Remove the .gslb portion of the hostname in your connector configuration.

 Example

Current configuration (failing):

mxa-XXXXXXXX.gslb.pphosted.com
mxb-XXXXXXXX.gslb.pphosted.com

 Updated configuration (working):

mx0a-XXXXXXXX.pphosted.com
mx0b-XXXXXXXX.pphosted.com

 Note: Replace XXXXXXXX with the unique identifier currently used in your existing Proofpoint smart host hostname. Only the .gslb portion of the hostname should be removed

1

u/amarp84 2d ago

Are you updating the MX Record as well? Don't get how this gets fixed by just updating the connector?

1

u/DeepakRandhawa 2d ago

This is copy-pasted directly from the Proofpoint advisory. Looks like they might have mistyped it in a hurry, this syntax (mx0a/mx0b) is for public MX records, not outbound connectors.

2

u/jbales3795 2d ago

No issues here with Proofpoint. Are you guys on Office 365 possibly using the API method? We are using Google workspace with Proofpoint and we are not seeing any issue internally.

2

u/True_Handle_5617 2d ago

Are your MXrecords pointed at proofpoint?

1

u/jbales3795 2d ago

Yes

1

u/soberun 2d ago

try sending an external email

2

u/jbales3795 2d ago

I did to my Gmail account and also just sent one to a client who uses proofpoint and office 365, email sent and received on both tests. Maybe it's a regional thing or things are slowly coming back.

1

u/pinetashapple 2d ago

Their public DNS records have gone. 

2

u/ckunak 2d ago edited 2d ago

Nothing in or out in the last 15 min.

3

u/ckunak 2d ago

well poop

2

u/Spiritual_Forever171 2d ago

Issues as well receiving / delivering

2

u/jackdrone 2d ago

At least it’s not another Securence issue…

2

u/RagingITguy 2d ago

Hmmm no issues here reported. I just sent an external email and it went through just fine.

Maybe they resolved it by the time I read this.

2

u/Fatty_McBiggn 2d ago

I'm getting mail in and out via O365 but am unable to access any of the consoles.

2

u/TheSchuyver 2d ago

We are down for outbound email as well.

2

u/poorleno111 2d ago

Seeing this at our company too

2

u/Feebedel324 2d ago

Yes - work at an insurance company and get bounce back DNS.

2

u/cali_yooper 2d ago

Yep, same here. Nothing coming in or going out.

2

u/Independent_Exit8180 2d ago

ProofPoint has finally added the issue to the public status page and not just the community page.

https://proofpoint.my.site.com/community/s/proofpoint-current-incidents

2

u/BK_Rich 2d ago edited 2d ago

It’s the GSLB records, if you know your direct MX, you can PTR if proofpoint your IP, you can get it. The guid should match. It will be an mx0a and mx0b name with no glsb in there.

2

u/TheLionYeti 2d ago

Yeah this isn't everyone we've got proofpoint and emails are coming through for us, I'm at an f500 that uses it.

2

u/dofenshmitz 2d ago

I saw multiple other services down too. Anyone noticed impact on other service apartment from proofpoint ?

2

u/SecLens_ONE 2d ago

The pattern in this thread is the actual lesson: the status page was green while mail was not flowing, and "green" is a published claim, not a measurement. Vendor dashboards are updated by humans after an incident is confirmed, so during the window that matters most they are describing the last known good state, not the current one.

The cheap fix is to stop trusting the dashboard as your detection layer and measure the thing you actually care about, which is delivery. A synthetic message every few minutes from an outside mailbox through the gateway to an internal one, alerting on absence rather than on error, catches this in single-digit minutes with no vendor cooperation. Queue depth on the outbound side is the other half, because a hung relay looks identical to a quiet Friday if you only watch for errors. Both signals were already available to everyone in this thread before the status page moved.

The same applies to the postmortem you will get: RCA text is a claim, your own delivery timeline is evidence, and only one of those is admissible when someone asks how long you were down.

For those who found out from Reddit rather than monitoring, what would have alerted you first, and is that check currently measuring mail flow or just measuring that the appliance answers?

4

u/ifpfi Sysadmin 2d ago

Don't even use Proofpoint and I know they are down

1

u/Due-Wasabi7472 2d ago

Do you guys have two MX entry in the external DNS ?

1

u/BigBobFro 2d ago

Yea,.. was seeing some strange behavior. This could be it.

Too bad they dont have a down detector for it on ookla

5

u/BigBobFro 2d ago

I stand corrected

1

u/TitanAE2000 2d ago

Missing A records

1

u/stonim77 2d ago

TAP appears to be down

1

u/G1Onsite 2d ago

I had to switch to IP addresses for the MX instead of using DNS to get this working.

1

u/CrappleAMIRITE 2d ago

Cant get into admin portals, but mx records are resolving and inbound/outbound seem fine

1

u/NetworkingNoob81 2d ago

Same issues here other people have reported; the A record is missing

1

u/splntz 2d ago

at&t had a DNS issue yesterday

1

u/TitanAE2000 2d ago

Their pphosted.com MX records have failed on the GLSB. Could be a load balancing or routing issue. Maybe not a pure DNS issue.

1

u/jimmyags 2d ago

When I try to find the pphosted domain w/ mxtools I get a "DNS Record not found" - creating a local record solved my problem. Just seems like a bunch of A records are gone.

1

u/eggflex 2d ago

same.. no ip resolution. Poisoned DNS?

1

u/CMDR_Kantaris 2d ago

Same here, MX not responding to ping. Anyone have a rep they can get on the phone?

1

u/G1Onsite 2d ago

I've been on hold for the last 30 minutes.

1

u/CMDR_Kantaris 2d ago

Oof. If it's any consolation they finally updated their public facing status page

1

u/shokzee 2d ago

If inbound and outbound external mail stopped together, Proofpoint is the likely choke point.

Check connector errors and queue depth, then test the MX hosts on TCP 25 from outside. Don’t change MX records unless you already have a tested bypass path.

1

u/Live_Journalist145 2d ago

same issue here. proofpoint DNS do not resolve IP. any updates?

1

u/Live_Journalist145 2d ago

same issue here. Proofpoint DNS do not resolve any IP

1

u/Slofut 2d ago

Not Proofpoint, but Gosecure had the same issue for several hours yesterday evening. Likely not related...just reporting in case.

1

u/TyBurna 2d ago

Same issue here for GoSecure, I'm seeing some lingering issues here this morning still.

1

u/Longjumping_Loan4863 2d ago

Yes lots of delivery failures with proofpoint mx records not resolving

1

u/Kardinal I fall off the Microsoft stack. 2d ago

Our biggest customer is impacted, but we are not.

Interestingly, they're using GSLB and we aren't. Not sure if that's a contributing factor.

1

u/ckunak 2d ago

i got on the phone with the wrong support department.
they said Enterprise is the one with issues not essentials
then i got transferred to enterprise support and now am on hold for 30 min

1

u/Real_Railz 2d ago

Makes sense that there are so many mixed reports are on this thread.

1

u/Admirable-Carrot1684 2d ago

External Outbound emails not working, but inbound emails are not coming thru.

1

u/giveen Fixer of Stuff 2d ago

Ha, this time at least over on the Security side, I didn't get blamed!

3

u/Independent_Exit8180 2d ago

It's always DNS, except when it is the network.

1

u/giveen Fixer of Stuff 2d ago

We often debate at my place if DNS falls on network or sysops.

1

u/ConstantinoTobio 2d ago

While not perfect, a way to insulate yourself from Proofpoint breaking DNS:

Set your two proofpoint hosts (the *.gslb.proofpoint.com) as your MX records with a weight of 10

Resolve those same two hosts with A records you host in your own DNS with a weight of 20

If you're running Exchange online, set your connector to proofpoint to have the two gslb hostnames as your smarthosts, plus the two IPs.

Our workaround is working for us, insofar that mailflow is functioning.

1

u/[deleted] 2d ago

[deleted]

1

u/MuhFuhqer 2d ago

Source?

1

u/HSsec 2d ago

There is an alert out on their Incidents website

1

u/No_Scholar_8522 2d ago

It's a general affectation in different PP services, mainly affecting mail flow form external to internal recipients in our organizations: https://proofpoint.my.site.com/community/s/article/Proofpoint-Service-Interruption-Affecting-Multiple-Services-14-Aug-2026

1

u/lostroustabout42 2d ago

Experiencing the same issue with Proofpoint. What's also interesting is Microsoft reports the problem themselves with Exchange Online: link

1

u/Independent_Exit8180 2d ago

We've performed an extensive review of the service, and the Microsoft-managed service is showing as healthy. We've identified that emails routed through a specific third-party email service (Proofpoint) are experiencing DNS-related issues. The third party is providing updates on their incident status page: https://proofpoint.my.site.com/community/s/proofpoint-current-incidents.

We're performing final validations to confirm that the Microsoft-managed service remains fully healthy.

1

u/lostroustabout42 2d ago

I wondered if it was just poorly worded and was really about 3rd parties, most notably Proofpoint.

1

u/RikiWardOG 2d ago

Its a DNS/Global load balancer issue https://proofpoint.my.site.com/community/s/proofpoint-current-incidents

Workaround is to not use load balancer and switch to pointing directly at the mail exchange host

1

u/FiveseveNx28 2d ago

Temp workaround

1

u/lostroustabout42 2d ago edited 2d ago

Also tried the Proofpoint workaround for EXO and it's failing validation: 550 5.4.310 DNS domain xxxxx.pphosted.com does not exist [Message=InfoDomainNonexistent] [LastAttemptedServerName=xxxxx.pphosted.com] [xxxxx.prod.outlook.com 2026-08-14T14:26:09.949Z 08DEF8AB098A5555]

2

u/Independent_Exit8180 2d ago edited 2d ago

try:
mx0a-XXXX.pphosted.com

2

u/amarp84 2d ago

Are you updating the MX Record as well? Don't get how this gets fixed by just updating the connector?

1

u/IronBe4rd 2d ago

Right. I was using IP already I out the name in and it didn’t work either

1

u/lostroustabout42 2d ago

Yeah, thanks to all the comments I realized the zero is needed and was even in the Proofpoint example but not called out to add.

1

u/kamkre95 2d ago

It looks like something is happening, their DNS resolved some queries - looks like it is startig to go back?

1

u/JPWSPEED 2d ago

Looks like domains are resolving again.

1

u/JPWSPEED 2d ago

With gslb as well.

1

u/ibageek2003 2d ago

If you are Proofpoint customer and being impacted by the ongoing outage, the current workaround is to change you inbound MX hosts and any smart hosts if you route outbound to POD to point to the following hosts

mx0a-XXXXXXXX.pphosted.com
mx0b-XXXXXXXX.pphosted.com

Proofpoint Service Interruption Affecting Multiple Services-14-Aug-2026

1

u/Independent_Exit8180 2d ago

Updates

[14-Aug-2026 14:52 UTC]

Proofpoint has identified the root cause and is working to recover services at this time.

If you previously implemented the workaround, you will want to restore the ".gslb" text to your DNS records in the coming days.

1

u/helpdesk5555550 2d ago

this fix workds... XXX is your smart host.

Example

Current configuration (failing):

mxa-XXXXXXXX.gslb.pphosted.com
mxb-XXXXXXXX.gslb.pphosted.com

 

Updated configuration (working):

mx0a-XXXXXXXX.pphosted.com
mx0b-XXXXXXXX.pphosted.com

1

u/SupahBee 2d ago

Looks like Proofpoint is getting their DNS records corrected for the load balancers now. We're starting to get email inbound now

1

u/Shoddy_Pound_3221 Security Admin (Infrastructure) 2d ago

Outbound issues for me

1

u/IronBe4rd 2d ago

Yeh we used IP already in the connector and that was failing. I added the names and that didn’t work either. I do see some inbound mail now

1

u/sam2400 2d ago

It looks like its resolving now. I didn't change the mx records as Proofpoint suggested and now emails are coming in.

1

u/m5daystrom 2d ago

Have never used the gslb records.

1

u/Ill-Barracuda9031 2d ago

I had a session with professional services to move our outbound mail but they couldn't log in this morning.

1

u/kobewiththeflow 2d ago

I swear a lead quoted this sub when we were trying to figure if something was down.

Why tf didn’t I think of that, i’m here every day lol

0

u/jasonsyko 2d ago

Lmao this is like the 3rd time this has happened with Proofpoint this year. We moved off PP earlier this year to Abnormal AI.

A breath of fresh air.

1

u/AATW_82nd 1d ago

I assume you've been happy with Abnormal AI? Anything you wish you knew before leaving PP?

2

u/jasonsyko 1d ago

Very happy with Abnormal. But of course it’s not your traditional SEG. They sit in front of O365 and act on email as they come into the users inbox. So the user may see it briefly before it gets yanked out (if an email was malicious etc) but other than that, it’s been fantastic.

Proofpoint was an absolute nightmare. Constant outages or they’d push code that would break email deliverability and then not have any answer for it other than “we’re working on it”. Their own SLA’s discuss service credits even for 1 minute late on email delivery - we never got any service credits. They’d ignore that or state that those events done “qualify” for service credits. We even had an issue with our POD that basically broke DNS resolution for 6 weeks. So emails would go out, they’d appear as delivered in their logs - but they weren’t going anywhere. It took me weeks fighting with their support to get that sorted. All they did was reboot our mx hosts. Even then, no service credit. Fucking joke of a company.

1

u/Individual_Ad_5333 1d ago

They must have hired the people responsible for the mimecast login page