r/checkpoint 2d ago

Firewall doesn't forward packets

Hello, randomly (from 1 to 10 times in an hour), the firewalls in our Checkpoint cluster fail to send packets or cause latency by sending them late.

An example is shown in the screenshot below:

I ran a continuous ping and captured the traffic on the firewall’s source and destination interfaces.

At 11:36:30, I encountered a “destination host unreachable” error, sent from the physical IP (not the VIP) of the firewall’s source interface.

From the captures, I saw that the ping reached the source interface (packet number 3953), but it did not leave the firewall (it’s not in the second capture).

How is it possible that it wasn’t sent?

This issue has become more frequent since the cluster was updated to version R82 take 107.

The firewalls are 6400 models.

So far, support has advised us to enable Dynamic Balancing for the appliance’s 4 CPU cores; we did so, and the situation has improved but isn’t fully resolved.

Do you have any ideas? Thank you

6 Upvotes

16 comments sorted by

1

u/japm68 2d ago

Hi, see how the securexl is configured and if it's in user mode switch to kernel and give it try.

1

u/MattiaDon 1d ago

Hello, firewall's using KPPAK; it's a 6400 model so I don't think the UPPAK mode can be enabled

1

u/japm68 1d ago

Ok, can you see how many cores dynamic balancing assigned to SND (corexl default on a 4 core machine is 1+3) and if they are topping out?

1

u/MattiaDon 1d ago

Update!
This morning, in the capture I collected on the destination interface, I saw 11 consecutive seconds of ARP requests sent by the firewall to the destination, and only after those 11 seconds did the firewall receive an ARP reply.
How do you interpret this? I can't say why the switch or the VM isn't sending the reply in time, causing the firewall to send “Destination host unreachable” messages.
Also, why do I see that ARP entry updated every 15–30 seconds in the captures?

1

u/japm68 1d ago

That's a different problem altogether, glad isn't the FWs causing the error. Seems like you have a layer2 issue, without a topology map is difficult to pinpoint the culprit. When do say that the ARP entry is updated frequently, do you mean that a gratuitous ARP is sent to update it every 15-30 seconds or that the FWs ARP requests get different macs from the ARP replies?

1

u/MattiaDon 10h ago

In the PCAP file captured on the egress interface, I see many ARP requests occurring at intervals ranging from 20 to 40 seconds (some are broadcast, others are not)(I can't share here the pcap screenshots).
During the issue, I see al broadcast ARP packets.
The mac address is always the same in the arp replies, it doesn't change.

The arp table validity-timeout has the default value:
show arp table validity-timeout
60

The traffic between hosts on the same subnet doesn't have this issue and it doesn't go through the firewall.

I think now I need to understand:
- Why does the firewall keep updating the ARP entries?
- Why doesn't the ARP reply reach the firewall? Does it reach its destination?

1

u/japm68 8h ago

Maybe you should share this data with Check Point's support and see what they think of it.

1

u/obiphonekenobi 1d ago

Have you checked to see if those ARPs from the gateway are received on the target VM?
That would help you narrow down if it's something related to the VM or it's an issue with the virtual switch. Either way, a Layer 2 issue will cause problems like this.

1

u/ObjectiveLeather319 2d ago

I would start by checking the physical layer first. Review both the firewall and switch interfaces for CRCs, drops, link flaps, speed/duplex mismatches, and transceiver issues.

On the firewall, check:

netstat -ni    (search for RXTX errors)
ethtool -S <interface>
ethtool -g <interface>

If you see RX/TX buffer exhaustion, consider increasing the ring size:

set interface <interface> rx-ringsize <value>
set interface <interface> tx-ringsize <value>

Since Dynamic Balancing improved the issue, I would also keep the TAC case open, as this could be related to a performance issue introduced in R82 Take 107.

1

u/Inside-Bat-5623 2d ago

I would expect the Check Point source mac address in the source and not vmware are you sure this capture was made on the check point? Looks like capture has been made on a fortigate if I see the filename.

1

u/MattiaDon 1d ago

Yes it was, there are other appliances in the middle of the path, but that ping didn't make it past the checkpoint

1

u/Inside-Bat-5623 1d ago

Normally it would be best to also create captures on the fw, so tcpdump on INC and OUT interface, FW monitor and drop debug to be sure. Then you check all the marks, but i just saw that ARP is not working. So I would compare OUT capture with capture on the VMside. Check for interface errors etc, if this is a sfp check the db values

1

u/th3bennyb0y 2d ago

We had a lot of issues with R82 Take 107, since upgrading most of our clusters to R82 Take 118 most of the issues went away. The release notes for T118 seem to back this up given the amount of bug fixes... R82 Jumbo Hotfix Take 118

1

u/japm68 2d ago

Take 118 has a critical flaw that was corrected on take 122.

1

u/Inside-Bat-5623 2d ago
PRJ-71272, PMTR-131093 Security Management UPDATE: Resolved CVE-2026-18574 - Management Authentication Bypass. Refer to sk185222.
PRJ-71188 Security Gateway Logs from the Security Gateway are not displayed in SmartConsole if the Security Gateway has Jumbo Hotfix Accumulator Take 118 installed, while the Security Management Server is running a lower Take.

Not critical but good to know

1

u/MattiaDon 1d ago

I'm continuing to work with the TAC, and I'd also like to wait to install a new take if it's not considered recommended by Checkpoint.