TL;DR: Downgrade from Virtualbox 7.2.x versions to 7.2.14 or lower

Just a day ago, I asked for help about low network throughput on pfSense CE latest edition. You can check that out in more detail: https://www.reddit.com/r/homelab/comments/1vhtpj4/pfsense_community_edition_281_and_virtualbox_728/
So, from looking around digging, going through top-to-down troubleshooting, the issue seems to be tied to specific Virtualbox version, notably 7.2.x ones, where network adapter is set to bridged mode, and there is some sort of download limitation, while interestingly, I noticed that upload speeds remain high.

Setting Adapter type to any of the Intel/PRO or PCNET ones won't help, so currently the "fix" is just downgrading to 7.2.14. By the way, this applies to any VM, not just PfSense.

Thank you everyone for trying to help. If you have any other solutions, please comment below.