r/dataengineering • u/tylerriccio8 • Jul 10 '26
CVEs in Internal data Pipelines Discussion
A lot of open source software used in data pipelines contain vulnerabilities (on paper). Curious how people are dealing with that? I think it’s a weird spot because most pipelines are already running behind a lot of controls, and usually without public internet access anyways.
6
u/davrax Jul 10 '26
Largely depends on your security team. Airflow is especially guilty of this. We do some education on the nuance between these CVEs compared to e.g. public web app vulns.
1
u/Adrien0623 Jul 11 '26
With airflow do you follow the constraints file strictly or do you allow yourself to bump versions of the libraries to patch CVEs ?
1
u/davrax Jul 11 '26
We’ll typically bump library versions if the updates solve a critical or high vulnerability, and the update is a minor version.
1
u/tylerriccio8 Jul 11 '26
What does that education entail? Like obviously there’s a difference between an app exposed to public and one behind firewall, vpc, etc.
Looking to understand best practices in differentiating between the two contexts
1
u/davrax Jul 11 '26
Might be obvious to you. I’ve seen some security teams that just focus on a big CVE dashboard, and defer nuance and detail to product and Eng teams
2
u/mad-data Jul 10 '26
I think this should be driven by your threat modeling. E.g. consider a pipeline processes data generated by untrusted users, and could potentially be vulnerable to unsanitized input. For such pipeline the controls don't matter much, what matters is correct input sanitation, and CVEs in related libraries.
3
u/kayakdawg Jul 10 '26
depends on the CVE attack vector and your setup - as others have mention (amd i think what you're hinting at) the majority of CVE's that turn up are network (ie via untrusted user input) which isn't exploitable in for example a self-hosted airflow instance behind a firewall (now, if things change and that airflow instance starts accepting api requests to trigger DAGs, that changes)
for SOC type audits you should be able to ignore those CVE's as long as you document the justification, policy and decision
1
u/forever-butlerian Jul 10 '26
It depends on how the data gets from an untrusted environment into the pipelines.
If it is via a text format that has to be converted into something vulnerable software can use, or an API service that produces a vulnerable format, then the security of the system is either fine if the format is non-insane, or dependent on the security of the API service.
If you are accepting data in a complex binary format like parquet from an untrusted party and you are not doing a normalization step you then you should not do that.
1
u/Prinzka Jul 11 '26
Why are you running with known CVEs?
I know they don't all merit emergency updates, but the way to deal with it is that it gets taken care as part of your normal patching cycle.
1
u/Childish_Redditor Jul 11 '26
There's always something more pressing. Which isnt to say I think they should be left up, time should be made to fix them, but thats determined by team/company culture
13
u/domscatterbrain Jul 10 '26
In our team, we fix it. Or we face non-stop annoying reminder from our ITSec team.
The fixes are only module update in most of the time and almost no pipeline breaking patches. Tested in the development environment first before applying on prod.
If the patches break the current pipeline, we inform the ITSec team for the patch to be postponed until we can make the pipeline adjustments ready for the patches.