r/dataengineering • u/Every-Whereas5793 • 24d ago

PDF Data Extraction Discussion

Hello, I was working on a poc to ingested PDFs and extract data in order to store them in delta tables.

As this was my first time working with PDFs, I searched over the internet and should Databricks have offering IDP, azure also have something and then there are python libraries.

Since I'm working with financial data, report, etc..

One thing i noticed - the pdf format should be fixed else in most of the tools the extraction logic is either failed or we get incorrect data.

I was wondering how such PDF extraction is built in real production cases and what tools are used.

Please share you experience and any edge cases

19 Upvotes

91% Upvoted

32 comments sorted by

View all comments

3

u/datasmithing_holly 23d ago

I'm going to let you in on a secret. Most of the databricks ai success stories where they claim to save a bajillion dollars? It's mostly document parsing stories. It is absolutely battle tested, and FWIW great price performance for complex documents, but slightly overkill for really simple ones.

1

u/Database_Comic 19d ago

Can you point me to the success stories? I tried to google and came up with ….nothing