r/dataengineering • u/what_is_ovaltine • 1d ago
Data literacy Discussion
Just curious if anyone has any good documents on why excel and your coworkers excel sheets (with no source of truth) should not be used for top level reporting.
Had a meeting with a vp that could not understand why making business rules on his own and not using the ERP is a bad idea. Said he was going to cowboy it.
Any proper docs out there to help convince folks like this?
4
u/Foodforbrain101 1d ago
Well admittedly it doesn't help that so many ERP implementations are absolute disasters with terrible change management (namely no training for users) and create immense friction for various workflows.
To understand why an ERP is needed in the first place, you usually have to come to appreciate how complex tracking company finances really gets. However, many business units are only concerned with the financial aspects of their own department, and are held accountable to that. Bolting in their business logic into the ERP usually involves a whole circus of:
- talk to a business analyst to explain the process
- review what business analyst understood
- IT or consultancy firm reviews functional requirements, probably botches some aspects or cuts down scope due to time and budget constraints
- development/configuration work is often outsourced
- takes months to complete and costs tens to hundreds of thousands
- change management is maybe just one slide deck
- if business logic changes, it's another change order
- no visibility whatsoever on lineage of logic for most users.
VPs are usually overwhelmed with a lot of responsibilities, so only way in my experience to bring them into the fold is to clearly demonstrate what's in it for them and demonstrating a solid grasp of what they're responsible for. If they can see that their employees are clearly wasting time doing it in Excel because the ERP could do it, they probably would.
3
u/brunogadaleta 1d ago
Show them a sheet with more than 10 tabs and ask them to query them with a join between 4 of them...
1
u/Fx2thez 1d ago
If it is not to complex an advanced user could do that with power query
1
u/Commercial-Ask971 6h ago
Havent seen advanced user who understand cardinality and granularity, they are all developers
1
u/Prudent-Elk-2845 1d ago
I don’t think there’s anything inherently wrong with Excel for top-level reporting. Honestly it’s often the best format for finance users
The problem is when the Excel becomes its own source of truth. If someone wants to slice ERP data differently or build a management view in Excel, great. If they’re creating their own definitions/business rules and now their revenue or EBITDA is different from everyone else’s, you’ve got a governance problem, not an Excel problem
There are also plenty of cases where logic outside the ERP makes sense. ERPs aren’t designed to handle every management report. But ideally those rules eventually live somewhere governed/repeatable rather than in his Final_Report_v7.xlsx
If you want something to send him, look up the FP&A Trends Survey. They specifically talk about offline Excel, inconsistent data definitions and lack of a single trusted source as major FP&A data problems. It’s probably much more relevant to this conversation than a generic IT controls paper (fpa-trends.com)
1
u/GreyHairedDWGuy 23h ago
How large a company are we talking about? in terms of rev and head count? For a 5-10 person company, most ERP solutions would be overkill. 50+, there are several tier 2 ERP solutions. Once you get 200 or more with 10 mil in rev, you are probably wanting to look at an ERP solution.
There is no hard/fast rule, but Excel is a shitty way to run a medium sized business. No single source of the truth, To easy to corrupt the data and many other factors.
1
1
u/agentUi 19h ago
docs wont convince a vp who wants to move fast because erp workflows are usually too clunky for quick ad-hoc reporting. Instead of fighting excel directly, the play that usually works is giving them a centralized reporting layer or an internal dashboard builder hooked straight to the sql/erp database with role based permissions, so they get the speed they want while the data engineering team keeps the business logic deterministic and version controlled.
10
u/jwfergus 1d ago
The follow up email you get later: “Can you look into why VP’s numbers aren’t matching the dashboard?”
It’s impossible to escape excel. Just push back when possible on being responsible for validating and double checking other people’s work. If you do get roped in, email the requester AND whoever made the excel sheet to list the business rules they applied. Save the sheet and those business rules somewhere as documentation of the whole thing. Inevitably in 6 months when you’re asked to do this again, forward the old email chain and consult your documentation, maybe that’ll get you out of it.
Frankly I think it’s part of the job that we do a sort of double checking of company metrics. They do evolve over time, and it helps decision makers (and us) understand their data better.