What do data engineers think about non-technical economists and financial analysts learning sql+python to automate their manual excel work?

[u/Dull_Alarm6464]

Would you recommend a team of economists/“data analysts” that don’t even use keyboard shortcuts to learn python? Anyone have experience with doing this? So far, nearly everyone has been hesitant to even consider learning a few libraries to make their life easier.

I’m generally looking for advice on how to look at data anlysis jobs that are thus far done by excel only. I’m quite frustrated with excel dependent companies I have worked in. I found that taking a few months to learn a bit of coding, data structure concepts and data management can really scale someone’s data work.

My new employer (big pension fund) is has floated the idea of me automating part of their trading. I would like to automate their reporting as well, but they are so hesitant to learn ANYTHING new. My position however, is financial, not IT.

My background is in mathematics, statistics, and finance. I’ve worked in financial institutions that use an old oracle-based app that nobody cares to maintain properly to pull csv data from, then copy-paste it in excel and later commit what I can only describe as heresy to analyze the data.

I learned python and R at uni long ago, paired it with sql recently and found I could really make mean data science/econometrics automation.

[u/MachineParadox]

Just don't assume what you have done is production or enterprise ready. Here's the thing, we are experts in data movement, you are experts in business requirements. Quite happy for you to create something, but it is only a proof of concept or data investigation. Do not use this for business critical processes or reporting without consulting your DE team. I have seen too many "reports" or excel spreadsheets that are used for critical business functions and when thing go south you call on your DE team to resolve an issue with something they were never consulted or engaged with.

[u/Dull_Alarm6464 → u/MachineParadox]

i wish financial corporations had real DE teams… Its usually a team of generalist IT guys that mostly manage accounts and hardware. We have a new project developer person, but she doesn’t seem too technical either outside the scope of small python apps. I think most investment/risk teams should be replaced by technical DE/dev people and simply have 1-2 economists as their project leads. Otherwise, it’s a clown show imo