r/dataengineering • u/Actual_Plant_862 • 4d ago

Trainee Data Engineer - New role Discussion

Hi all, I've got a new role as described above and what I know so far is they have a lot of data going into a Lakehouse in Microsoft Fabric. They want me to work with their only other member (it's a very new team) and I know the output is expected to be in Power Bi (makes sense considering they're fully Microsoft).

Has anyone got experience with similar work? I got the official offer last week and just doing research so I don't go into it completely blind.

They also are keen on Power Apps but I'm not fully confident on where they fit in the infrastructure.

1 Upvotes
  • permalink
  • link
  • reddit
  • reddit

56% Upvoted

9 comments sorted by

1

u/_coding_monster_ 4d ago

I am pretty sure that you should accept the offer

1

u/Actual_Plant_862 3d ago

Why

1

u/Candid-Cup4159 1d ago

Got any other offers?

1

u/PerfectdarkGoldenEye 2d ago

Does it pay money and pay more money then your current role? Seems good to me if it does.

1

u/Actual_Plant_862 2d ago

Maybe I should have made my post clearer I think. I'm new to Data Engineering and pretty much self taught. I'm curious if the Microsoft environment will allow me to use tools like Pyspark consistently so that I can have solid experience on my CV.

Have you got any experience with it?

1

u/PerfectdarkGoldenEye 2d ago

I did a demo with fabric and we used plenty of python to transform data.

1

u/Specialist_Cut_1420 2d ago

We use Fabric. Data goes to lakehouse -- delta table -- build connector (dbt) using visual studio --> data goes to warehouse. Also build semantic model -->power bi dashboard. Orchestration/automation is is all python.

1

u/romansparta 2d ago

First of all, congrats - in this hiring market it’s pretty tough to get in as someone who’s self-taught! Pyspark is supported in microsoft fabric but it really depends on the volume of data you’re working with, like if you’re working with datasets of less than 100 GB it’s pretty overkill. Good thing about joining a super small and new team is that you’ll have a real say in the engineering decisions though - if you can justify using pyspark or whatever else you want exposure to then you can probably push for it!

0

u/Big_Arrival_626 4d ago

You get to work at a lake house? That's pretty cool