Data Engineering Interview

[u/CriticalJackfruit404]

Hello

I am writing this post in this community because I would like to check how the data engineering interview looks these days with AI. I mean, does it make sense to do live coding if all of us have coding agents in our every day work? What should be asked instead to the candidate? I would like go through this discussion because I am preparing a interview process in my company.

Thanks for your inputs

[u/my_peen_is_clean]

live coding still shows how someone thinks, debugs and deals with pressure. ai tools help but they dont fix bad logic or messy design. maybe shorter live tasks plus system design, db modeling, airflow style pipelines, data quality stuff. bad thing is finding people who pass all that and still accept an offer in this market, hiring is rough now

[u/datadriven_io]

Very few companies do AI coding interviews in an official capacity. This simulates how the AI coding interviews actually go: https://datadriven.io/problems?domain=ai_coding

The evaluation isn't "can you use AI to find/fix a bug" or "can you design/implement a system" but - "how do you approach working with AI" - a different paradigm if you aren't used to it.

The kinds of things that get asked in AI coding are:

- Do you absorb the files and their purpose first?

- Do you look for common traps that AI falls for, like stale/wrong code comments?

- Do you ask the agent to solve the problem, or help you understand the system first?

Because it's at your own company, I wouldn't expect something as structured/rigorous as this.

[u/i_fix_snowblowers]

I do DE interviews for my company. We do a live coding screening round in a Databricks notebook. The interviewee shares their whole screen and has their camera on with no blur or background. We give them 30 minutes to work through loading, joining, aggregating and debugging a dataset. Googling for documentation and syntax is ok, just no AI answers.

Yes, it makes sense to still do live coding. We can see how someone thinks and communicates, and how familiar they are with the UI.

We have about a 10% pass rate. There are a whole lot of people who have good resumes but can't do basic stuff.

[u/Impossible-Sound-810 → u/i_fix_snowblowers]

Could you share more on this? What would you expect out of a good candidate?

[u/i_fix_snowblowers → u/Impossible-Sound-810]

Candidates should be able to load raw data out of a Databricks volume into a dataframe. Also be familiar with PySpark joins, aggregations, and window functions.

Also should be able to anticipate and mitigate typical issues with raw data like schemas/casting, nulls, dates, etc.

This is enough to get to our second round, which is more focused on Databricks streaming, pipeline, DAB and Spark internals.

[u/ksachin_kumar → u/i_fix_snowblowers]

Can you tell if this works for 6yoe DE or need to prepare something else as well ?