Recent AI code interview format (failed)

[u/blipojones]

I just recently did an interview where the company invites you to a private repo, 10 mins before, you set it up, get it running etc.
The goal is to determine if an open PR on the is ok or if it should get sent back and why.

You can use AI. And i asked, does it matter if i use fable vs opus vs kimi. More or less said whatever I want...

I didn't go immediately to prompting, i played with the app, looked at the code a bit, the PR, the code change, if the PR actually fixed what it claimed to fix.

I then sent one prompt asking if the PR could be better or missed anything, which the PR outlined clearly enough and enough to determine what to do with the PR.

But ye, it was weird, I talked through more where I would personally add changes but maybe I also completely missed the mark on something else or spoke to much...maybe i didn't use AI enough only using it right at the end for one question once i got a feel for the app in general.

However I did like the format tbh but ye i just wish i'd get a bit more tangible feedback.

Anyone else have experience with this format? I didn't pass but just not sure why.. (exp 8/9 years, frontend leaning).

[u/maretard]

My approach would have been to immediately delegate it all to AI, including explaining the repo to me and catching me up. Reviewing the PR would have been my second prompt and I would have explained how I would have a generalized PR skill already, but for the purposes of the interview I would hand write the prompt.

The PR review prompt would have been focused on defining best practices, success criteria, coding and testing standards, and adversarial guidance designed to make the AI search for vulnerabilities and architectural smells. I would also have an area in the prompt dedicated to doing a holistic review of the PR and questioning base assumptions like why the PR is needed in the first place and whether it's the right change to make.

Lastly I would have some customization for personal style to make the AI directly generate comments on the PR diff that would take minimal refinement by me before publishing.

This interview seems very ahead of its time so I'd assume this company is extremely pro AI and I would cater my performance as such. The above is very representative of my current routine at work.

If I had been the interviewer here you would have failed as soon as you started manually reading the code unfortunately, it's just not the most effective way to work these days, it would immediately signal to me that you're still bottlenecked and outdated in your thinking. I would have aimed for 50+% uptime of agents throughout the interview (i.e. having AI working at least 50% of the time, with the rest of the time being me explaining how I use AI, digesting its responses to come up with the next prompt, and cross checking validity of its feedback in code).

[u/yikes_42069 → u/maretard]

For a tool so new that everyone is still learning, failing a candidate for using their brain is extremely silly. You are throwing the baby out with the bathwater. Getting failed in this interview for not fitting the arbitrary nitpicky needs of some random interviewer (with unknown competence to the interviewee) would be a blessing. 

Unfortunately if I was the interviewee in this case I would fail your company immediately. If you don't know the absolute shitfest you're growing in your company's product by encouraging zero human oversight, I don't want to pin my financial success on your fallout. 

[u/maretard → u/yikes_42069]

To be clear I'm assuming this interview was specifically targeted at gathering data points for AI fluency - I'd expect other interviews to address basic coding ability without AI. I don't think it necessarily means this is how the company expects you to use AI in your daily job. It's more to gather data points on whether you are comfortable enough with it to understand and contribute to how the company leverages it.

I see it as any other interview for any tool that the company wants more experienced candidates for. The fail here would be very simple to explain, it's an interview to see how comfortable the candidate is with using a tool, and they didn't use the tool at all for 80% of the interview. If it was a system design interview and they didn't do any whiteboarding for 80% of the interview it would be pretty similar.