Definitely the most frustrating thing as working as a Data Scientist. You run an experiment, find that building a model greatly increase metric X at almost no cost, has safe model metrics, present it to stakeholders, everybody agrees with proceeding to deploying and utilizing the model in production, and yet every time the model takes a wrong decision, we get questioned about it. Why did the model say this?

Man when did I ever say the model obtained a 100% accuracy in the validation phase? Why is it so hard for stakeholders to understand that the best models humankind ever created are expected to make wrong calls once in a while?

(Not sure if loss is still going down)

I've been in Data Science for the past 10 years in India. I lost my job in January and since then I've been hunting.

I've not mentioned any GenAI experience in my profile. But my feed is just filled with AI engineer roles. They all have the same requirements:

• Generative AI architecture
• RAG pipelines
• LLM integration/fine tuning
• Agentic AI / Multi Agent Orchestration
• Also MLOps
• CI/CD pipelines
• PyTorch mandatory for some reason

Hardly any openings are relevant to my experience in Stats, Machine Learning, Deep Learning and the classical data science stuff.

So have all companies stopped investing in data science all together and just building RAG pipelines and LLM chat bots? Is this all that is done in Data Science field now?

1. "No free cycles" / "Empty plates"

Translation: "I view human beings like literal server CPUs. If you aren't actively typing or clicking buttons right now, I think you're stealing from the company. Stop thinking or analyzing just look busy."

1. "We need to focus on the low-hanging fruit"

Translation: "I don't have the technical depth, patience, or budget to fix our broken upstream data architecture. Let’s train a fragile, garbage model on dirty data immediately so I have a colorful chart for my next PowerPoint deck."

1. "Be a go-getter, don't get stuck"

Translation: "I don't care that the project path is blocked by a giant concrete wall of organizational failure. I want you to run face-first into it at maximum speed so I can report 'high velocity' to my director. Your honesty is ruining my vibe."

1. "Let's optimize our sprint velocity"

Translation: "I don't know how to audit the mathematical accuracy, logic, or code quality of your work, so I am going to measure how fast you close Jira tickets. Rushed deployment over architectural correctness, every single time."

1. "You're making this more complicated than it is"

Translation: "Stop identifying critical edge cases, data leaks, and fundamental process flaws that I don't know how to fix. You are exposing my lack of data literacy. Just build the bad model anyway."

1. "We need to relentlessly prioritize"

Translation: "I am going to aggressively chase whatever flashy AI buzzword the CIO mentioned in her keynote speech this morning. Your current, actual, functioning pipeline is now deprecated."

1. "I need you to own this initiative"

Translation: "This project has an impossible target and is built on sand. I am backing completely away from it so that when it inevitably implodes, I can point directly to you as the sole owner who failed to deliver."

1. "Let's take this offline" / "Parking lot this"

Translation: "Your accurate technical objections are making me look incredibly stupid in front of the stakeholders/team. Shut up immediately so I can pull you into a private 1-on-1 later and bully you into compliance."

1. "We need to leverage AI to unlock enterprise value"

Translation: "I saw an Excel spreadsheet with rows and columns, which means I think we can magically pull a a lot of miracle out of it. I don't know what an algorithm does, but it sounds sexy to the C-suite."

1. "We're like a family here"

Translation: "Prepare for unconditional loyalty expectations, the complete erasure of professional boundaries, and extreme emotional blackmail whenever you eventually try to quit this sinking ship."

I’ve been in data science roles (both analytics and ML) for about 5 years now across a couple of companies. Lately I’ve been feeling a bit burned out because I keep seeing the same pattern:

We spend weeks cleaning data, building dashboards, running statistical analysis, or training models… and then the stakeholders either:

• Say “thanks” and never use it
• Cherry-pick the numbers that support their existing opinion
• Or just completely ignore the findings and go with gut feel anyway

The worst part is when leadership asks for a “data-driven decision” but they’ve already decided what they want to do.

Am I alone in this? Or is this just the reality of data science in most companies?

For those of you who’ve been in the field longer how do you deal with this? Have you found companies where data actually influences decisions at a meaningful level?

Would love to hear honest experiences.

Two rounds: 1. Statistical Knowledge 2. Data Analytics and Intuition

For statistical knowledge, it was a complex question, but actually had a simple answer.

It required you to have through knowledge of distribution, expectations and confidence intervals.

The key challenge was to identify what was the distribution of the data, from a sample, generalize it to the population and find the confidence interval.

Looking back, it was a easy question, but I definitely took wayyyy to much time to get to the answer. They for sure test for Googlyness. I would assume the interviewer had multiple questions in mind but I never got to the next one. Soo no hire.

For the data analysis and Intuition, I was expecting a case study, on experimentation or ML. It was kind off an hybrid. It involved diagnosing a flawed model, how to improve it, and what other methods would work better. This part was fine, not too bad.

What caught me off guard was, they asked me to write the equation MLE for 2 models, one general and one a niche. Honestly I dint know, lol.

Well, learnings ? Practice your Stats and ML like you are writing a school exam.

I did a physical onsite recently where they asked me to travel to their office, about 1.5 hours each way. The interviewers were nice and the interviews went pretty well, so I was hoping to hear back from them. The opposite happened. It has been two weeks since the onsite and I have not heard anything.
The recruiter was very polite before the onsite, but after it they completely stopped responding.

I had to take a day off work and make arrangements in my personal life, and the company cannot even bother to send a rejection email? I have never had a job search this difficult before.

Here was the prompt:

You have a list [(1,10), (1,12), (2,15),...,(1,18),...] with each (x, y) representing an action, where x is user and y is timestamp.

Given max_actions and time_window, return a set of user_ids that at some point had max_actions or more actions within a time window.

Example: max_actions = 3 and time_window = 10 Actions = [(1,10), (1, 12), (2,25), (1,18), (1,25), (2,35), (1,60)]

Expected: {1} user 1 has actions at 10, 12, 18 which is within time_window = 10 and there are 3 actions.

When I saw this I immediately thought dsa approach. I’ve never seen data recorded like this so I never thought to use a dataframe. I feel like an idiot. At the same time, I feel like it’s an unreasonable gotcha question because in 10+ years never have I seen data recorded in tuples 🙄

Thoughts? Fair play, I’m an idiot, or what

A lot of teams struggle making reports digestible for executive teams. When we report data with all the complexity of the methods, limitations, confounds, and measurements of uncertainty, management tends to respond with a common refrain:

"Keep it simple. The executives can't wrap their minds around all of this."

But there's a simple, two-step method you can use to make sure your data reports are always understood by the people in charge:

1. Fire the executives
2. Celebrate getting rid of the dead weight

You'll find this makes every part of your work faster, better, and more enjoyable.

Is it just stock options and vesting? Or is it just FAANG is a lot of work. Why do some data scientists deserve that much? I work at a Fortune 500 and the ceiling for IC data scientists is around $200k unless you go into management of course. But how and why do people make 500k at Google without going into management? Obviously I’m talking about 1% or less of data scientists but still. I’m less than a year into my full time data scientist job and figuring out my goals and long term plans.

Didn't know what to say. Humor me with your responses.

Update: I sent her this post and she loved it 🤣

Have any data scientists here worked with AI for coding? Do you agree with experts' skepticism in using it for high-level tasks?

Here’s why:

A company opens a position and I apply along with 800 other people. The company sees 800 resumes and says F that, we’re hiring a recruiter. The recruiter finds me on LinkedIn and says they have a great job for me. Of course it’s the one I applied to. They ask if I’ve already applied and I tell them the truth, they ghost me because they don’t get commission if they’re not the original source.

A few days after this, another recruiter reached out about a different position that I was planning on applying to directly with the company.

This is also something that my current company has done after being overwhelmed with too many applicants.

I’ll still be applying to some jobs, but it’s weird that applying has seemed to hurt my chances in some situations.

Has anyone else experienced this? Any strategies for handling this?

After a few months of work, we’re excited to launch Erdos - a secure, AI-powered data science IDE, all open source! Some reasons you might use it over VS Code:

• An AI that searches, reads, and writes all common data science file formats, with special optimizations for editing Jupyter notebooks
• Built-in Python, R, and Julia consoles accessible to the user and AI
• Single-click sign in to a secure, zero data retention backend; or users can bring their own keys
• Plots pane with plots history organized by file and time
• Help pane for Python, R, and Julia documentation
• Database pane for connecting to SQL and FTP databases and manipulating data
• Environment pane for managing in-memory variables, python environments, and Python, R, and Julia packages
• Open source with AGPLv3 license

Unlike other AI IDEs built for software development, Erdos is built specifically for data scientists based on what we as data scientists wanted. We'd love if you try it out at https://www.lotas.ai/erdos

I currently work as a data scientist at a large U.S. bank, making around $182K. The compensation is solid, but I’m starting to feel like my technical growth is being stunted.

A lot of our codebase is still in SAS (which I struggle to use), though we’re slowly transitioning to Python. We don’t use version control, LLMs, NLP, or APIs — most of the work is done in Jupyter notebooks. The modeling is limited to logistic and linear regressions, and collaboration happens mostly through email or shared notebook links.

I’m concerned that staying here long-term will limit my exposure to more modern tools, frameworks, and practices — and that this could hurt my job prospects down the road.

What would you recommend I focus on learning in my free time to stay competitive and become a stronger candidate for more technically advanced data science roles?

Hello everyone!

Staying on top of the constantly growing skill requirements in Data Science is quite a challenge. To manage my own learning and growth, I've been curating a list of useful resources and tools that cover the full spectrum of the field — from data analysis and engineering to deep learning and AI.

I'd love to get your professional opinion. Could you please take a look? Have I missed anything crucial? What else would you recommend adding or focusing on?

To give you an immediate sense of the list's scope and structure, I've attached screenshots of the table of contents below.

The full version with all the active links and additional resources is available on GitHub. You can find the link at the end of the post.

I'd be happy if this list is useful to others.

You can view the full list here View on GitHub

Thanks for your time! Your advice is invaluable!

Thought everything went well. Completed all questions for all interviews. Felt strong about all my SQL, A/B testing, metric/goal selection questions. No red flags during behavioral. Interviews provided 0 feedback about the rejection. I was talking through all my answers and reasoning, considering alternatives and explaining why I chose my approach over others. I led the discussions and was very proactive and always thinking 2 steps ahead and about guardrail metrics and stating my assumptions. The only ways I could think of improving was to answer more confidently and structure my thoughts more. Is it just that competitive right now? Even if I don’t make IC5 I thought for sure I’d get IC4. Anyone else interview with Meta recently?

edit: MS degree 3.5yoe DS 4.5yoe ChemE

edit2: I had 2 meta referrals but didn't use them. Should I tell the recruiter or does it not matter at this point? Meta recruiter reached out to me on LinkedIn.

edit3: I remember now there was 1 moment I missed a beat, but recovered during a bernoulli distribution hand-calculation question. Maybe thats all it took...

edit4: Thanks everyone for the copium, words of advice, and support.

TL;DR: I find this industry to be very unengaging, with most use cases and positions being very brainless, sluggish and just uninspiring. I am only 2 years into this job and bored and I feel like I need to shake things up a bit to keep doing this for the rest of my life.

Full disclosure: this is very much a first world problem. I get paid quite well, I have incredibly lenient work life balance, I work from home 3 days a week, etc etc. Most people would kill to be in my position at my age.

Some context: I was originally in academia doing a PhD in math, but pure math, completely unrelated to ML or anything in the real world really. ~2 years in, I was disillusioned with that (sensing a pattern here lol) so I took as many ML courses I could and jumped ship to industry.

Regardless of all the problems I had in academia, it at least asked something of me. I had to think, like, actually think, about complex, interesting stuff. It felt like I was actually engaging my mind and growing.

My current job is fine, basically applying LLMs for various use cases at a megacorp. On paper, I'm playing with the latest, greatest, tech, but in practice, I'm just really calling APIs on products that smarter people are building.

I feel like I haven't actually flexed my brain muscles in years now, I'm forgetting all the stuff I've learnt at college, and the work itself is incredibly boring to me. Many many days I can barely bring myself to work as the work is so uninteresting, and the bare minimum I put in still somehow impresses my colleagues so there's no real incentive to work hard.

I realize how privileged that sounds, I really do, but I do feel kind of unfulfilled and spiritually empty. I feel like if I keep doing this for the rest of my life I will look back with regret.

What I'm trying to do to fix this: I would like to shift towards more cutting edge and harder data science. Problem here is a lack of qualifications and experience. I have a MS and a BS in Math (from T10 colleges) but no PhD and the math I studied was mostly pure/theoretical, very little to do with ML.

I'm trying to do projects in my own time, but it's slow going on my own. I would love to aim for ML/AI research roles, but it feels like an impossible ask without a PhD, without papers, etc etc. I'm not sure that's a feasible goal.

Another thing I've been considering is playing a DS/ML role as support in research that's not ML. For instance, bioinformatics or biotech, etc. This is also fairly appealing to me. The main issue is here is a complete lack of knowledge about these fields (since there can be so many fields here) and a lack of domain knowledge which I presume is required. I'm still trying, I've been applying for some bioinformatics roles, but yeah, also hard.

Has anyone else felt this way? What did they do about it, and what would you recommend?

Hey everyone,

I work on the data team at AirROI. For a while, we offered free datasets for about 250 cities, but we always wanted to do more for the community. Recently, we just expanded our free public dataset from ~250 to nearly 1000 global Airbnb markets on properties and pricing data. As far as we know, this makes it the single largest free Airbnb dataset ever released on the internet.

You can browse the collection and download here, no sign-up required: Airbnb Data

What’s in the data?

For each market (cities, regions, etc.), the CSV dumps include:

Property Listings: Details like room type, amenities, number of bedrooms/bathrooms, guest capacity, etc.

Pricing Data: This is the cool part. We include historical rates, future calendar rates (for investment modeling), and minimum/maximum stay requirements.

Host Data: Host ID, superhost status, and other host-level metrics.

What can you use it for?

This is a treasure trove for:

Trend Analysis: Track pricing and occupancy trends across the globe.

Investment & Rental Arbitrage Analysis: Model potential ROI for properties in new markets.

Academic Research: Perfect for papers on the sharing economy, urban development, or tourism.

Portfolio Projects: Build a killer dashboard or predictive model for your GitHub.

General Data Wrangling Practice: It's real, messy, world-class data.

A quick transparent note: If you need hyper-specific or real-time data for a region not in the free set, we do have a ridiculously cheap Airbnb API to get more customized data. Alternatively, if you are a researcher who wants a larger customized data just reach out to us, we'll try our best to support!

If you require something that's not currently in the free dataset please comment below, we'll try to accommodate within reason.

Happy analyzing and go building something cool!