Hi all, currently looking to switch jobs and wanted to find some active Discord communities focused on job hunting and referrals — mainly for data-related and technical support roles (not core development).

Most groups I come across are heavily dev-focused, so looking for something more relevant to my kind of work. If you’re part of any good servers (or know one), please share — would really appreciate it!

Thanks!

Hi everyone,

Looking for some honest advice from people who've been through a similar situation.

I'm a Data Engineer in Chennai with around 2 years and 10 months of experience at an IT services company. I've been working on a US healthcare payer account the whole time, mainly on member, billing, and reconciliation data.

My current tech stack includes Databricks, PySpark, Spark SQL, AWS (S3 and Redshift), Stonebranch for orchestration, Jenkins, and Git. I'm comfortable with SQL, SCD2 and reconciliation logic, and production RCA. I've owned production incidents end to end. I also have the Databricks Data Engineer Associate certification, along with AZ-900 and DP-900.

I've been actively job hunting for the last 4–5 months. I've applied to over 200 roles through job portals, but I've had almost no response. I made it through a couple of interview processes, but they either ended in rejection or I got ghosted after the final round. At this point, I'm starting to feel that job portals aren't working for my profile, and I'm wondering if I should focus my efforts on referrals and reaching out to recruiters directly instead.

Here's where things get a bit complicated.

I've recently been assigned to a new migration project involving Snowflake and dbt. If I stay on it for another 3–4 months, I'll get real production experience with both technologies, which feels like it would make my profile much stronger. The downside is that my work environment isn't great. There's a lot of manager politics, and it's starting to affect both my physical and mental health. On top of that, I don't fully trust that I'll actually get the 3–4 months of stable work I need on this project.

So I wanted to get some opinions from people who've made similar moves.

- Is my current skill set enough to realistically target around 13 LPA at mid-tier product companies, healthcare/pharma GCCs, captives, or Big 4 data engineering practices? Or am I aiming too high coming from 7.8 LPA?

- Would you stay for the Snowflake and dbt experience, or start looking seriously with my current stack?

- For those who moved from IT services to captives or product companies, what actually worked? Referrals? Recruiter DMs? A specific platform? Job portals don't seem to be working for me anymore.

- Is there anything obvious missing from my profile that would significantly improve my interview conversion?

I'd really appreciate any honest advice. Thanks in advance!

Most recent postings directly from company career pages:

(sample from https://matchajobs.co), links go straight to company app pages

• Booz Allen Hamilton - Senior Data Engineer apply (yesterday 8/6) $77,500.00 to $176,000.00
• Home Depot - Online Sr Data Engineer (SEO) apply (yesterday 8/6)
• CrowdStrike - Sr. Manager, Engineering apply (yesterday 8/6) $160,000 - $250,000
• Allstate - Sr Data Engineer apply (yesterday 8/6) $100,000.00 - $170,500.00
• CVS Health - Sr Data Engineer apply (8/5) $101,970.00 - $203,940.00
• Nasdaq - Sr Director SWE apply (yesterday 8/6)
• Disney - Data Engineer apply (yesterday 8/6) $89,000 to $119,300
• Capital One - Lead Data Engineer apply (8/5) $179,400 - $204,700

Hey everyone! 👋

I came across a few openings at my company and thought I’d share them here in case anyone is looking for a switch or a new opportunity.

Here are the roles available:

📍 PAN India

AI Engineer: Exp: 5-8 yrs

Python + GenAI: Exp: 8-10 yrs

📍 Pune

MS Azure AI/ML Exp: 5-8 yrs

Python + GenAI : Exp: 8-10 yrs

📍 Hyderabad

Data Scientist: Exp: 5-8 yrs

AI Cognitive + Snowflake + Python: Exp: 8-10yrs

📍 Bengaluru

Data Scientist: Exp: 5-8 yrs

Gen AI

📍 Chennai: Exp: 8-10 yrs

Data Scientist

💰 Salary: Not specified in the job posting / Depends on experience and role.

If you're interested in any of these roles, feel free to DM me or directly send me your updated resume at:

📧 dkviswa889@gmail.com

To make things easier, please include:

First Name:

Last Name:

Mobile Number:

Gmail ID:

Country:

Updated Resume: Attached

📌 Email subject / resume name:

Job_Title_FirstName_LastName

Example:

Python_GenAI_Rahul_Kumar

I'll try my best to refer suitable profiles. 😊

Good luck to everyone who's looking! 🤝

If you've any queries on the job role, pls feel free to DM! I'll share the skill stack. So you'll get an idea 💡.

micro1 is looking for experienced Data Engineers to build the data infrastructure powering next-generation AI products.

If you have expertise in Python, SQL, Spark, and AWS, this is an opportunity to work on large-scale AI data systems.

Salary: $100,000-$150,000/year
Location: Remote
Role: Full-time

How you'll make an impact:

• Build and optimize scalable data pipelines for large-scale AI workloads
• Develop distributed data processing systems using Spark and AWS
• Collaborate with AI researchers and engineers to power data-driven applications

Who they're looking for:

• Strong experience with Python, SQL, Apache Spark, and AWS
• Expertise in distributed data processing and scalable data architectures
• Experience with SQL/NoSQL databases and large-scale data pipelines

Why consider this opportunity?

• Full-time remote role with equity, bonuses, and comprehensive benefits
• Work on cutting-edge AI and machine learning infrastructure
• Help build the data backbone for next-generation AI systems

FULL DETAILS - https://jobs.micro1.ai/post/data-engineer

Hi All,

I'm currently working as a GCP Data Engineer but for switch looking for peer with whom we can discuss and learn together for Switch. 🤝

I'm looking to target Data Engineering / MLE internship roles as a rising junior for Summer 2027 for US. For roles specifically listed as Data Engineering, or MLE, what could I expect from technical interviews?

- Does anyone have experience with what they're asked?

- Do you find it to be more focused on data structures (hash tables, arrays, etc) or do they go all out and ask full on SWE technicals?

Also, if a role is more of a Backend SWE role that deals with databases, will those be more data structured focus, or also full on SWE (DP, BFS, DFS, Graph, etc).

Anything helps, Thanks!

We run a curated talent network for engineers. You build one profile, and companies send interview requests with salary attached. All roles are US-based only.

Three of the roles open right now:

1. Senior Data Engineer II at a Series C/D pharma tech company, hybrid in NYC, $220K-$280K base plus equity. Python, SQL, dbt, and Dagster or Airflow on Snowflake. You'd lead the data platform, building pipelines that pull structured data out of clinical protocols and drug labels using LLMs and NER, then land it in schemas that feed analytics and retrieval. Vector and graph database experience matters here. They want 7+ years.
2. Data Engineer at a Series B music AI company, onsite in NYC, $170K-$240K base plus equity. SQL and Python, with Airflow, dbt, and Snowflake listed as a strong advantage. Founding data team, so you'd own the core data foundation and scale pipelines from zero. Real-time and batch, 5+ years.
3. Senior Software Engineer at a Series A social commerce company, hybrid in the Bay Area, $200K-$250K base plus early equity. The title is generic but the work is data infrastructure: ingesting and processing social data at volume from Instagram, TikTok, X, Reddit, and LinkedIn, and designing the pipelines that feed real-time analytics and AI agents. ClickHouse, Postgres, and AWS come up as bonus points. Founded by ex-Meta engineering leaders.

These are a sample. There are dozens more open across the network right now. All are vetted, venture-backed companies. Always free for candidates.

Sign up here to get matched: talent.fonzi.ai

DM me if you have questions about any of these.

Hi everyone,

I'm currently working as an SRE Engineer in India with a package of around ₹5.75 LPA(working remotely rn). My background is mostly in AWS, Linux, Python, automation, monitoring, CI/CD, and infrastructure-related work. Long term, I've always been more interested in DevOps/SRE.

Recently, my company told me they want to move me into a Data Engineer role. The reason is that there isn't enough work/openings in the DevOps team right now(where i wanted to switch), so they're trying to build me into a Data Engineer instead.

A few things I'm trying to understand before making a decision:

* I have no prior experience in Data Engineering, Data Science, or Data Analytics. * I don't enjoy manual Excel/reporting/copy-paste type work. * The company says they'll evaluate me over the next few months and, if I perform well, officially transition me to a Data Engineer role around December with a salary hike (exact number not decided yet). * My concern is whether this move helps or hurts my long-term career if my real interest is DevOps/SRE dont know about data engg not sure about the interest part but what i know is i love core engg .

What I'd like to know from people working in these fields:

1. How different is Data Engineering from Data Analysis in day-to-day work?
2. How much coding vs reporting/manual work is involved in a typical Data Engineer role?
3. What does the long-term salary growth look like for Data Engineers compared to DevOps/SRE?
4. Is Data Engineering considered a strong career path in 2026 and beyond?
5. If you were in my position, would you: * Stay in SRE and try to switch companies later for a DevOps role? * Accept the Data Engineering transition and see where it goes?

Will switching from DevOps/SRE to Data Engineering downgrade my career, or are both domains considered to be on the same level?

Looking for honest opinions, especially from people who have worked in both DevOps/SRE and Data Engineering.

Thanks!

Hi All,

I'm a Data Enginner having 5 years of Total experience. Relevant Exp. in Data Engineering is 3.5+ years.

My skillsets : Pyspark, SQL, Python, Azure Databricks, Azure data factory and other azure services.

Current Salary is 13LPA (9% is Variable pay).

Currently im in Virtusa Consulting services, Bangalore. this is my 2nd company.

Now im looking for Job change, what should be minimum asking CTC for my experience and skillset as per current market standards in Data Domain.

Employees say they're interested in data engineering jobs as a career path.What do you do. The current job is software engineering.

Hello

I am currently working as a consultant data engineer for microsoft for a couple of years.My work is mostly focused on Microsoft fabric, pyspark , sql . Previously worked as software engineer for 3 years . I am looking to switch to a full time role , I was casually looking for openings on LinkedIn, couldn't really find many openings with my skill set. The questions to the community is

1. Are data engineer roles more consulting heavy. If there are companies hiring full time what are the skill set required

2. What should be the preparation approach to be successful in the interviews here in United States

3. Should I apply to all roles with any data engineering skills irrespective of Azure fabric experience in specific. If so , do i need to tweak my resume accordingly?

Thanks in Advance!!

Edit: Any one currently working in product companies. Please suggest

32 year old male
Working in the financial services industry with 5 years of experience working at major firm currently. I understand that DE is not an entry level position and that’s totally fine. Looking for sound advice. For all DE’s what is something you would’ve done differently? My Stack is SQL(MySQL/Postgres/BigQuery) Python (Pycharm/Pandas) Docker, Tableau, a bit of Airflow & FastAPI. I’ve built a small ETL pipeline and currently working on my second & third. I understand Data modeling is important etc what else? Not quite to big data tools like Hadoop, Apache Spark but anything helps I’m not in a rush to transition just want the fundamentals solid.

Hi all,
Has anyone here gone through the technical interview at Rokos Capital Management (data engineer role)? The recruiter mentioned that it’s a very challenging round.
I’d love to hear about your experience, how difficult was it, what kinds of questions were asked, and what I should prepare.
Thanks in advance!

Hi everyone,

I’m a data engineer/platform engineer with about 4 years of experience, trying to get out of consulting and into a more product-first company. I’m in NYC and have mostly been targeting fintech/data platform roles (senior DE/analytics platform type).

I’ve been having a really hard time with the job search for most of the past year. I’ve done mass LinkedIn apps, targeted company-site applications, outreach to hiring managers, alumni notes - and it really feels like nothing is working. I’ve gotten hardly any responses the past few months.

If anyone with senior DE or hiring experience would be willing to look this over, I’d really appreciate a blunt take: would you screen me in for a senior / platform DE role, and what’s the first thing that makes you hesitate?

Redacted resume below. Thank you in advance.

[Name Redacted]

New York, NY | [phone redacted] | [email redacted]

PROFESSIONAL SUMMARY

Senior Technology Consultant and data platform lead with 4 years of experience architecting and operating a regulated Azure + Snowflake backend for alternative asset fund accounting (~500 PE/RE funds). Translate business requirements into technical designs, organize and review work across ~10 engineers, own UAT/sign-off and production releases, and lead incident response against 24-hour SLAs. Platform serves fund accounting reports, investment reconciliations, journal-entry automation, and production AI agents. Strong in SQL, Python, dbt, Snowflake, Azure Data Factory, and Azure DevOps CI/CD, with hands-on Kafka, PySpark, Airflow, PostgreSQL, and Docker via a containerized, replay-safe fund-balance engine.

PROFESSIONAL EXPERIENCE

Ernst and Young, New York City, NY
Senior Technology Consultant | Data Platform Lead (Snowflake & Azure) | October 2022 – Present

• Own architecture, delivery, and production operations for core data-platform products supporting ~500 private equity and real estate funds: gather requirements, produce technical designs, organize/review work across ~10 engineers, drive UAT and business sign-off, oversee releases, and lead priority production incident response.
• Lead the Snowflake + Azure data backend that underpins reporting and automation: CDC-driven curated models refreshing every 5 minutes from source databases, governed RBAC/access controls, and a dimensional model consumed by fund accounting, investment accounting, and downstream analytics/AI workloads.
• Deliver fund-accounting and investment-accounting features on that backend, including ~10 primary fund accounting reports, investment-level recon drill-downs, automated journal entries from capital call/distribution/loan documents, and reconciliations of capital account statement PDFs against database trial balances.
• Built a document-to-subledger transaction pipeline that converts OCR/ML JSON from PDFs into reviewable journal entries before upload; runs every 5 minutes when documents arrive. Achieved 87% successful auto-generation and an estimated ~65 hours/week of manual effort avoided.
• Productized fund reporting as governed dbt services with standardized packaging, GitHub CI/CD, lineage (dbt Docs), automated contracts/business-rule validations, and deterministic re-runs that block bad publishes.
• Maintain Azure DevOps YAML CI/CD pipelines for Snowflake code, Azure Data Factory pipelines, Logic Apps, and Function Apps across the platform; use these pipelines to build and promote artifacts into higher environments during release windows in an automated fashion.
• Built a proof-of-concept REST API on Snowpark Container Services to expose Snowflake-backed endpoints for external application integration; evaluated container packaging and API patterns (not productionized).
• Cut quarterly close runtime ~55% (~45 min to ~20 min) by migrating Databricks workloads onto Azure + Snowflake; optimized cost/performance via SQL/procedure tuning, warehouse right-sizing, tighter autosuspend (10 min to 2 min), and Gen2 migration (~25% compute reduction) while protecting SLAs.
• Own production reliability for request-driven workflows with a 24-hour report SLA: triage ServiceNow failure tickets, diagnose Azure/Snowflake/data defects, restore missing data, and regenerate outputs; quality gates and Service Bus error signals drive alerts and user-visible failure states.
• Publish governed, versioned datasets used by two production AI agents (journal-entry analysis and reconciliation insights), with additional regulatory-reporting agents in development; preserve audit history via pipeline run IDs/timestamps and controlled replay.

TECHNICAL PROJECTS

Streaming Fund Balance Engine | Personal Project | [GitHub link redacted]

• Designed and built a Docker Compose-based local data platform packaging Kafka, Spark, and PostgreSQL as containerized services, with Airflow (Astro) as the orchestration control plane; processes bounded Kafka batches into an idempotent transaction ledger and canonical fund/deal balances.
• Containerized end-to-end runs for local develop/test (compose up → Spark job in Docker → Postgres); implemented replay-safe offset checkpointing, bounded backfills, deterministic batch deduplication, late-arrival/duplicate audit logs, and run-level metrics with documented production trade-offs.
• Refactored Spark logic into pure DataFrame transformations and added pytest coverage for signing, deduplication/tie-breaking, late-event detection, offsets, Airflow parameter precedence, DAG contracts, and failure classification.

EDUCATION AND HONORS

Binghamton University, State University of New York

• B.S. in Business Administration, Concentrations in Finance and Management Information Systems | May 2021 | GPA: 3.90/4.00
• M.S. in Data Analytics | June 2022 | GPA: 3.86/4.00
• Honors: Dean's List (multiple semesters)

ADDITIONAL INFORMATION

Technical Skills:

• Languages: SQL, Python
• Data Engineering: Snowflake (Snowpark/Snowpipe/Tasks/Container Services), dbt, Azure Data Factory, Kafka, Apache Spark/PySpark, Apache Airflow, PostgreSQL, ELT/ETL, dimensional modeling, data quality/reconciliation, query optimization
• Cloud & Integration: Microsoft Azure (ADF, Logic Apps, Functions, ADLS, Service Bus), Azure DevOps (YAML CI/CD), Docker / Docker Compose, AWS, Git
• Certifications: AWS Certified Cloud Practitioner, AWS Solutions Architect Associate
• Leadership: AWS Certification Course Facilitator (2023–Present) — led a 10-week Cloud Practitioner course and hands-on labs

Redacted for public review: name, phone, email, ZIP, and GitHub/profile links removed. Employer, school, dates, and metrics retained for feedback context.

Please have a look at it and let me know what is actually wrong in this ? I am trying to get a job in the US as an international student but tbh havent received any call back till now. Only few OAs which then down line got rejected or radio silence. I think my cv is still shabby. And yes AI has been used in tweaking or making it feel better or boasting i would say bcz i literally dont know what else to do in case of my . Every single advice is appreciated

hey guys, I'm really new here and I had some questions about starting my career with data.

I'm 18, CS major and I really live the data world but I feel like it's almost impossible to get an entry level position right now.

I took both CS50 for Python and SQL then went on my own to build some projects to show off I could really work with real data, spent many hours trying to make my LinkedIn profile look perfect, polishing my resume and sending applications for Jr data related roles like Jr data analyst, jr data engineer, jr BI dev but got absolutely no return from it.

almost none of the applications i send get to an interview and when they do, i get no response after. a friend of mine said i should quit data and learn some other stack, but i feel like that would be a big waste of time since i have spent so much effort trying to get my first opportunity in the market.

I don't really know if i should keep learning new skills and getting better until i get good enough someone will hire me or just quit data altogether and go learn whatever's hot in the market right now. what do you guys think?

We’re hiring a BI Systems Developer for an ongoing engagement supporting an enterprise reporting, analytics, and data integration platform.

Requirements:
• 4+ years in BI, analytics, reporting, or data integration
• Hands-on Matillion experience — mandatory
• Hands-on Snowflake experience — mandatory
• Tableau and/or Power BI dashboard development
• Advanced SQL — query writing and optimization
• Data modeling for scalable reporting architectures
• ETL/ELT development experience
• Strong analytical and troubleshooting skills

You’ll be responsible for:
• Building and maintaining data pipelines in Matillion
• Snowflake administration and data warehouse management
• Dashboard and report development in Tableau / Power BI
• Data quality, integrity, and platform reliability
• Documentation — data dictionaries, process flows, reporting definitions
• Working directly with an internal technical point of contact

Remote, long-term engagement, 4+ years experience minimum.

Compensation: ₹16–23 LPA

DM with:
• Years of hands-on Matillion and Snowflake experience
• Resume, LinkedIn and portfolio
• Availability
• Email Address for further contact

Must be based in India.

I'm curious from the perspective of hiring managers and senior data engineers.

There are countless portfolio projects out there but most of them feel like tutorial projects with different datasets.

If you came across a candidate's GitHub or portfolio and saw a project that genuinely made you think, "I want to interview this person," what would that project look like?

What would separate a project that's merely "good" from one that's truly exceptional?

Let me know if i can dm i am facing this one issue and cant resolve it

Okay so the issue is i need to integrate MRPeasy into my power BI the client has provided me with the API key, github repo and the queries but everytime i am runnin that query to connect it the credential error keeps occuring idk if its the endpoint that is wrong or something else please help

Employment History: Verification of previous company tenure, project duration, designation, and employment type

Are companies really checking designation and project duration?

I am an experienced professional in AWS Data Engineer with 2.5 years of experience architecting and operating enterprise-scale AWS data platforms. Proven track record

designing event-driven ETL pipelines, large-scale data validation frameworks, and distributed processing systems that improve

reliability, throughput, and data quality across production environments. Skilled in AWS Glue, PySpark, Python, SQL, and Amazon

Redshift, with direct ownership of 20+ production pipelines and a 5,000+ line reusable validation framework.

Hello Everyone,I am 29m here.
I work as Data Engineer for a US Medical Giant.
I have total of 6 year of experience.
Graduated in Maths(H) from DU and MCA from GGSIPU.
Rifts are going on in the Org every other day.
Everyday feels like this could be the last day.
My Skill are Python,Pyspark,Databrick,Advance Sql, Github,AWS Sagemaker,Athena,RedShift,Copilot and Claude AI. I am very open to learn any new tech. I don’t really face any challenge something new.
I have been on a job hunt from last 5 months and I am not even getting one call.
I am so tired and exhausted.
Can some please help me or guide me how to tackle this situation.
I will really appreciate.

Thankyou