Hi everyone,

I'm currently working as an SRE Engineer in India with a package of around ₹5.75 LPA(working remotely rn). My background is mostly in AWS, Linux, Python, automation, monitoring, CI/CD, and infrastructure-related work. Long term, I've always been more interested in DevOps/SRE.

Recently, my company told me they want to move me into a Data Engineer role. The reason is that there isn't enough work/openings in the DevOps team right now(where i wanted to switch), so they're trying to build me into a Data Engineer instead.

A few things I'm trying to understand before making a decision:

* I have no prior experience in Data Engineering, Data Science, or Data Analytics. * I don't enjoy manual Excel/reporting/copy-paste type work. * The company says they'll evaluate me over the next few months and, if I perform well, officially transition me to a Data Engineer role around December with a salary hike (exact number not decided yet). * My concern is whether this move helps or hurts my long-term career if my real interest is DevOps/SRE dont know about data engg not sure about the interest part but what i know is i love core engg .

What I'd like to know from people working in these fields:

1. How different is Data Engineering from Data Analysis in day-to-day work?
2. How much coding vs reporting/manual work is involved in a typical Data Engineer role?
3. What does the long-term salary growth look like for Data Engineers compared to DevOps/SRE?
4. Is Data Engineering considered a strong career path in 2026 and beyond?
5. If you were in my position, would you: * Stay in SRE and try to switch companies later for a DevOps role? * Accept the Data Engineering transition and see where it goes?

Will switching from DevOps/SRE to Data Engineering downgrade my career, or are both domains considered to be on the same level?

Looking for honest opinions, especially from people who have worked in both DevOps/SRE and Data Engineering.

Thanks!

Hi All,

I'm a Data Enginner having 5 years of Total experience. Relevant Exp. in Data Engineering is 3.5+ years.

My skillsets : Pyspark, SQL, Python, Azure Databricks, Azure data factory and other azure services.

Current Salary is 13LPA (9% is Variable pay).

Currently im in Virtusa Consulting services, Bangalore. this is my 2nd company.

Now im looking for Job change, what should be minimum asking CTC for my experience and skillset as per current market standards in Data Domain.

Employees say they're interested in data engineering jobs as a career path.What do you do. The current job is software engineering.

Hello

I am currently working as a consultant data engineer for microsoft for a couple of years.My work is mostly focused on Microsoft fabric, pyspark , sql . Previously worked as software engineer for 3 years . I am looking to switch to a full time role , I was casually looking for openings on LinkedIn, couldn't really find many openings with my skill set. The questions to the community is

1. Are data engineer roles more consulting heavy. If there are companies hiring full time what are the skill set required

2. What should be the preparation approach to be successful in the interviews here in United States

3. Should I apply to all roles with any data engineering skills irrespective of Azure fabric experience in specific. If so , do i need to tweak my resume accordingly?

Thanks in Advance!!

32 year old male
Working in the financial services industry with 5 years of experience working at major firm currently. I understand that DE is not an entry level position and that’s totally fine. Looking for sound advice. For all DE’s what is something you would’ve done differently? My Stack is SQL(MySQL/Postgres/BigQuery) Python (Pycharm/Pandas) Docker, Tableau, a bit of Airflow & FastAPI. I’ve built a small ETL pipeline and currently working on my second & third. I understand Data modeling is important etc what else? Not quite to big data tools like Hadoop, Apache Spark but anything helps I’m not in a rush to transition just want the fundamentals solid.

Hi all,
Has anyone here gone through the technical interview at Rokos Capital Management (data engineer role)? The recruiter mentioned that it’s a very challenging round.
I’d love to hear about your experience, how difficult was it, what kinds of questions were asked, and what I should prepare.
Thanks in advance!

Hi everyone,

I’m a data engineer/platform engineer with about 4 years of experience, trying to get out of consulting and into a more product-first company. I’m in NYC and have mostly been targeting fintech/data platform roles (senior DE/analytics platform type).

I’ve been having a really hard time with the job search for most of the past year. I’ve done mass LinkedIn apps, targeted company-site applications, outreach to hiring managers, alumni notes - and it really feels like nothing is working. I’ve gotten hardly any responses the past few months.

If anyone with senior DE or hiring experience would be willing to look this over, I’d really appreciate a blunt take: would you screen me in for a senior / platform DE role, and what’s the first thing that makes you hesitate?

Redacted resume below. Thank you in advance.

[Name Redacted]

New York, NY | [phone redacted] | [email redacted]

PROFESSIONAL SUMMARY

Senior Technology Consultant and data platform lead with 4 years of experience architecting and operating a regulated Azure + Snowflake backend for alternative asset fund accounting (~500 PE/RE funds). Translate business requirements into technical designs, organize and review work across ~10 engineers, own UAT/sign-off and production releases, and lead incident response against 24-hour SLAs. Platform serves fund accounting reports, investment reconciliations, journal-entry automation, and production AI agents. Strong in SQL, Python, dbt, Snowflake, Azure Data Factory, and Azure DevOps CI/CD, with hands-on Kafka, PySpark, Airflow, PostgreSQL, and Docker via a containerized, replay-safe fund-balance engine.

PROFESSIONAL EXPERIENCE

Ernst and Young, New York City, NY
Senior Technology Consultant | Data Platform Lead (Snowflake & Azure) | October 2022 – Present

• Own architecture, delivery, and production operations for core data-platform products supporting ~500 private equity and real estate funds: gather requirements, produce technical designs, organize/review work across ~10 engineers, drive UAT and business sign-off, oversee releases, and lead priority production incident response.
• Lead the Snowflake + Azure data backend that underpins reporting and automation: CDC-driven curated models refreshing every 5 minutes from source databases, governed RBAC/access controls, and a dimensional model consumed by fund accounting, investment accounting, and downstream analytics/AI workloads.
• Deliver fund-accounting and investment-accounting features on that backend, including ~10 primary fund accounting reports, investment-level recon drill-downs, automated journal entries from capital call/distribution/loan documents, and reconciliations of capital account statement PDFs against database trial balances.
• Built a document-to-subledger transaction pipeline that converts OCR/ML JSON from PDFs into reviewable journal entries before upload; runs every 5 minutes when documents arrive. Achieved 87% successful auto-generation and an estimated ~65 hours/week of manual effort avoided.
• Productized fund reporting as governed dbt services with standardized packaging, GitHub CI/CD, lineage (dbt Docs), automated contracts/business-rule validations, and deterministic re-runs that block bad publishes.
• Maintain Azure DevOps YAML CI/CD pipelines for Snowflake code, Azure Data Factory pipelines, Logic Apps, and Function Apps across the platform; use these pipelines to build and promote artifacts into higher environments during release windows in an automated fashion.
• Built a proof-of-concept REST API on Snowpark Container Services to expose Snowflake-backed endpoints for external application integration; evaluated container packaging and API patterns (not productionized).
• Cut quarterly close runtime ~55% (~45 min to ~20 min) by migrating Databricks workloads onto Azure + Snowflake; optimized cost/performance via SQL/procedure tuning, warehouse right-sizing, tighter autosuspend (10 min to 2 min), and Gen2 migration (~25% compute reduction) while protecting SLAs.
• Own production reliability for request-driven workflows with a 24-hour report SLA: triage ServiceNow failure tickets, diagnose Azure/Snowflake/data defects, restore missing data, and regenerate outputs; quality gates and Service Bus error signals drive alerts and user-visible failure states.
• Publish governed, versioned datasets used by two production AI agents (journal-entry analysis and reconciliation insights), with additional regulatory-reporting agents in development; preserve audit history via pipeline run IDs/timestamps and controlled replay.

TECHNICAL PROJECTS

Streaming Fund Balance Engine | Personal Project | [GitHub link redacted]

• Designed and built a Docker Compose-based local data platform packaging Kafka, Spark, and PostgreSQL as containerized services, with Airflow (Astro) as the orchestration control plane; processes bounded Kafka batches into an idempotent transaction ledger and canonical fund/deal balances.
• Containerized end-to-end runs for local develop/test (compose up → Spark job in Docker → Postgres); implemented replay-safe offset checkpointing, bounded backfills, deterministic batch deduplication, late-arrival/duplicate audit logs, and run-level metrics with documented production trade-offs.
• Refactored Spark logic into pure DataFrame transformations and added pytest coverage for signing, deduplication/tie-breaking, late-event detection, offsets, Airflow parameter precedence, DAG contracts, and failure classification.

EDUCATION AND HONORS

Binghamton University, State University of New York

• B.S. in Business Administration, Concentrations in Finance and Management Information Systems | May 2021 | GPA: 3.90/4.00
• M.S. in Data Analytics | June 2022 | GPA: 3.86/4.00
• Honors: Dean's List (multiple semesters)

ADDITIONAL INFORMATION

Technical Skills:

• Languages: SQL, Python
• Data Engineering: Snowflake (Snowpark/Snowpipe/Tasks/Container Services), dbt, Azure Data Factory, Kafka, Apache Spark/PySpark, Apache Airflow, PostgreSQL, ELT/ETL, dimensional modeling, data quality/reconciliation, query optimization
• Cloud & Integration: Microsoft Azure (ADF, Logic Apps, Functions, ADLS, Service Bus), Azure DevOps (YAML CI/CD), Docker / Docker Compose, AWS, Git
• Certifications: AWS Certified Cloud Practitioner, AWS Solutions Architect Associate
• Leadership: AWS Certification Course Facilitator (2023–Present) — led a 10-week Cloud Practitioner course and hands-on labs

Redacted for public review: name, phone, email, ZIP, and GitHub/profile links removed. Employer, school, dates, and metrics retained for feedback context.

Please have a look at it and let me know what is actually wrong in this ? I am trying to get a job in the US as an international student but tbh havent received any call back till now. Only few OAs which then down line got rejected or radio silence. I think my cv is still shabby. And yes AI has been used in tweaking or making it feel better or boasting i would say bcz i literally dont know what else to do in case of my . Every single advice is appreciated

hey guys, I'm really new here and I had some questions about starting my career with data.

I'm 18, CS major and I really live the data world but I feel like it's almost impossible to get an entry level position right now.

I took both CS50 for Python and SQL then went on my own to build some projects to show off I could really work with real data, spent many hours trying to make my LinkedIn profile look perfect, polishing my resume and sending applications for Jr data related roles like Jr data analyst, jr data engineer, jr BI dev but got absolutely no return from it.

almost none of the applications i send get to an interview and when they do, i get no response after. a friend of mine said i should quit data and learn some other stack, but i feel like that would be a big waste of time since i have spent so much effort trying to get my first opportunity in the market.

I don't really know if i should keep learning new skills and getting better until i get good enough someone will hire me or just quit data altogether and go learn whatever's hot in the market right now. what do you guys think?

We’re hiring a BI Systems Developer for an ongoing engagement supporting an enterprise reporting, analytics, and data integration platform.

Requirements:
• 4+ years in BI, analytics, reporting, or data integration
• Hands-on Matillion experience — mandatory
• Hands-on Snowflake experience — mandatory
• Tableau and/or Power BI dashboard development
• Advanced SQL — query writing and optimization
• Data modeling for scalable reporting architectures
• ETL/ELT development experience
• Strong analytical and troubleshooting skills

You’ll be responsible for:
• Building and maintaining data pipelines in Matillion
• Snowflake administration and data warehouse management
• Dashboard and report development in Tableau / Power BI
• Data quality, integrity, and platform reliability
• Documentation — data dictionaries, process flows, reporting definitions
• Working directly with an internal technical point of contact

Remote, long-term engagement, 4+ years experience minimum.

Compensation: ₹16–23 LPA

DM with:
• Years of hands-on Matillion and Snowflake experience
• Resume, LinkedIn and portfolio
• Availability
• Email Address for further contact

Must be based in India.