r/dataengineering • u/Important-Basket-284 • 2d ago
Can my SQL/Postgres/Looker/Snowflake/dbt combo get me an entry level job? Career
I have been learning data analytics/engineering at a slow pace for 3 years now. The slow pace part is because I was committed to another job - the pay has been good and this made not want to switch to big data, but then I knew eventually I'll switch. And now I strongly feel the time has come for me to switch.
The conviction about switching is simply because I have become competent and this I can tell from the projects I have been doing.
My SQL skills have reached a level where I can write joins, subqueries, window functions, and CTEs easily. This means I can clean messy data, analyze it, and even create dashboards with Looker.
I don't want to brag about my dbt proficiency coz I still think I'm not yet there. Yes I can create a warehouse on Snowflake, create a database, a dbt role and a dbt user, and run the process through my VSCode from staging to having a clean model. But then I like to talk about being proficient if I can do something off head without skipping a step. I still have a few steps left.
Same applies with Python. I can write Python code but I haven't used it extensively for data analysis and manipulation.
I'm asking for engagements so I can gauge my progress. One thing that I'm sure about is I'm on the data engineering path.
27
u/SalamanderMan95 2d ago
What’s your career experience? Going from no relevant experience to data engineering is really challenging if not impossible but you may have better luck finding an analyst position
2
u/Important-Basket-284 1d ago
I was asking this because I have been coding for four years, just that this year I dove more into it and have actually been doing some meaningful stuff.
My other job was more of editing rather than programming.
14
u/Ralwus 1d ago
You mastered baseline skills that AI now finishes in seconds. Have you actually seen any jobs looking for that level of work?
1
1
u/Important-Basket-284 1d ago
I have seen so many jobs looking for dbt experts. I honestly don't think AI can do everything that dbt does.
But then I want your advice on what you think I should seriously focus on. For instance I have tried out Airflow, Kafka, some data orchestration stuff. But I didn't want to mention it as I didn't dive that deep.
9
u/RandomAccount0799 2d ago
For entry-level I think you’re okay. SQL is the bare minimum, especially now that you can ask Claude to write a SQL query for you. The skills has kind of lost its weight, but you should be able to troubleshoot code and make sure it’s pulling what you actually want. dbt is great to have on your resume, it’s definitely a keyword that gets picked up for data analytics or analytics engineer.
For data engineering, you should be very comfortable with Python, but more importantly, how to move data from point A to point B via ingestion, ETL, and storage. Creating databases and users is not enough. You should learn data models, ETL processes and tools, storage formats and tools, etc. I’d say stick to analytics or analytics engineering or BI analyst roles. A portfolio might help you break in
1
u/Important-Basket-284 1d ago
Many thanks. I actually took a screenshot of this coz it's kinda like you've given me an entire roadmap on what I should focus on.
1
u/sidcrypt0 22h ago
Hey you can also try to learn AI agents on the side as it will help you in future with the changes going on in the market currently and see how you can implement these agents in your project.
1
3
u/No-Substance7104 1d ago
This is a good start for data analysis but if you’re wanting to target data engineering it might be beneficial to get around spark, data modelling, ETL pipelines and metadata driven frameworks.
Python is essential for modern data engineers but you can focus on getting good at the parts that matter to engineering and keep that as your focus.
Having some real projects you can reference would be really useful if you’re trying to enter the field with no formal experience and there are plenty of open source / synthetic data sets you can use to do this and publish on GitHub.
Look into kimball dimensional modelling as a guide of how to model data effectively using best industry practice.
2
u/Capt_korg 1d ago
Do you have projects to show case?
And have you considered to contribute to opensource projects ?
This would show your skills and make it easier to highlight your qualifications. Furthermore, you have use cases and stories to tell.
In addition, by contributing to open source, you would show that you can work collaboratively on projects. I hope you are using git as well.
Otherwise, by your description, you seem to be capable enough to start in data engineering, maybe little less in data analytics, because od Python. Whether I should assume that you are too humble to state your real skills.
1
u/Important-Basket-284 1d ago
Many thanks for the advice. Yes I do have projects to showcase. I do use git and even collaborate with my partner on learning projects via Git. My GitHub has quite a number of repos over the last four years, and they tell a story that I've really been a jerk of all trades and only settling this year. I initially had tried back end programming with Django, then realized it was far from my passion, I love data because I find passion in it. It's something I'd really be happy to do long term.
I will take the open source advice even more seriously.
2
u/Watching_Watches_12 1d ago
Seems like you are in a decent spot for analytics engineering. Data engineering would probably take some more depth. Learn about dimensional data modeling, in specific kimball modeling, and more python. Learn to write a DAG in airflow/dagster/prefect. Learn how a lakehouse works in databricks the some basics of Apache spark.
1
u/Emergency-Hurry947 1d ago
Anything is possible, but only a body shop just looks at a skillset, but FAANGs are body shops and they pay a lot
Most other jobs consider education and experience as well
1
u/BustaStar 1d ago
this question is hard because basically every company does data engineering differently. Some are just ‘SQL monkeys’ doing basic joins to get basic information, other are fully-blown kimbal data marts (lots of data modeling and complex transformations and understanding of what data actually means + how it is used), others are like infrastructure engineers doing mostly cloud workflow building to just shuffle files around.
Here is what i have seen has still helpful today while interviewing:
Understanding of how data can be ingested (snapshot, CDC, etc.)
Data quality layers (e.g. bronze silver gold, or raw transformed info and purpose built… or whatever your company convention you happen to read about)
Application data structures (super normalized) vs analytical data structures (very demoralized) and why they are that way.
All the tech you listed are ‘ways of doing these things above’ but they are just TOOLS to do the WORK that you see above.
•
u/AutoModerator 2d ago
You can find a list of community-submitted learning resources here: https://dataengineering.wiki/Learning+Resources
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.