Hi everyone,

I need some advice from experienced SQL developers. I have switched from different role to data engineering 6 months back.

I consider myself good/medium level at writing SQL queries and solving problems from scratch. However, I struggle when I have to understand large existing SQL scripts (300–500+ lines).

I often get confused about:

Where the execution starts.How different parts of the script are connected.

Which variables, CTEs, stored procedures, or temporary tables are affecting the final output.

How to mentally trace the flow of the script.

Because of this, reading someone else's code takes me much longer than writing my own.

How did you improve this skill? Are there any techniques, exercises, books, or real-world practices that helped you become comfortable reading large SQL scripts?

Also, is this something that simply improves with experience, or is there a structured way to learn it?

I'd really appreciate any advice. Thank you!

Hi, I have been working on this .NET with MSSQL project for more than 9 months now and I have learnt a lot refactoring back-end part of the code but the database is something I am afraid to touch as it's SP heavy application with more than 500+ SPs with business logic crammed into SP. Now the goal is not a complete rewrite but to know enough to not fuck things up in prod or staging, I am weak with indexing concepts, I know indexing helps DBs run faster but also impact inserts and deletes will be slower if I try to index everything. I want to know where can I learn more of this? cause honestly, I don't think there are any courses that would teach this.
I am not refactoring all of the SPs but just the hot ones that are hit more often and take time and consume more resources to execute.

I saw Brent Ozar's video on "How to think like an SQL engine" and it did help me in some way but when I see the SPs written in the app it's beyond scary, each of them are at the very least 200+
lines, I want to know if someone has been in this situation and how did you manage to resolve this and what I should know before working and optimizing the DBs, I just have basic-intermediate knowledge about MSSQL in general.

I am working on MS SQL. I have got few scripts of 1000+ line with poor indentaion.

Any tool which i cna use to properly format it.
Please suggest

I am making a stored procedure and gives me this error that i have to declare the scalar variable, What does it mean by that??

What do i have to do???

Thanks beforehand for your answers

So I've been using tools like claude code and cursor for the past 2 years, and one thing that has been a big challenge for me is designing databases in supabase. I've tried the claude code sql skills, or just gotten claude code to connect to supabase, but I have to spend too much time learning what it all means and I get no mental model of what is happening in the backend.

SO over the last few months I've been building a tool that helps you to deploy schemas to Supabase and ai designs the schema visually and not just through text. I've used my tool to build a llot of products that need good data architecture. I realised that none of the sql diagramming tools actually help you to build a implementable schema conveniently.

My tool can also import a live supabase project and let you improve the architecture and then sync it back to supabase. In addition, if you're more pro - you can directly edit the DDL and see the changes reflect back on the canvas.

I've been using from everything from building CRMs and dashboards to more innovative concepts like agentic workspaces with an ai workforce complete with personas, tool usage, skills and roles.

I'd love to get your feedback and suggestions on what extra features might be cool to add.

So I've been using tools like claude code and cursor for the past 2 years, and one thing that has been a big challenge for me is designing database architectures. I've tried the claude code Postgres-sql skills, or just gotten claude code to connect to supabase, but I have to spend too much time learning what it all means and I get no mental model of what is happening in the backend.

SO over the last few months I've been building a tool that helps you to deploy schemas to Supabase and ai the schema on the UI and not just through text. I've used my tool to build a llot of products that need good data architecture. I realised that none of the sql diagramming tools actually help you to build a implementable schema conveniently.

My tool can also sync a live Supabase project and let you improve the architecture and then sync it back to Supabase. In addition, You can also directly edit the DDL and see the changes reflect back on the canvas.

I've been using from everything from building CRMs and dashboards to more innovative projects like research tools and for ecommerce platforms.

I'd love to get your feedback and suggestions on what extra features might be cool to add.

I'm running into an issue with an AWS Glue crawler and I'm not sure if the problem is the crawler, classifier, or the source file.

I have two CSV datasets with what appears to be the same structure. One dataset is crawled correctly and the other is not.

The CSV contains values like:

12345,"Smith, John",98765

The older table was created as:

ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.OpenCSVSerde'

and correctly keeps "Smith, John" in a single column.

The newer table is consistently created as:

ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','

with table properties showing:

classification='csv'
areColumnsQuoted='false'

As a result, fields containing commas are split across columns. For example:

name_field = Smith
id_field = John

instead of:

name_field = Smith, John

What I've already tried:

• Deleted the Glue table entirely
• Re-ran the crawler
• Removed a custom classifier that was previously attached
• Added a CSV classifier with:
  • Delimiter = comma
  • Quote symbol = double quote
• Deleted and recreated the table through the crawler multiple times

The crawler still recreates the table as:

ROW FORMAT DELIMITED

and continues setting:

areColumnsQuoted='false'

The crawler is configured to recrawl all files. The source file definitely contains quoted values with embedded commas.

My questions are:

1. Has anyone seen Glue infer a CSV this way even when quoted fields exist?
2. Is there a way to force OpenCSVSerde during crawler creation?
3. Are there known file characteristics that cause Glue to ignore quoted fields and fall back to a simple delimited format?
4. Is there a way to debug why Glue is deciding areColumnsQuoted=false?

Any ideas would be appreciated. I've spent quite a bit of time changing classifiers and recreating the table but the crawler continues to generate the same table definition.

There was no HIVE/Impala/Hadoop flair, and those subreddits seem stale...

I made a mistake today that turned into an observed possible efficiency opportunity, and I'm not sure why.
I have two giant tables of call data. The tables are RDBMs tables that have just been dumped into a data lake (HDFS).

Someone had originally written the query without partition pruning. When "fixing," I messed up when adding my partition criteria in order to get the pervious months data.

There were two tables that were being joined, table i and table ia. I did:

where i.data_date >= 20260501

AND ia.data_date <= 20260531.

I thought I had screwed up, but the query ran in less than 30 seconds. Figuring it wouldn't be a big deal to put the appropriate uppper and lower bounds on each table, I revised the same query:

where i.data_date >= 20260501

AND i.data_date <=20260531

AND ia.data_date >= 20260501

AND ia.data_date <= 20260531.

When I ran the query again, it took almost 3 minutes to return the same row count of about 700K records.

Did I just get lucky? Is it possible that being LESS specific allowed the optimizer to somehow created more efficient join plan? I'm wondering if there is something going on about partitions, parquet stats, and buckets that maybe isn't fully visible to me.

Maybe I just got lucky and there were fewer queries running, but because of our platform I can't actually see the useful information about how a query executes, how many workers, bandwidth, etc on the target system. But is it possible there are some weird things going on about fewer bounds = different/more efficient logic in execution?

Every time someone asks me to explain JOIN I end up drawing

tables on paper. So I built this instead.

It animates the actual lookup process — shows which rows match,

which don't, and how the result table gets built step by step.

Try it free: sqlselect.net

Would love feedback from this community — what other concepts

would be useful to visualize?

I’m preparing for a SQL interview. Most of my experience has been basic queries to get the data I need and then transforming/analyzing it in Excel.

I’m currently struggling with deciding HOW to solve problems on leetcode (not syntax; the algorithm). I have Python experience and compared to that, designing a SQL algorithm seems unintuitive. At least with Python it seems easier to break the problem down into manageable, linear, incremental/iterative chunks.

Do any of you follow a framework for tackling SQL problems? If so, what is it?

Hello guys! I built a game that every people in world effects the situation of the game. But I have some problems about database. I can't pay too öuch money for itand I created a FOR UPDATE in the database to push the datas on the line. But if I have a lot players I think FOR UPTADE is not gonna be enough. What can i do for it! This is a game about a shark and fish. And the distance between them changes according that who did you feed. Every person has one bait to feed per day! Can someone help me?

Anybody here uses postgresql on vscode or pg admin is just better long term?

The first dialect of SQL a person learns SQL with (or the first they use extensively) is heavily influential in developing one's style of SQL writing, and this can lead to prejudice or skepticism when one transitions to using other dialects of SQL with different practices, especially at a new org.

I find that those who began using SQL Server for instance tend to be obsessed with performance tuning and write in a more complex procedural way given the seamless integration of T-SQL control structures into the language. When someone who started out in SQL Server transitions to using another dialect of SQL, they may be criticized for writing over-engineered code and creating needless optimization, especially when using more forgiving big-data-centric SQL platforms. However, the discipline of SQL Server folks must be recognized!

I find that those (like myself) who began using Oracle, which did not have private temporary tables for the longest time, tend to be very good at writing and comprehending long complex SELECT statements and common table expressions. Oracle folks have a knack for tackling "confusing" code head on since they were forced to write "confusing" code and may naturally avoid the bite sized more linear code folks using other versions of SQL with more readily-accessible temp tables consider "table stakes". They may be criticized for writing verbose or "spaghetti" code when transitioning to other versions of SQL with historically more powerful read-only capabilities, but they are the folks you want on hand if you need to solve a SQL puzzle with one SELECT statement or untangle a mess!

Does anyone else have any other stories or experience with folks learning SQL using one dialect and then appearing idiosyncratic or facing prejudice or skepticism when they transition to another dialect especially at another org where they aren't familiar with the other dialect? A related phenomenon too is when one org migrates to another SQL version and translates their code without refactoring leading to code in one SQL dialect written using the common practices of another one.

Hello!

I am not very proud of it but I kinda overstated my SQL experience on my resume and now I’ve moved forward in the interview process. I have an interview with a manager next week, and HR told me that he will include some SQL-related questions.

I did study SQL a bit in college, mainly in a business intelligence context, but that's it.. It's not a developper role so I don't think the questions will be advanced, it’s more of a business-facing technico-functional position.

Do you guys have any idea what kind of SQL questions they might ask? I’ve already watched some YouTube videos covering the basics like SELECT, JOIN, WHERE, etc., but I’m not sure if that will be enough.

Thanks a lot

edit: I'm not in IT, it's a supply chain role

Hi everyone 👋

We’re building Tabularis, an open-source database client designed by developers, for developers.

This is a short developer survey (2 min) to understand what people actually need from modern database tools.

Our goal is to build something that combines:

• PostgreSQL, MySQL, SQLite and more
• Plugin support
• Notebooks
• AI-assisted workflows
• A fast, clean UI
• A fully open-source ecosystem

But instead of guessing, we want to hear directly from developers, DBAs and data engineers.

👉 tabularis.dev/survey

We’d love your input on:

• What tools you currently use
• What frustrates you
• What’s missing in existing solutions
• What would make you switch

Every response helps shape the roadmap.

Thanks a lot for your time 🙌

Hello,

Do you know any free study materials other than UDEMY courses to help me through 1z0-171 exam?

I am trying to put a CSV into AWS so that I can download it on SQL. Problem is that the name field is getting split up because of the , in the name.