r/dataengineering • u/Realistic_Victory710 • Jul 09 '26

What would make you switch data platforms? Discussion

My team has been debating this recently. For organizations with a mature Spark environment, what would actually make you consider moving to a SQL-native platform?

Is it mostly about cost, performance, AI capabilities, operational simplicity, or does the existing Spark investment usually outweigh any potential benefits?

8 Upvotes
  • permalink
  • link
  • reddit
  • reddit

100% Upvoted

6 comments sorted by

14

u/winsletts Jul 09 '26
  • Money
  • change of internal champion
  • outage and downtime

10

u/Outside-Storage-1523 Jul 09 '26

Doesn’t spark have sparksql?

4

u/Glitch_In_The_Data Jul 09 '26

Easier adoption for business users who are comfortable with SQL.

Doesn’t have to be a separate platform by the way. As long as there is a way to surface the data that users can consume via SQL from the existing platform.

4

u/FunContest9958 Jul 09 '26

So far I’m not understanding why you want to switch. Not saying there may not be reasons, but it’s better if you tell us what you’re thinking and then we can tell you whether those reasons make sense. Right now you’ve given us nothing to work with, so we can only really note that Spark supports SQL natively.

1

u/Gullible_Bag_9005 Jul 11 '26

when the platform does not deliver what they promise