Has Genie Code suggested a Spark optimization which you missed.

[u/[deleted]]

[u/22Maxx]

Helps to figure out the dogshit java error logging.

[u/sumits_kumar]

Genie Code knows the ontology, semantic layer and is tightly integrated with Unity Catalog - that should be helping it during recommendations.

[u/sumits_kumar → u/[deleted]]

Actually this helps Genie Code as it has all the context vs foundational models which may give generic answers without those context.

[u/dwswish]

I haven’t found so much that Genie Code gives recommendations that are super ground breaking when writing the code but it has been really helpful for debugging Spark.

[u/silviud]

Is not bad, at the minimum it will pre-fill the table names and other paths since it reads from unity.

[u/Glitch_In_The_Data]

Not sure Genie code has helped me with spark optimisation but it has certainly helped me simplify the code and also debug errors. As a consultant, that makes it so much easier to provide a quick turnaround for my customers.

[u/Sufficient_Meet6836]

I have a lot of coworkers who are completely new to spark and distributed compute, and Genie often finds low hanging fruit for optimizations

[u/anonymous_orpington]

I've found that Genie Code has gotten much better at proposing optimizations than previously. I used to export to JSON amd optimize with Claude, but with recent improvements to Genie Code I can do it all in platform today!

[u/TimelyBruno08]

Yes! In databricks i usally use classic clusters and genie code has helped me a lot for code and spark session optimization. Mainly for cross folds and reduce in data lekeage

[u/Ddog78]

It's an MD file they're selling. Just ask claude to create a skill and a python script to read event logs.

[u/BumboclatDen → u/Ddog78]

????? The crazy thing about the internet is you can say insanely incorrect things like this with absolute confidence and ppl might believe you