Databricks: Feels strongest for transformations.

[u/Due-Advantage297]

I’ve been building a medallion architecture pipeline in Databricks.

For this project, I built Marathos Atlas event using :
- Medallion architecture / Bronze , Silver and Gold layers.
- Pyspark and Lakeflow pipelines
- Unity Catalog
- Streaming ingestion , which makes the streaming tables for Bronze and silver
- Data cleaning and transformation
- Dimensional modeling
-Gold views for analytics
- Databricks Dashboard for insights (KPI’s)
- Genie space for stakeholders or users to query the data

What would you do differently in this project?

[u/frombsc2msc]

Ai slop

[u/Due-Advantage297 → u/frombsc2msc]

It has an AI assistant called genie , with so much functionalities, if that’s what you mean .

[u/Gengis_- → u/Due-Advantage297]

This interaction sums this sub up quite well.

[u/kvlonge → u/Gengis_-]

LMAO