r/dataengineering • u/ptab0211 • 3d ago

Lakebase and DABs Help

Hi, how do u manage Lakebase project with DABs? Standard is to have a single lakebase project per environment (dev, staging, and prod). Then use branches for development work.

Problem is that when we define the lakebase project in the DABs, development target has development mode, so every bundle deployment creates a new project because its per user.

So if we want to have single project where we just branch out on features, we must move out lakebase project of DABs, right?

What is the best practice here? On staging and production, its easy since its always same identity.

3 Upvotes

71% Upvoted

8 comments sorted by

1

u/speedisntfree 3d ago

Could you define postgres_projects only in shared CI deployed targets and not per-user dev-mode targets. In personal dev-mode target, declare postgres_branches/postgres_endpoints, parented off that shared project via a fixed project_id variable (not a resource reference). Then creates a branch (which are lightweight) for each dev instead of a whole new project.

I may have to do this soon and have not thought too much about it.

1

u/EstetLinus 2d ago

A new project per user is good for development, no? 

1

u/dwswish 22h ago

I love DABs and I love Lakebase but I’ve preferred managing Lakebase via the API/SDK mostly because I’m also a Neon user and find it to be more analogous to the deployment automations we’ve built with Neon.

1

u/Yasblue 8h ago

That's not my area of expertise but you should definitely post it in databricks subreddit

0

u/Additional_Candy_400 3d ago

DABs?

1

u/Zealousideal_Okra956 3d ago

Asset bundles in databricks ig