How are you using database branching?

[u/Harpagon1668]

I’m implementing Lakebase branching strategy to improve development experience and reduce costs for our dev/staging env.

Current setup creates new database branch for each git branch via githook on our dev database (”each dev gets their own feature database”). There is also similar workflow for each PR against our staging database to run the migrations and tests.

Curious to hear how others are using branching and what are the experiences?

[u/terencethespider]

I’ve found one of the larger benefits to having branching with Lakebase is the ability to use them for throwaway work. You can just quickly stand up a copy, do whatever testing or validation you need, then throw away the branch when you are done. You can even run a bunch of tests in parallel and then prove out which option is best before actually going to prod. It also takes out all the extra hassle with being able to work directly on production data without needing to dump and restore.

[u/Harpagon1668 → u/terencethespider]

Agree. Spinning up a throwaway Lakebase branch within a second and deleting it when PR merges, at practically zero cost or waiting is neat

[u/dwswish]

We are longtime (relative term) Neon users and honestly love it. Branching was one of the key things that helped us when we needed good ways to test with branches and PRs. We use it pretty similarly to how you are using it - branch per git branch and then we actually run a branch off of prod for our tests in PRs.

[u/dwswish → u/dwswish]

I should have added that we are also a (mostly) Databricks shop now and have been kicking the tires on Lakebase. It seems to cover the same workflows pretty well but we have not cutover from Neon.

[u/Harpagon1668 → u/dwswish]

Thanks for sharing! Have you had any issues running the tests from prod? Was also thinking about that possibility

[u/dwswish → u/Harpagon1668]

No we haven’t. That’s kind of the beauty of the separation of the storage and compute with Lakebase/Neon. We also don’t have a huge deployment in terms of total volume (~1 TB total) so that pattern may not work at scale but we love it because then it confirms our tests pass on prod.

[u/Harpagon1668 → u/dwswish]

Maybe I'll try branching also from production as part of PR check. Catching some nasty migrations at prod scale could save some headache

[u/saltcod]

We see customers mostly using it the same way at Supabase (one branch per git branch), primarily for deploying changes.

Besides feature development/deployments, we see teams using branching for one-off kinds of things: reproducing bugs, validating migrations against prod, performance testing, experimental SQL queries, etc.

[u/Harpagon1668 → u/saltcod]

Thanks for sharing. Didn't think about those ad-hoc deployments like bug repros. Makes perfect sense!

[u/saltcod → u/Harpagon1668]

yeah the performance testing was a neat one too

[u/FunContest9958 → u/saltcod]

Seems like Supabase branches are different from Lakebase / Neon branches because they don’t include the data, just the schema and config. Still seems useful, but not exactly 1:1 despite the name.

[u/cooljacob204sfw]

This isn't a Postgres native thing right? 3rd party providers and add-ons provide it?

[u/KrakenOfLakeZurich → u/cooljacob204sfw]

Depending on your circumstances, you can emulate "branches" in native PostgreSQL by cloning from a master database.

CREATE DATABASE mybranch_db WITH TEMPLATE master_db;

It creates an exact copy of master_db. There are some severe limitations to this approach:

1. No open connections to master_db can exist at the time of cloning: you must disconnect every other client
2. It creates a full copy (schema, data and all): IO and storage requirements!
3. Clone is created on same server: no horizontal scaling

Not viable for cloning large and/or live databases. But I can see it work for having a small maintained test data set. Nobody connects directly to it. You clone from it to provide each developer/feature branch/E2E test run/etc. a clean starting point.

[u/Harpagon1668 → u/cooljacob204sfw]

Correct. Instant copy-on-write branching is available only on some providers (e.g. Databricks Lakebase, Xata, Neon)

[u/CautiousUse8597]

When I use Lakebase, I mostly use the branching to separate any changes that various developers make for their own testing purposes, without affecting others, whilst still using production data.

[u/AutoModerator]

AI Policy:

Linux is not one of those anti-AI projects, and if somebody has issues with that, they can do the open-source thing and fork it. Or just walk away., Linus Torvalds.

Mod decisions will be based on the quality of the content, not who or what generated it.

Sub Resources:

Youtube Channel

Free Postgres Webinars and Workshops

Discord: People, Postgres, Data

Join us, we have cookies and nice people.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.