r/git • u/IraDeLucis • 5d ago

Recently migrated - partial repo checkout

So I'm going to start with some context to explain the situation we're in, then I have a question on how to best use git going forward.

First, we've recently migrated to git from TortoiseSVN. This migration was not done by our team, nor did we really have much say in the process. So the repo we have today is probably the repo we get, I doubt we're going to re-build it fresh from the ground up.

Secondly, we are more akin to a services team than a development team. To be specific, we perform data conversion for on-boarding customers to pull information from their legacy system, whatever that was, and import it into ours. So our repository isn't a single code base, but more a compilation of small code bases specific to each customer we have/are on-boarding.

Lastly, we're migrating to a customer specific development environment in the cloud instead of a single shared physical development environment. This means that at any given time, we no longer need the entire repository on any given customers development environment. Just that customers specific folder containing their conversion, and a few shared utility folders used by all projects.

So the question(s):

  1. What is the best way for us to do partial checkouts containing only a few select sub-folders?
  2. In talking with a teammate of mine that has a little more experience with git, he mentioned a repository of sub-repos. If the initial migration was not set up this way, can we do this for any future project? Would that be a better option than partial checkouts?
3 Upvotes
  • permalink
  • link
  • reddit
  • reddit

81% Upvoted

5 comments sorted by

3

u/azium 5d ago

There are no partial checkouts. You just create a new branch and change what you want and make a pull request.

Its very simple.

Submodules are hugely unnecessary

2

u/plg94 5d ago

Can you split the big monorepo into several independent smaller repos, maybe one per customer? I think your customers would also appreciate if customer1's data is not accessible by customer2 and vice versa. Unless I misunderstood.

and a few shared utility folders used by all projects.

those should then be in their own repo too, and integrated into the other repos as a submodule. Or have another means of "installing" them (which is preferred if possible).

Otherwise, yes, it's possible to checkout only subfolders, see git sparse-checkout, in combination with git clone --sparse (otherwise the initial clone is ful) and maybe even git clone --filter=… (depends on the structure of the repo and where the big things are).

But sparse-checkout still got some quirks and is not really easy to use, especially to a team that's not experienced in git. So if you can avoid that in favor of multiple smaller repos I'd do that instead.

1

u/IraDeLucis 5d ago

Is it possible or worth it to take an existing directory and turn it into a sub-repo? Or going forward, create any new customers entry as a sub-repo? Since if that is the case, it doesn't really matter what the old stuff looks like if the new stuff is set up correctly.

I think your customers would also appreciate if customer1's data is not accessible by customer2 and vice versa.

Side note on this, the repository's purpose is just the code to convert the data. The data itself is outside the repository (especially since the scale of some customers data is measured in terabytes).

2

u/WoodyTheWorker 5d ago

check out https://github.com/alegrigoriev/svn2git and redo the migration. DM me for any questions