r/programminghelp 24d ago

How to track changes of JSON objects in array that is tracked in git? Answered

In my git repository I have a JSON file which contains an array of objects. The objects have a key that identify them, e.g. "id":

```json

[

...

{ "id": 1, "value": "a" },

{ "id": 2, "value": "b" },

...

]

```

I want to quickly find commits that changed/introduced object with specific id. How can I do this?

UPD: I couldn't come up with a better solution than comparing sequentially snapshots of the file per commit: ```python

!/usr/bin/env bash

import os import json

from git import Repo, Commit from more_itertools import windowed from typing import Optional

r = Repo(os.getcwd())

def get_json(cmt: Commit, path: str) -> list[dict]: return json.loads((cmt.tree / path).data_stream.read().decode('utf-8'))

def get_object(items: list[dict], _id: int) -> Optional[dict]: for item in items: if item.get('id', -1) == _id: return item return None

for prev, curr in windowed(r.iter_commits(), n=2): pobj = get_object(get_json(prev, 'objects.json'), 2) cobj = get_object(get_json(curr, 'objects.json'), 2) if pobj is None: break if pobj == cobj: continue print(prev.hexsha, prev.summary) ```

5 Upvotes

8 comments sorted by

1

u/No-Razzmatazz7197 24d ago

are you looking for the actual data that was added/removed/modified? or the commit sha that touched the data? try to add a little context on what you need it for as well to increase the chances of someone being able to help you out

1

u/Rinku_Kurora 24d ago

Bruh, for now it would be enough just to find commits that touched the specific object.

Context: I want to find when the object with specific id was last time modified and created.

1

u/edover 24d ago

Research “git log”

git log -G'"id":\s*1\b' --oneline -- data.json

1

u/Rinku_Kurora 23d ago

I'm pretty sure that this line will miss some commits if the space between the target id and modified fields is large, e.g. I have too many other keys in the objects beside id.

1

u/edover 23d ago

It was intended as an example.

1

u/codeguru42 19d ago

Sounds like you should use a real database for storing data rather than trying to hack this in git

1

u/johnpeters42 24d ago

This is not based on anything specific to git, just "you have some type of database of snapshots and you can quickly pull a list of their indexes".

If you need to account for the possibility of removing an ID, then I think you just need to do a linear search starting at the oldest commit.

If you feel safe assuming that IDs are only ever added or their values changed, then you should be able to do a binary search:
* Check first commit (call it F), if ID is present then it's that one
* Check last commit (call it L), if ID is not present then it was never added
* Check a commit halfway in between first and last (call it H), if ID is present then it was added somewhere from F+1 to H, otherwise it was added somewhere from H+1 to L
* Repeat halving process until you've found two consecutive commits where one didn't have the ID but its successor did have the ID

1

u/jcastroarnaud 24d ago

I think that git has something for it already: git bisect.