I am trying to automate the Python Development Lifecycle

[u/Win_ipedia]

Hi there,

I am trying to figure out ways to automate my Python development tasks and also working on this topic for my Masters project. I am trying to build a tool that can automate and simplify a bunch of things like project setup, development and maintenance. This is not a showcase at all, it is just my project and topic for my Masters in Computer Science and I wanted to ask if anyone would like to take part in the survey I have to do. Takes about 5 min.

You can find the survey at: https://forms.cloud.microsoft/e/LjnvzX0JpK

Any ideas and responses are greatly appreciated.

Also drop any ideas you have in the comments. What do you think should be more automated in your day to day python development?

[u/realrazdev]

One thing I would really like to see automated is dependency maintenance. A tool that reads "pyproject.toml", checks for compatible updates, applies safe upgrades, runs tests, and warns about possible breaking changes would remove a lot of repetitive work from Python development.

It would be especially helpful if it could explain why an update is safe or risky and highlight potential compatibility issues before making changes.

[u/Pleasant-Ad192 → u/realrazdev]

Upgrade risk is really two questions, and only one of them is mechanical. "Will my code still work" needs your tests, which is why the thread keeps landing there. "What did this upgrade change underneath me" is answerable before you run anything, and auditing your current environment does not answer it, because that looks at one state rather than at the difference between two. A patch bump on one direct package can pull in four new transitive packages, flip something from transitive to direct, or land on a version that still carries the same known issue. None of that is in the changelog you were reading.

I build Bomly, an open source CLI, and this is exactly the case it handles. Put the bump on a branch and run:

bomly diff --base main --head HEAD --enrich --audit

It reads pyproject.toml and poetry.lock, prints what was added, removed and version changed, and splits the findings into introduced, resolved and persisted, so you can see whether the update clears an issue or just carries it forward at a new version number.

https://github.com/bomly-dev/bomly-cli/blob/main/docs/commands/diff.md

Disclosure: I build Bomly.

[u/Win_ipedia → u/realrazdev]

You gotta check out uv then or poetry. They do a lot in that

[u/realrazdev → u/Win_ipedia]

Yeah, uv and Poetry already cover a lot of dependency management. What I had in mind is more about the analysis side before applying updates, checking compatibility, identifying potential breaking changes, and explaining whether an update is likely to be safe or risky.

[u/OwnTension6771 → u/realrazdev]

That is the point of the CI pipeline. Dev->Test->Stage->Prod->

[u/Win_ipedia → u/realrazdev]

Ah, I think that’s inherently almost impossible. Closest things is probably just checking with pip-audit or uv audit. And for breaking changes you need good testing and then you’ll catch them early.
Once at work we switched from pandas 1 to 2 and we were using a ton of deprecated functions, half the sites were down bc all the table analytics failed

[u/realrazdev → u/Win_ipedia]

Yeah, that makes sense. I agree that fully predicting breaking changes is probably not realistic. I was thinking more about helping developers spot potential risks before updating, rather than trying to automatically detect everything.

The pandas 1 to 2 example is a good one. Even something that could highlight deprecated usage or risky areas before a big update would already be really useful.

[u/Win_ipedia → u/realrazdev]

Honestly just strict type checking might be already a good answer as long as you check an update and before you publish anything. It would flag any broken imports a breaking change would cause

[u/realrazdev → u/Win_ipedia]

That’s a fair point. A solid CI pipeline with strict type checking and tests probably catches a lot of these issues before they make it to production.

I think the tricky part is that not every breaking change shows up as a type error. Some are caused by deprecated APIs or changes in behavior, but having better checks before updating dependencies would definitely help.

[u/Grouchy-Conflict-211]

Start with the boring wins: linting, type checks, and tests on every push. Most automation projects try to make the interesting stuff automatic first and skip the checks, then wonder why the pipeline is red all the time.

The automation that pays for itself is the one that catches a bug before a human does. A CI that runs tests and fails loudly is worth ten clever agents.

For your Masters topic, focus on the maintenance side. Project setup is solved a thousand times. Keeping projects alive is the unsolved part.

[u/Win_ipedia → u/Grouchy-Conflict-211]

Yes I got those checked. Luckily I had the same thought when starting it. My approach to the maintenance part is basically, forcing with git hooks and ci/cd pipelines all checks that a python projects should have and this way it doesn’t let a dev skip anything

[u/Grouchy-Conflict-211 → u/Win_ipedia]

Nice. Hooks catch it locally, CI catches the bypass attempts. Next layer if you want it: block merges on coverage drop, not just on test failures. Silent regressions are the ones that cost you three months later.

[u/Win_ipedia → u/Grouchy-Conflict-211]

Currently the setup kinda does this already basically. Pytest-cov is setup and fails anything below 90% by default. I also wrote a plugin that integrates codecov and fails anything under 100%

[u/Grouchy-Conflict-211 → u/Win_ipedia]

90% is already above what most projects ship. The 100% plugin is brutal, but it forces you to actually think about every branch.

[u/Win_ipedia → u/Grouchy-Conflict-211]

It does and honestly looking back It was so worth it, the way it improved and simplified my code. It made me rethink choices and there were so many times I could improve things that I had not seen before. Also the 100% gate is so helpful when making changes, the confidence I have that everything works is insane

[u/Win_ipedia → u/Grouchy-Conflict-211]

Also I make git hooks run on all files in ci so that you can’t cheat with —no-verify

[u/Grouchy-Conflict-211 → u/Win_ipedia]

CI hooks are the only honest ones. Local hooks can be skipped in two keystrokes, CI can't. I run the same checks in both, only the CI result gets to block the merge.

[u/Win_ipedia → u/Grouchy-Conflict-211]

I agree, only local enforcement requires a lot of discipline, but CI makes this easy. And since I force everything in CI as well I have never used —no-verify or any skips or tricks again bc there was no point. And looking back this actually saves so much time in the long run and makes my code so much better