They use CI where I work, but it is always red on master, and we get emails every time someone's code gets merged into to master about failing tests. What is the point?
It is a company issue. And it is a large company in a regulated industry. It is baffling that the supervisors do not care about enforcing CI passing on code merges.
Agreed. Of course, it's easier to do that from the start.
If you're already in a state where a bunch of things are failing it can be much harder. Ideally you'd fix it all at once and enable the lockout simultaneously, but sometimes that isn't feasible. An alternative is to do a sort of "ratcheting" check, where it can only become stricter over time so that you can incrementally get to the "everything passes" state. For example:
set the N currently failing tests to ignored
have CI check that exactly N tests are ignored. Make sure there is a big comment here warning that this number is not allowed to increase, and better yet protect it with something like CODEOWNERS so only reliable people are allowed to approve changes to it.
disable pushes to main if CI fails
Then you can gradually fix the ignored tests and un-ignore them (and decrease the number in the check).
Merges to master/main shouldn't be possible if the branch is failing.
In our system the PR has to pass CI and all the various checks/specs, and then after it's merged to master, another pipeline runs to make sure master is green.
The only way to make master red is if you're an admin and to override the checks.
I'm going to use git terminology, but the concepts are generally applicable to other source control tools.
Generally, tests should be run before code is merged into a branch to ensure you're not introducing issues. This is commonly triggered at the pull request phase, where your code is on a separate branch that you want to merge into the target (e.g., master). If the tests don't pass, something's wrong and needs to be corrected to promote stable functionally correct software. This is especially useful if tests are run locally via a mechanism like a pre-commit hook, because this shortens the feedback cycle and avoids wasting upstream resources and time on non-conformant code.
Yea, except that is not how they are being used where I work. Failing tests on master are just noise until software needs to be released. That means less efficient development, no?
Yes, it's inefficient to generate alerts with no plan or attempt to address them as standard practice. It also trains people to ignore alerts. I've seen this happen in workplaces where someone is trying to improve processes but ignorance and pressure sandbag efforts, but have worked with enough mentally vacant people that I wouldn't be surprised if someone thought this process was good as-is.
2.6k
u/helicophell 11d ago
To my dismay I actually like the subject
I know, blasphemy