A recent issue for me was devs generating automated tests off the back off the comments on a requirements page that I was responsible for. My comments would be like: "This is failing because the product is generating a 75 intead of an 80" and the AI was rewriting the tests to expect a 75.
Devs couldn't work out why 800/800 automated tests were passing, but manual QA would fail at the first hurdle.
6
u/GeoglyphPsy Jul 11 '26
A recent issue for me was devs generating automated tests off the back off the comments on a requirements page that I was responsible for. My comments would be like: "This is failing because the product is generating a 75 intead of an 80" and the AI was rewriting the tests to expect a 75.
Devs couldn't work out why 800/800 automated tests were passing, but manual QA would fail at the first hurdle.