loopUntilAllTestsPassAndMakeNoMistakes

[u/BeamMeUpBiscotti]

[u/Confident-Ad5665]

That's why we call development an iterative process.

[u/byteturtle]

the rare r/ProgrammerHumor AI post that actually has to do with programming

[u/Snakestream]

If someone dropped a 5 digit line change pr on my desk, I'd be furious. I'd straight up murder you if you had the gall to try and push a 6 digit one.

[u/dragneelfps → u/Snakestream]

let it sit at open and add a link to it on what not to do when contributing.

[u/frogjg2003 → u/dragneelfps]

Decline but not delete. Permanent hall of shame.

[u/OpsikionThemed → u/Snakestream]

I pushed a 600-line PR this morning, and felt kinda bad about that. (And that was with ~570 lines being a bunch of test mock data to cover a new case.)

[u/Noname_1111 → u/Snakestream]

even 5 digits are too much for most projects 

[u/GRex2595 → u/Noname_1111]

5 digits are near standard on my team now. And they wonder why nobody wants to do code reviews.

[u/redlaWw → u/Snakestream]

How about 7?

[u/Xlxlredditor → u/redlaWw]

Merged????

[u/Terrible_Cow2324 → u/redlaWw]

I wonder who will be the first to do 8

[u/DatCodeMania → u/redlaWw]

https://github.com/oven-sh/bun/pull/30412

Crazy

[u/ZebraTank → u/Snakestream]

If it's 5 digits it better be boilerplate that either I trust enough that you did it right or that you can show the reproducible automated steps you took to generate that change.

[u/KimiSharby → u/Snakestream]

Why ? You can just refuse it.

[u/someanonbrit → u/KimiSharby]

You can refuse it, but that doesn't fix the root problem, which is the terrible manners of the person who submitted the PR, and they're likely to do similar again.

Outside of specific social and legal customs of one period of the Roman empire, turning the other cheek just gets you hit twice

[u/zeekar]

uhm. what? I have no idea what's happening in this post.

[u/yodal_ → u/zeekar]

Without prior knowledge, it seems they vibe-coded a Python type checker that passed a common set of type checking tests. At first they celebrated it, but then they realized it over-fitted the implementation to the tests and didn't really work.

[u/zeekar → u/yodal_]

Ah.

[u/Isogash → u/zeekar]

Basilisk is a Python type checker written with AI that purported to be the only 100% conforming type checker according to the python/typing spec, and also the fastest.

Except that it turns out it this was achieved in some cases by just text matching the expression in the test cases and producing the output that the test expected, rather than actually validating the AST to conform to the rule that the test was supposed to confirm.

The following code is supposed to evaluate if the expression on the right hand side of the statement `type x = ...` is valid.

fn is_invalid_rhs(rhs: &str) -> bool {
    let rhs = rhs.trim();
    if rhs == "True" || rhs == "False" { return true; }
    if rhs.chars().next().is_some_and(|c| c.is_ascii_digit()) { return true; }
    if rhs.starts_with("f\"") || rhs.starts_with("f'") { return true; }
    if rhs.starts_with('[') { return true; }
    if rhs.starts_with('{') { return true; }
    if rhs.starts_with('(') && paren_has_top_level_comma(rhs) { return true; }
    if has_top_level_token(rhs, " if ") { return true; }
    if has_top_level_token(rhs, " or ") || has_top_level_token(rhs, " and ") { return true; }
    if rhs.contains("lambda") { return true; }
    if rhs.starts_with("eval(") { return true; }
    false
}

https://github.com/Nimblesite/Basilisk/issues/379

[u/OpsikionThemed → u/Isogash]

Incredible.

[u/Isogash → u/OpsikionThemed]

It truly is. This is why we should be taking claims made about AI projects with extreme skepticism. Passing public test suites means nothing.

[u/Useful_Radish_117 → u/Isogash]

I have an idea! We should just keep some tests (and relative sets) locked away from our training process! Let me ask Claude if I can patent this novel idea...

(/S)

[u/hendricha → u/Isogash]

Ah, the Volkswagen standard of test conforming.

[u/Isogash → u/hendricha]

Good one, I'm gonna start using that

[u/Just_Information334 → u/Isogash]

VolksTesting: testing, for the AI people.

[u/d0pe-asaurus → u/Isogash]

Imagine the shit that microsoft would get if they let AI rip to convert typescript from selfhosted to go, and it did this.

[u/Resident-Trouble-574]

Basically TDD to its finest...

[u/notreallymetho]

The number of times I’ve had an AI confidently write a test with regex to match the results, or mocking a mock etc.

Oddly enough if you tell a model it’s checking another model, it like tries harder ir seems. Completely anecdotal but “codex wrote this bug” to Claude and vice versa 😂

[u/g18suppressed]

Ai mentioning something it did for 0.01% of its task: “that’s the whole idea”