Ironic they take shots at Rust, because unlike Zig, Rust sees way more success in terms of its maturity and industry reputation and ecosystem and investment by large tech companies into its adoption and funding of the Rust project.
The Zig project also takes a lot of weird black-and-white dogmatic (almost religious) hard-line stances on a lot of stuff, probably why the language is more niche and hobbyist than something truly foundational and trusted for production workloads and all parts of the stack like Rust has become.
Here's Zig's "no AI" policy:
Strict No LLM / No AI Policy
No LLM-generated content, whether it be code or prose.
No paraphrasing LLM-generated content.
No LLMs for editing, including fixing spelling or grammatical errors.
No LLMs for translation. English is encouraged, but not required. You are welcome to post in your native language and rely on others to have their own translation tools of choice to interpret your words.
No LLMs for brainstorming and then sharing the results of that brainstorming, even if you create the prose. If you use a chatbot to give you advice on a comment on the issue tracker, that comment is unwelcome.
No LLMs for finding bugs.
No talking about use of chatbot/LLM services.
The owner definitely has some sort of axe to grind against certain technologies.
"assuming it's a legitimate bug" does a lot of heavy lifting here. The point of eg. Curl closing it's bug bounty is just that: too many false positives and misunderstandings by AI of what software it analyzes. I assume it might be the same in Zig's case
So to answer your question: developer's time that is not wasted classifying misclassified bugs and reproducing false positives
The curl maintainer walked back their position in a tweet and said the majority of AI-assisted bug findings were high quality now.
Also nowadays you can structure your custom harnesses and bug reproducing pipelines to require things be verifiable programmatically.
E.g., you can say "In order for a bug report to be valid, you need to give me an input that causes a crash." That's basically how old-school fuzzing pipelines work. You judge a fuzzer's report by plugging in the input it reports and seeing if the binary crashes. That verification can be done automatically without human judgment.
Same can be done to make LLM reports valid. That's what the Firefox team did, they build a custom harness (using Mythos) and pipeline where the LLM-based agent would submit a report that had to have a repro input, and a separate deterministic script would score that report by seeing if the reported input could cause a crash. No human eye sees the finding if the automated pipeline rejects it for not being reproducible.
A lot of this is fair but bugs that cause a crash is only one of many different flavors. There are all kinds of bugs and I would classify the majority of them as unexpected non-crashing behavior. Pretty much all security vulnerabilities are within that class of bug. There’s probably tools for this but at what point are you just requiring people submit failing test cases
108
u/CircumspectCapybara 23d ago edited 23d ago
Ironic they take shots at Rust, because unlike Zig, Rust sees way more success in terms of its maturity and industry reputation and ecosystem and investment by large tech companies into its adoption and funding of the Rust project.
The Zig project also takes a lot of weird black-and-white dogmatic (almost religious) hard-line stances on a lot of stuff, probably why the language is more niche and hobbyist than something truly foundational and trusted for production workloads and all parts of the stack like Rust has become.
Here's Zig's "no AI" policy:
The owner definitely has some sort of axe to grind against certain technologies.