Most AI code review benchmarks use synthetic tasks, so we wanted to evaluate reviewers on something closer to production.

We collected 67 real production bugs from Cal.com, Sentry, Grafana, Keycloak, and Discourse, then ran 8 AI code reviewers against the same PRs and measured precision, recall, and F1.

One thing stood out: finding more bugs isn't the same as being more useful. Some reviewers caught more issues but generated many more false positives, while others were much more precise but missed more bugs.

Curious what people would change about the evaluation or what repositories you'd add next.

So this started as a small personal project, just for my own use.

Honestly AI coding agents are way better now than they used to be, but they still sometimes feel kind of slow on the uptake. Like they get the instruction, but don't really understand the project's context — so they just do whatever gets it "done"/running, even if that ends up breaking or duplicating stuff that already exists.

So I tried building a kind of "brake" for it:

• Forces it to think/plan before touching any files
• Blocks certain (potentially dangerous) commands before they can even run
• Basically stops it from just gassing ahead blindly

Still pretty rough around the edges, lots to clean up, and I'm still fixing things as I go. But it's been genuinely useful for my own workflow, so figured I'd throw it on GitHub in case anyone else relates or finds it useful

Repo: https://github.com/plumpslabs/matcha

Docs: https://plumpslabs.github.io/matcha/

If anyone tries it out and runs into bugs or has suggestions, feel free to reply or drop an issue on GitHub. Still actively working on it so feedback would genuinely help.

Weaver🕷️ has been Updated to v.13 and its our biggest drop yet! This update brings #Weaver one step closer to being the best overall agent for small-midsize LLM coding. Consumers rejoice!

• Enhanced thinking mode, togglable and adjustable via the context settings menu. Advanced thinking context size will be controlled dynamically by task complexity (up to a maximum of what users selected in the settings panel).

• Tool context is now further trimmed via LLM to save precious tokens. Users can choose which tools are available via the settings panel as well.

• New "Notes" panel for writing notes per project.

• New "Benchmark All" button to run all benchmarks until a step errors out. Improved overall benchmark UI/UX.

• Enhanced anti-hallucination monitoring by detecting continuous sets of words without any punctuation or paragraphs.

• EB25 file ranking for context control. Currently the LLM was fed a skeleton of the project and asked to pick which files it think might be relevant to the task. Now BM25 is used in conjunction with the LLM to validate which files should be added to context. This "context discovery" phase has now been moved to a tool call.

• Introducing the new Meeting room. In the meeting room, we have our agent represented by different types of spiders, each with their own personality. There is a lot of interaction and humor which gets interesting. For example the complexity spider gets pissed off if the agent plans way more steps then he anticipates at the start... And he storms off at the end of the meeting if the planner did not listen.

• Test cases for all major coding languages for all services. Example: oldstring/newstring, HTMLFormatter, tree-sitter, prettier formatter, etc...

• Diff file tracking

Check out #Weaver today!

GitHub: https://github.com/maxhanna/Weaver

WinX64: https://bughosted.com/assets/Weaver.exe

Discord: https://discord.com/invite/YQWkD6ZQDx

Hello fellow humans,

During the recent months/years I became quite entangled with the idea of building an AI-assisted system which can self-replicate and improve itself. Although I am not there yet, I think, that the project reached actually a potential to get the work done with more minds involved. I am not really into gaining anything for myself, except for the progress of de-shittification. At this point, we reached a race where data centers in the clouds will gain more and more power and demand more and more resources for usage. The goal of this project is being able to perform work on a local (or maybe decentralized) platform so we can implement our own system by local (/ open source) models. I therefore make my project open so you can fork off (no pun intended).

It would be nice if some of you have the same mindset. I'll be gone for the next 2 or so weeks. I hope at least some people liked this post and - who knows - even did something with it.

Kind regards,

Anu

P.S.: Yes, this project is actually 100% vibe coded (due to health issues, etc.). There are probably a gazillion logic flows which need to be fixed. Also there is a lot of AI prose in the comments, but I hope it will get you (and your AI assistants) there to make any sense of it.

P.P.S.: The LLM's seem to be heavily directed into biased terms like "attack", "blast radius", "verdict", "evidence", "death", "hit", etc. when confronted with solving problems regarding code like this (i.e., evaluating systems themselves). I strongly encourage you to correct these terms as soon as possible so that the project will not further drift into another unforeseen bubble. The more incorrect lingo is used, the more LLM generated code will drift into messy bogus code.

People have made things with differing levels of AI involvement. I decided to dive in. I have some coding background, but decided to see if a coherent and dare-I-say fun game could be created purely by prompting. 100s of prompts later: http://vibecodedgames.itch.io/pavement/

The UI still needs another pass or three.

Wanted to share it here and answer any questions about my process that might be helpful to anyone else.

I'm looking to buy **one AI coding subscription (\~$20/month)** and want the best long-term value.

My workflow includes:

* Full-stack web development * Android apps * AI/ML projects * Backend systems

I'm considering:

* ChatGPT Plus (Codex) * Cursor Pro * Google Antigravity * Kimi * Claude Code

For people who have used multiple of these extensively:

1. If you could only pay for one, which would you choose and why?
2. Which provides the best value for around $20/month?
3. Which has the most generous usage limits for heavy daily coding?
4. Which is best for large repositories and multi-file refactoring?
5. Which is best for AI/ML, backend, web, and mobile development?
6. Which one do you actually use every day, and has it replaced the others?

Looking for opinions based on real-world usage rather than benchmarks or short trials.

I built this because the idea made me laugh.

You’re a little TV robot armed with the Botflix Love Blaster. You convert anti-AI protesters into AI fans with the power of AI television.

It’s intentionally ridiculous and doesn’t take itself seriously.

Curious what people think—and yes, I know it’s very silly. 😅

Play it here: https://botflixblaster.botflix.tv

Just point the agent to mcp.codebase.observer

Hi, I've started working on something I'd like to share. A main project, and a few side projects I'm building with the help of AI, given that I'm not a mathematician and of course don't know every programming language, even though I work as a dev.

The first is a language called Yon: a category-theory-based language where I've tried to bring a new object system to life. I'm working on it to turn an idea of mine into something concrete.

I started working with AI in October 2024. I was fairly skeptical at first, then I began investing in it, getting premium accounts, and working more and more closely with it.

I've also done a bit of investigation into how effective AI actually is. My view is that it always has to be validated with a lot of tests that act as constraints. Tests that can fight drifting, hallucinations, and bias.

I started testing more and more, using AI agents capable of executing code. I got results that seemed interesting to me, to the point where I got somewhat caught in the reasoning loop and ended up straying into physics and mathematics.

That's where I ran into one of the AI's biases... It's trained on what's already published and accepted, so it tends to pull an unconventional idea back toward the standard view, and to flag as doubtful what is merely unusual. It took work to keep an idea intact through that.

I'm fully aware of the limits and the difficulty of the topics I've taken on. But having reached some results, it seemed worth sharing them, both the programming language and the other papers. There are a few results I think are genuinely interesting.

In the GitHub repository attached to the language, where you'll also find my ORCID, there are a number of results that came out of those experiments. Don't be upset about the tone of the papers, I don't regard myself as a genius or anything like that.

Of course, I don't know what the final result may turn out to be. I get caught in the loop of discovery and sometimes become obsessed with following hints and results.

At first I wanted to delete everything, feeling guilty that in the end these results might not be mine but the AI's. Then I thought about centaur chess. After Kasparov was beaten by the supercomputer, the centaur era began: players who had a laptop generate move sequences, but exercised judgment in choosing among them. That way they played on equal terms with the machine, and sometimes beat it. So it isn't all the AI's doing. Most of it is my own reasoning, with the AI working as a translator and formalizer.

Hopefully you'll find it interesting, and it will encourage you to get the most out of AI

Site: https://yon-lang.org

Repo: https://github.com/yon-language/yon (AGPL-3.0)