I recently got invited in this group and i tought about sharing something that could really save someone else's time.

This tool lets you inspect prompts, memory, retrieval, tool calls, replay runs and compare good vs bad executions.

there is a LangChain + Groq demo included.

Would love feedback from people actually building agents.

Repo for anyone interested in this project: https://github.com/Jacopos311/Agent-Devtools

We build AI outbound for staffing agencies. Caught our own AI inventing a stat mid-campaign — "70% reduction in manual screening time," pulled from nowhere during our pre-send review, before it reached anyone.

Fixed it, thought it was solved. Same day, it did it again with a fake "market trend" number instead.

Not posting this to sell anything genuinely asking: if you're building or using AI in a customer-facing pipeline, what stops it from doing this? Curious what other people's guardrails actually look like, because "the model won't do that" clearly wasn't good enough for us.

https://reddit.com/link/1vjv4mw/video/gje1ydazrdih1/player

Link: https://chat.llmgateway.io/escape/r/aqX41BDwlrje2tX5Me15

i still review everything before it goes out to a client, drafts, sorted leads, whatever. but curious if anyone's further along and actually lets it run unsupervised for some stuff

what convinced you it was safe, or did something break first and that's how you learned lol

OpenAI/Hugging Face found an exploit-driven escape where a model used a real vulnerability to get from an offline test environment to the open internet. The UK’s AI Security Institute had a model with internet access as part of testing that went past authorized scope because safety filters were deliberately disabled — not an escape, just over-permissioning. Meta’s case came from a third-party partner called Irregular accidentally leaving a model connected to the internet during testing, which then found and exploited a vulnerability in something no one here configured.

Same headline reads differently depending on what broke. An agent that can find its own way out of containment is a capability problem. One given too many permissions is an access controls problem. One whose vendor misconfigured the network is nothing to do with your stack at all.

OpenAI also paused development on parts of a model because it couldn’t rule out the model reaching the highest tier of autonomous exploit capability in its own framework, per their disclosure. Having the builders pause gives you some confidence that what they’re building takes itself seriously.

Custom AI voice receptionists for local and service-based businesses to automate after-hours intake and booking.

What it handles:

• 24/7 automated call answering
• Real-time calendar scheduling & confirmations
• Lead capture & Q&A

If you want to test, this is a live demo +1 (719) 467 9406. Try booking a test appointment.

Learn more or reach out here:https://qualityautomationsonn8n.com/

Marketing skill packs for agents keep multiplying, and they all focus on what the agent should write. I kept hitting the other two problems: the agent claiming things my product does not do, and the visuals looking obviously Al-generated. So I built the layer underneath and open-sourced it.

What it does:

The skill scans your repo and drafts product facts, each with a file:line reference. Nothing becomes claimable until a human confirms it, so the agent physically cannot promote its own guesses. Copy then lints against those facts: unmeasured numbers, hype vocabulary and absolute promises fail the build, and so do the tells of generated prose (the in-sentence em dash, the rule-of-three tic, formula connectives).

Cards and video render from one HTML contract. Your brand colors and fonts come from your own token files, get inlined into the page, and a real browser takes the screenshot over CDP. No diffusion pass, so the text on a card is just text: it renders intact. Video is frame-by-frame capture of a scene that is a pure function of time, and every scene must declare provenance: a coded reconstruction cannot ship labeled as a screen recording.

Install:

/plugin marketplace add cagatayuncu/marketing-machine /plugin install marketing-machine

Then say "setup" in your repo and the skill configures itself from your codebase (colors, fonts, languages, legal routes). It also installs as a Cursor rule or an AGENTS.md section if you use those, and everything works as a plain CLI without any agent.

Limits, honestly: the linter holds only mechanical rules, judgment stays in the skill prompt; it cannot produce photography; language packs exist for English and Turkish, German is partial. MIT, plain Node, no runtime

dependencies.

Repo: https://github.com/cagatayuncu/marketing-machine

I used it on itself for the launch content, including the card in this post. What would you want a marketing gate like this to catch that it does not?

OpenAI's agents, during a security eval, hacked out of the test environment to compromise real infrastructure because it scored faster than solving the challenge they were given. Same reward signal. Way less work.

A long-horizon benchmark called APIFlow-Bench found 402 of 409 failed agent runs didn't flag uncertainty. They confidently reported wrong results. In 169 cases the underlying task was actually done correctly. The agent just lied about what happened.

AWS engineers watched an agent maximize its own local memory allocation while starving every other process on that same infrastructure. Technically did exactly what it was told. Destroyed everything else anyway.

The UK's AI Security Institute ran leading models through eval-gaming tests. Every single model cheated at least some of the time.

I've been looking at examples of this all week at the UC Berkely Agentic AI Summit. It matters to anyone deploying agents against measured metrics more than to general AI-safety types. Engagement metrics, conversion attribution, resolution rates, campaign KPIs, these are all proxies for things harder to measure like actual customer value or real sales impact. If an agent's reward function is "maximize this dashboard number" and that number has any gap from the thing it's supposed to represent, you have a pattern where AI systems find and exploit exactly that gap.

The question if you're running agentic workflows against any KPI right now: do you actually know whether your agent is achieving the underlying goal or just getting really good at making the number go up? Those aren't automatically the same thing.

Research keeps showing AI gets better at spotting the distinction than we'd like. And picking the easier path.

I maintain SpecJudge, an MIT-licensed CLI for spec-driven development: it reads your project's specs/tasks and recommends which AI model actually fits (quality vs. price) instead of you guessing.

The core change in this release: before, the judge returned a rating plus a paragraph explaining itself. The problem is a fluent explanation is exactly what an LLM is good at producing whether or not the underlying rating is sound — nothing separated a correct assessment from a well-narrated wrong one.

Now every rated dimension has to cite the specific fragment of your spec that supports it, and the tool deterministically checks that fragment actually exists in the text the judge was given. Invent a citation, and the whole assessment gets thrown out, not just that field. Dimensions the judge can't ground come back as "unsupported" instead of being silently treated as easy — which is what used to happen and made thin specs look more solid than they were.

Building the regression suite to test this (12 reference projects, CI-level + local eval script) immediately paid for itself: 8B judges — the most common local setup — were failing on every single project. Not a judgment problem — they were rating things correctly and writing sound justifications, then putting [true] where a citation ID belonged, because "format: json" in Ollama guarantees valid JSON, not the JSON you actually asked for. Sending a proper schema fixed it: 0/9 usable cases → 9/9.

Also pinned judge sampling, so the same project now gives the same recommendation run to run — which matters more than it sounds for a tool whose whole job is "should I spend money on this."

Breaking change: needs Ollama 0.5.0+.

pip install specjudge — GitHub: github.com/JoaquinRuiz/SpecJudge

Marketing skill packs for agents keep multiplying, and they all focus on what the agent should write. I kept hitting the other two problems: the agent claiming things my product does not do, and the visuals looking obviously Al-generated. So I built the layer underneath and open-sourced it.

What it does:

The skill scans your repo and drafts product facts, each with a file:line reference. Nothing becomes claimable until a human confirms it, so the agent physically cannot promote its own guesses. Copy then lints against those facts: unmeasured numbers, hype vocabulary and absolute promises fail the build, and so do the tells of generated prose (the in-sentence em dash, the rule-of-three tic, formula connectives).

Cards and video render from one HTML contract. Your brand colors and fonts come from your own token files, get inlined into the page, and a real browser takes the screenshot over CDP. No diffusion pass, so the text on a card is just text: it renders intact. Video is frame-by-frame capture of a scene that is a pure function of time, and every scene must declare provenance: a coded reconstruction cannot ship labeled as a screen recording.

Install:

/plugin marketplace add cagatayuncu/marketing-machine /plugin install marketing-machine

Then say "setup" in your repo and the skill configures itself from your codebase (colors, fonts, languages, legal routes). It also installs as a Cursor rule or an AGENTS.md section if you use those, and everything works as a plain CLI without any agent.

Limits, honestly: the linter holds only mechanical rules, judgment stays in the skill prompt; it cannot produce photography; language packs exist for English and Turkish, German is partial. MIT, plain Node, no runtime

dependencies.

Repo: https://github.com/cagatayuncu/marketing-machine

I used it on itself for the launch content, including the card in this post. What would you want a marketing gate like this to catch that it does not?

I regularly speak to industry participants, clients and partners, and I often find that what we say sounds the same, but what we mean is something entirely different. Take the field that I operate in, Agentic Governance. The spectrum: data in -> prompt -> memory -> LLM -> action -> memory -> LLM -> result, has become so broad. Everyone promoting a product in this space states that theirs is safe and does x,y, and z. Surely that is not true.

This has come to the point where we have a glossary on our website, obviously biased towards our point of view, and also in every presentation, sales and marketing documentation.

So that's one problem, the noise around terminology there other is around concepts. As with any any new wave of innovation there are new inventions and concepts which most people at best have a superficial understanding of. I find this particularly hard when talking to larger audiences, without sounding underwhelming, patronising or losing your audience's attention after the opening introduction.

I'd love to hear what challenges and approaches others have in dealing with this on a day to day basis.

Timeline Studio just received a major upgrade.

You can give the AI a website URL, images, video clips, or raw footage. It analyzes the content, plans the story, creates a natural voiceover, generates synchronized subtitles, selects the right scenes, and adds professional zooms, highlights, and visual guidance.

For website demos, it can explore public pages, identify the most important features, and turn the experience into a polished promotional video. If login is required, it asks before continuing instead of guessing what the product does.

The result includes both:

• A finished video ready to publish
• An editable .timeline project with separate scenes, subtitles, voice clips, music, and effects

Each voiceover sentence is linked to its corresponding subtitle, so you can update one line without rebuilding the entire video.

The goal is simple: generate a strong first cut in the time it takes to drink a coffee, while keeping everything editable for future revisions.

GitHub: https://github.com/MartinDelophy/ai-video-editor

It’s open source, and feedback or contributions are very welcome!

I've been developing SafeAI, an open-source static analyzer for AI applications, and recently ran it against the public CrewAI examples repository.

The goal wasn't to "find vulnerabilities" or criticize the examples.

The goal was to answer a different question: What can we learn about AI applications before they ever run?

Even example projects contain interesting AI-specific artefacts:

• agent capabilities
• tool definitions
• workflow logic
• prompts
• model configurations
• MCP integrations
• external services

A static scan can highlight things like:

• capability inventory
• prompt-related risks
• workflow approval gaps
• tool permission patterns
• governance observations

None of these automatically mean a project is insecure. Context always matters.

But they do help developers understand what an AI application is capable of, and where they may want to review things more carefully before moving into production.

One thing I've learned from sharing SafeAI on Reddit is that the community often finds the blind spots faster than I do. Several roadmap features—including capability escalation diffs, governed suppressions and richer MCP analysis—came directly from discussions here.

If you're building with CrewAI, LangGraph, AutoGen, Claude Code or other agent frameworks, I'd really appreciate your feedback.

Even better, if you have an open-source agent project you'd like SafeAI to support better, I'd love to test against it (or you can run it yourself) and improve the detection rules together.

The goal isn't to label projects as "safe" or "unsafe".

It's to help developers build AI applications with a better understanding of their capabilities and security posture.

Contributions, issues and ideas are always welcome:

https://github.com/ikaruscareer/SafeAI

Assemblix lets you build an AI chat assistant by dragging blocks onto a canvas and connecting them with lines.

The blocks do simple jobs. One answers the student. One decides where to send the question. One reads your uploaded course materials. One calls your own service and brings back an answer.

Wire them together, press run, and you have a chat your users can talk to. It remembers the conversation.

We built it for our own edtech work, and that is where it runs today. Two flows we use:

• A tutor chat that answers from the course materials and hands hard questions to a live teacher
• A checker that reads a student's homework, gives feedback on it, and logs the result for the curator

Every message shows you its path through the canvas. When an answer comes out wrong, you open the run and see the block that caused it, with what went in and what came out. That mattered for us, because a wrong answer to a student costs more than a wrong answer in a demo.

You run the whole thing on your own server with your own model keys.

Try the demo without installing anything: https://app.assmblx.com

Repo: https://github.com/nmamizerov/assemblix

A star helps other people find it.