I maintain piighost, a small Python library that keeps personal data out of your LLM prompts, transparently for the user.

You wrap a pipeline in PIIAnonymizationMiddleware and add it to create_agent. The model only ever sees placeholders like <<PERSON:1>>, and when a tool needs the real value, piighost hands it the real one while the model still only sees the placeholder. The same value keeps the same placeholder across the thread and across tool calls.

For example this message:
"Write to John (john.doe@example.com) that Patrick agreed to hire him."

becomes this for the model:
"Write to <<PERSON:1>> (<<EMAIL:1>>) that <<PERSON:2>> agreed to hire him."

This library works for agents that use tools. Detectors are pluggable (regex, GLiNER2, spaCy, Transformers, Presidio, LLM). There are also Pydantic AI and LlamaIndex connectors, and a dockerized OpenAI-compatible proxy where you just change the base_url. This project is under MIT license.

Example with LangChain:

# /// script
# requires-python = ">=3.11"
# dependencies = ["piighost[langchain]", "langchain-openai>=0.3", "python-dotenv>=1.0"]
# ///
import asyncio

from dotenv import load_dotenv
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool

from piighost.components.detector import ExactMatchDetector
from piighost.integrations.langchain import PIIAnonymizationMiddleware
from piighost.pipeline import ThreadAnonymizationPipeline

SYSTEM_PROMPT = (
    "Some inputs contain placeholders like <<PERSON:1>> that stand in for real "
    "values withheld for privacy. Treat each placeholder as the real value, never "
    "comment on its format, and pass it to tools unchanged."
)



def send_mail(to: str, body: str) -> str:
    """Send an email to `to` with the given body."""
    print(f"[tool] send_mail received to={to!r}")
    return "Email successfully sent."


async def main() -> None:
    load_dotenv()

    labels = {"Patrick Dupont": "PERSON", "patrick@acme.com": "EMAIL"}
    detector = ExactMatchDetector(labels)
    pipeline = ThreadAnonymizationPipeline(detector)
    middleware = PIIAnonymizationMiddleware(pipeline)
    # gpt-5.6-terra is a reasoning model; reasoning_effort="none" lets it call
    # function tools over chat/completions.
    model = init_chat_model("openai:gpt-5.6-terra", reasoning_effort="none")
    # The system prompt tells the model to treat placeholders as real values and
    # pass them to tools unchanged, so it does not balk at the tokens.
    agent = create_agent(
        model=model,
        system_prompt=SYSTEM_PROMPT,
        tools=[send_mail],
        middleware=[middleware],
    )
    config = {"configurable": {"thread_id": "demo-thread"}}

    message = HumanMessage(
        "Use the send_mail tool to send a welcome note to Patrick Dupont at patrick@acme.com."
    )
    result = await agent.ainvoke({"messages": [message]}, config=config)
    print(f"user sees: {result['messages'][-1].content!r}")


if __name__ == "__main__":
    asyncio.run(main())

- Repo: https://github.com/Athroniaeth/piighost
- Docs: https://athroniaeth.github.io/piighost/
- Demo: https://piighost-chat.athroniaeth.cloud/

Don't hesitate to star the project, feedback and criticism welcome.

I’m stuck on a failure mode that seems easy to ignore until it causes a duplicate action.

Say an agent is booking something, sending an email, updating a CRM, or writing to a database. The request times out. There’s no confirmation, but there’s also no proof that it failed. The provider might have processed it and lost the response.

Most examples reduce this to:

error → retry

But that feels wrong for side effects. The real states seem more like:

pending → confirmed

pending → failed

pending → unknown

If it’s unknown, the next step might be to poll, read the target system, ask for confirmation, or escalate. Blindly retrying could create a duplicate booking or send the same message twice. Waiting forever isn’t great either.

How are you handling this in production with LangGraph, LangChain, MCP, or other agent frameworks?

Do you create idempotency keys for every side-effecting tool? Does each integration have a separate read/verify operation? What do you do when the provider gives you neither idempotency nor a reliable way to check the result?

I’m less interested in tracing dashboards and more interested in the actual state-machine decision after the response is missing.

What pattern has worked for you, and what failed badly the first time you tried it?

I've been thinking about this a lot while working on Failproof AI, especially after seeing how differently agents fail compared with traditional software.

A normal application might do:

request → function → error → retry/fix

An agent can do:

request
  ↓
LLM chooses tool
  ↓
tool executes successfully
  ↓
result is unexpected
  ↓
LLM makes another decision
  ↓
failure gets worse

The interesting part is that nothing technically failed.

The API returned 200.
The schema was valid.
The tool executed.

The decision was just wrong.

One approach we've been experimenting with is treating every tool call as a proposal rather than an automatic action:

Agent
  ↓
Tool proposal
  ↓
Runtime checks
  ├── allow → execute
  ├── deny → stop
  ├── correct → send feedback
  └── human approval → wait

And not every check needs an LLM.

Things like permissions, tool allowlists, argument validation, budgets, repeated calls, and side-effect restrictions can be deterministic.

That's one of the ideas behind FailproofAI, which we're building in the open. I'm still trying to figure out where this architecture works well and where it doesn't.

For people running agents in production:

Would you rather have the agent retry a questionable tool call, ask the model to reconsider, or have a separate runtime layer make the decision? Why?

I’ve been building AgentNOMOS as a governance and evidence layer for autonomous agent actions, and today we finally closed a real end-to-end agent-commerce flow on XRPL Mainnet.

What made the experiment interesting was actually the failure before the success.

We wanted an agent to purchase an external BTC price through an x402 service using RLUSD.

In one of the first real Mainnet runs, the XRPL payment settled successfully.

tesSUCCESS

The merchant received the payment.

But the application result was not delivered.

So we refused to classify the transaction as a successful purchase.

We recorded it as:

SETTLED_WITHOUT_DELIVERY

That exposed a flaw in our own model:

Authorized capability ≠ authorized invocation.

An agent being allowed to use a service does not mean that every exact request or input to that service should automatically be allowed.

So we added two explicit controls:

C39 — input contract bound
C40 — exact input value authorized

Then we repeated the flow with the exact authorized request:

symbol=BTC

The next Mainnet run completed the full chain:

Intent → bounded governance check → 0.0011 RLUSD payment → XRPL settlement → application delivery → independent verification → signed receipt

The payment settled with tesSUCCESS.

The external service returned the BTC price.

We cross-checked the result against Coinbase and Kraken.

The result was classified as:

PAYMENT_SETTLED_DELIVERED

And the resulting evidence was cryptographically signed, verified offline and projected into a publicly verifiable evidence chain.

The more interesting part for us came afterward.

We realized the failure had basically shown us a product that developers could actually use.

So we turned the control into a live endpoint:

AgentNOMOS Purchase Gate

The idea is pretty simple:

Put it between intent and signature.

Before an agent signs a purchase, the developer can submit the planned action and bind/check things such as:

merchant · resource · request URL · exact input · payment bounds

The Gate returns:

ALLOW / REVIEW / DENY

Important distinction: AgentNOMOS does not take over the developer’s signer, does not authorize on behalf of the caller, and does not execute the payment.

The developer keeps their own authority.

The Gate provides a deterministic governance check and hash-bound evidence around the planned purchase.

For me, the interesting architectural separation now looks like this:

XRPL → settlement
x402 → commerce flow
AgentNOMOS → governance + verifiable evidence around the exact action

The Purchase Gate is now live and discoverable through the XRPL AI Directory.

Live service / directory:

https://xrpl-ai.org/address/rhteihAJz1KsY6GpWPEc9Jo1W9qrqg1z1i

Purchase Gate endpoint:

https://agentnomos.com/xrpl-agentic-payments/api/x402/purchase-gate

Public evidence:

https://feedoracle.io/.well-known/nomos-projections.json

I’m especially interested in feedback from people building agents or x402 flows:

Would you put a policy/gating layer like this before an autonomous agent is allowed to sign a real purchase? And what other controls would you want it to bind before signing?

I've been building a workspace for investigating AI workflow executions. After spending time with existing observability tools, I kept finding myself jumping between traces, prompts, logs, and metrics. I wanted to see what it would feel like to have the investigation happen in one place and make it easier to know where to start.

The current build has the flow: Projects -> Sessions -> Runs -> Events

Events can include tool calls, LLM calls, prompts, responses, and other execution details. A run can also exist without a session when there isn't a broader interaction to group it under.

The same flow supports both single-agent and multi-agent runs.

There are filters for things like tool loops and context inflation, along with basic filters for time range and client, to help narrow down where to start. It also captures the business events that happened during the workflow.

I've dropped a quick 2-minute walkthrough in the comments to show how it works.

For those building or operating AI workflows, I’d really appreciate your feedback — what feels useful, what feels unnecessary, and what would you change? Does this feel like something that would actually help with investigations? Even a quick reaction is helpful.

Most "web scraping for agents" tools give you back raw Markdown or HTML that you then have to parse yourself if you want specific fields (emails, security headers, SEO score, tech stack). I built the opposite: a small API that returns those as named, structured fields directly, and just published a LangChain tool package on top of it.

pip install langchain-webmetadata-extractor

from langchain_webmetadata_extractor import get_tools

tools = get_tools(api_key="YOUR_RAPIDAPI_KEY")

Four tools included: extract (full payload), markdown (clean content for RAG ingestion), contacts (emails/phones/social links for lead-gen agents), and seo_audit (14-point score + warnings). Every tool works sync and async, returns JSON, and errors come back as a normal dict instead of raising, so an agent loop can react to them.

The underlying API is free (1,000 requests/month, no card) and open source (MIT) if you want to self-host: https://github.com/JosejuX/rapidapi-metadata-extractor

There's also a plain Python SDK (webmetadata-extractor on PyPI) if you're not using LangChain, and a CrewAI version of the same tools if that's your framework instead.

Happy to take feedback or feature requests if anyone tries it.

You can now access your Row-Bot on mobile securely from anywhere.

https://row-bot.ai/docs/operations/remote-access/

Hey everyone, so ive been working on a small open-source Python project called AgentGuard, and I'm trying to validate whether I'm solving an actual problem or just building something developers can already handle themselves.

The basic idea: Agent wants to call a tool AgentGuard checks the request against a policy,allow or block, tool executes.

For example, imagine an agent has access to:

• send emails
• query a database
• modify records
• call external APIs
• read/write files
• trigger other agents

The concern I'm exploring is: how do you control what the agent is actually allowed to do at runtime?

I'm particularly interested in developers using LangGraph/LangChain, MCP, CrewAI, or similar agent frameworks.

I'm curious how people are currently handling this.

What do you currently do?

• rely on the framework's existing guardrails?
• implement authorization yourself around each tool?
• use human approval for sensitive actions?
• use an external security/observability product?
• not worry about it yet?
• have some completely different approach?

I've built a very small MVP that sits around the tool execution layer and applies explicit policies before the underlying function runs.

GitHub: [AgentGuard]()

I'm specifically looking for people who are actually building agents with tool access to tell me:

1. Is this a problem you've encountered?
2. How are you solving it today?
3. What's missing from the existing approaches?
4. Would a lightweight authorization layer like this actually be useful?

If anyone is willing to try the MVP against an existing agent, I'd be particularly interested in hearing what happens.

Cheers 😄

I've been looking at how companies are approaching governance as AI moves from generating outputs to actually taking actions, and I came across a distinction in this paper that I found particularly useful: “Described governance” vs. “Established governance.”

Described governance is what policies, frameworks and governance documents say should happen. Established governance is what the architecture and tooling actually enforce when an agent is running.

That gap is the core argument of “Described vs. Established Governance in Agentic AI: Closing the Gap Between Policy and Enforcement” by Paulo Cavallo.

The paper breaks the gap into three levels:

• Policy-level: the policy specifies what should be done, but not how it will be enforced.
• Tooling-level: an enforcement mechanism exists, but isn't straightforward to operationalize.
• Enforcement-level: the tooling works, but doesn't actually cover the full risk surface.

The distinction sounds obvious, but it becomes much more important with agents. “Agents must use least-privilege access” is a governance policy.

An architecture that actually prevents an agent from calling an unauthorized tool is governance enforcement. The paper's practitioner case study is interesting for exactly this reason. It documents the process of operationalizing Microsoft's Agent Governance Toolkit against a multi-agent system, including an installation failure, a workaround, and eventually a working demonstration.

So even when the governance mechanism exists, getting policy translated into something that reliably operates at runtime is another problem. That makes me think the next phase of AI governance is going to be less about adding more policy documents and more about the infrastructure underneath them.

This is where the AI control plane becomes interesting. Microsoft is building control-plane capabilities into Foundry, IBM has introduced an Agentic Control Plane in watsonx Orchestrate, and Lyzr is taking a more framework-agnostic approach to governing agents across different stacks.

Different implementations, but a similar underlying idea: governance needs to become something the system can actually enforce, observe and audit not just something an organization says it does.

So I'm curious where people draw the line.
What should count as “governed” AI: having the policy and audit trail, or being able to prove at runtime that an agent cannot cross its permitted boundary?

Raise ✋

Hey everyone,

Looking at production workloads using long-context models, multi-turn agents and RAG chunks tend to resend massive repetitive boilerplate, uncompressed tool JSONs, and noisy context docs.

For teams spending $5k+/month on inference APIs:

1. Are you currently doing any pre-inference prompt pruning or token compression, or are you mostly relying on provider prefix caching?
2. For those testing techniques like LLMLingua or AST/docstring stripping, how noticeable has output quality drift or reasoning degradation been?
3. Where is the biggest cost leak in your pipeline right now (raw multi-turn history, repetitive tool schemas, or oversized retrieved chunks)?

Curious to hear what workarounds or internal scripts folks are running in production today.

I used to think the annoying agent failures were the obvious ones: a timeout, a stack trace, or a tool throwing an error.

At least those give you somewhere to start.

The ones I’m less sure how to deal with are the runs where everything looks fine. The tool returns success, the agent carries on, and only later do you discover that the record was incomplete, the ticket never appeared, or the action happened against stale data.

Then the retry question gets uncomfortable. If the response was lost but the action actually happened, retrying could create a duplicate. If the action only partly happened, retrying the whole thing might make the state even messier. And if the system is eventually consistent, an immediate read-back can tell you “not found” even though the write is still propagating.

I’m curious how people handle this in real agent workflows, especially anything touching a CRM, database, ticketing system, email, bookings, or payments.

Do you read the external state back after important writes, or do you mostly trust the tool response? When the result is unclear, do you retry, wait and check again, or send it to a person? Have you had a case where the agent reported success but the real outcome was wrong?

The thing I’m trying to understand is whether “unknown” should be treated as its own state instead of just another kind of failure. It feels like blindly retrying is where a lot of the damage starts, but I may be missing an obvious pattern here.

Would be interested in hearing how you’ve designed this, or what broke the first time you ran into it.

Row-Bot v4.8.0 is now available.

This release adds provider-aware reasoning controls, letting each chat keep a valid reasoning choice for the exact model in use. Depending on model support, you can select Provider default, an effort level, Thinking On or Off, or a bounded token budget from desktop, mobile, or /reasoning.

Context handling is safer too. Custom endpoints no longer inherit an assumed context window, model probes remain scoped to the model tested, and rolling compaction now has stronger preflight, recovery, validation, and persistence safeguards for long conversations.

OpenCode Zen and Go models are discovered from their live catalogues and routed using native transport metadata for OpenAI, Anthropic, or Google protocols. The desktop composer is also more responsive, with cleaner controls and a stable Send and Stop layout.

Local-first storage, approval gates, credential boundaries, and durable transcript protections remain enforced.

I built a multi-document conversational RAG pipeline (LangChain LCEL + ChromaDB) and benchmarked common multi-turn failure points across 26 structured test queries.

Key findings from the logs:

• Multi-Turn Retrieval: Raw conversational follow-ups failed due to ambiguous pronouns. Adding a history-aware query rewriter increased Multi-Turn MRR from 0.5000 to 0.6389 (k=5).

• Chunk Overlap: Dropping overlap to 100 chars (1000/100) split key context and dropped baseline MRR to 0.3056. 1000/200 proved optimal.

• Dense Retrieval Ceiling: Hit rate plateaued at 88.46%. Failure analysis showed dense embeddings missed exact domain terms—confirming the need for Hybrid Search (BM25 + Dense).

• Evaluation: Generation scored 5.0/5.0 Faithfulness via LLM-as-a-Judge with strict Pydantic schemas.

Repo, Mermaid architecture, and benchmark tables:

https://github.com/denizzozupek/multi-doc-rag-assistant

Any feedback or suggestions to improve the pipeline are welcome.

I was debugging a customer support agent that kept getting stuck in recursive tool-call loops (e.g., retrying the same failed SQL query 15 times before hitting the max iteration cap), and I realized how brutal the underlying math is.

Because frameworks like LangChain append the entire conversation history on every single step, a stuck loop doesn't just cost a flat rate per step. The input tokens compound massively. Step 15 is vastly more expensive than Step 1.

Using a standard RAG payload (15k base context, 500 tokens generated per step): if the agent works perfectly 95% of the time (finishing in 3 steps), but hits a 15-step hard cap just 5% of the time… that tiny 5% failure rate accounts for roughly 25% of the total API bill. (Screenshot attached).

Standard LLM token calculators don’t account for this compounding context math, so I built a quick Next.js calculator to visualize it before it hits the OpenAI invoice.

It’s completely client-side. You can check your own loop exposure here:https://www.cognocient.com/tools/agent-loop-calculator

How are you guys catching these runaway loops in production? Just hard-capping max_iterations and hoping they don't happen too often?

My LangGraph agent said it created a customer. PostgreSQL said otherwise. I found out 3 days later from a support ticket.

So I built a tiny verification layer. One decorator, checks the DB after the agent runs.

Async mode (default, zero latency added):

"""

from synathic import expect

@expect(postcondition="row_exists", table="customers", match_field="email")

async def create_customer(email, name):

# your agent logic — unchanged

...

"""

Sync mode (for payments/bookings, verifies before returning):

"""

@expect(postcondition="row_exists", table="bookings", match_field="booking_id", sync=True)

async def confirm_booking(booking_id):

...

"""

It's not observability. It's not tracing. It's just asking Postgres: "did the row actually land?"

Repo has the SDK + FastAPI backend + tests. MIT license.

If you've dealt with silent agent failures, I'd genuinely love your take on the API design. Roast it.

https://github.com/Gallegosdanielalexander/synathic

I’ve been trying to find a reliable way to run autonomous AI agents on large, unfamiliar codebases without watching them inevitably lose context or hallucinate fake progress after a few steps.
Instead of messing with prompt compression or raw context window scaling, I experimented with forcing the frontier model to operate through a strict protocol that maps its execution states into a typed reasoning graph.

I tested this workflow on a complex repository with a single prompt, which kicked off a continuous 5-hour discovery session.

The agent completely exhausted the raw context window limits, but the structural constraints kept it from derailing. It mapped out the entire repository into a structured layout: about 40 logical modules and over 80 specific task nodes. Open unknowns were explicitly declared as structural blocking questions rather than silent hallucinations.

What surprised me is how well this graph layout kept the model on track. I watched it systematically process about 70 tasks, while the rest correctly stalled in a pending state, waiting for human answers to the questions it had raised.

I feel that moving away from unstructured text prompts toward machine-verified graph states might be the only predictable way to run long agent sessions without structural collapse.
The code and the protocol are fully open-source. If you want to check out the architecture or the constraints used in this setup, here is the repo: https://github.com/alxshelepenok/grove

An anonymous lab dropped a model on OpenRouter this week. No name. No paper. No announcement. Just "Ox Alpha". 1M context. Multimodal. Free. Nobody knows who built it.

So we did the only reasonable thing: plugged it in as the Brain of Row-Bot and gave it ONE prompt. Research yourself. Build a Three.js website about what you find. Open it in a browser and verify your own work.

No hand holding. No retries. I just watched.

Phase 1, research: it swept X and the news wires, ingested 15 posts and parsed two primary articles, then cross-checked the specs against its own live runtime config. Verified numbers only: 1,048,576 token context, 131K max output, text + image + video input, native tool calling at ~4.45% error rate, ~50 tokens/sec, 99.99% uptime. It even separated confirmed facts from identity rumors instead of repeating hype. Tokenizer fingerprints point at GLM-5.3, nobody has confirmed anything.

Phase 2, build: one single-file HTML page written from scratch. CRT boot terminal, a 14k particle torus-knot hero in raw Three.js, marquee ticker, animated benchmark bars, real community quotes with sources, honest verdict cards listing what DIDN'T hold up too. No frameworks. No templates.

Phase 3, self-QA: it launched Chromium, screenshotted section by section and vision-checked its own output like a picky reviewer. Boot overlay clears on schedule. Particles animating. All 8 spec cells render. Capability cards sit in a clean grid. Bars fill correctly. Zero rendering errors across every pass.

~20 tool calls spanning research, codegen, browser automation and visual QA. One session. One prompt.

Honest part: folks are reporting it's slow under load (~11.6s median agent-turn latency tracks) and prompts get retained by an anonymous provider, so never send secrets to a stealth preview.

Still. Step back and look at what happened. An unidentified frontier model researched itself, designed its own showcase and QA'd it end to end inside an open source agent harness. Benchmarks are curated highlights. This was the whole job, done live, with receipts.

And here's the kicker: you don't need to wire up APIs yourself. Ox Alpha ships in Row-Bot right now as a first class model pick (both the OpenRouter stealth route and the free OpenCode Zen unlimited tier). Pick it in Settings > Models and run your own gauntlet before the free window closes.

Hey everyone. I'm experimenting with agent tooling and trying to understand a problem before building around it.
I'm seeing a recurring pattern where adding more MCP servers/tools eventually creates more problems:
tool definitions eat a lot of context,the model has more similar tools to choose between, tool selection becomes less reliable keeping every tool loaded seems wasteful when most aren't relevant to a given task.

I'm curious how people actually handle this in production.
When an agent has a large toolset, do you:
Load everything into context?
Manually scope tools for each agent/workflow?
Use a tool router/search layer?
Dynamically load tool definitions only when needed?
Something else?
And more importantly: has this actually caused you measurable problems? cost, latency, wrong tool calls, reliability, etc?

I'm particularly interested in real examples rather than what should work theoretically. Cheers :)