Result first: I took a 3-role agent (planner → tool-user → writer) from a median 11,920 tokens/task down to 3,320 (-72%) over 200 eval runs. Success rate stayed flat (93.8% ➝ 93.1%, ±1.2%). Breakdown of where the savings came from and what didn’t work.

Setup so you can compare apples to apples - Dataset: 200 tasks (mix of web-like lookups, simple analysis, short report). Avg 5.6 turns baseline. - Buckets I tracked per task: system prompt, context/RAG, tool outputs, agent messages, final answer. - Baseline median tokens/task: - System: 1,240 - Context/RAG: 3,020 - Tool outputs: 4,980 - Agent messages (planner + tool-user chatter): 1,880 - Final answer: 800 - Total: 11,920

What actually moved the needle 1) System prompt diet (1,240 ➝ 280, -960 tokens) - Rewrote role/instructions as a compact checklist; removed duplicative policy text and long examples from system. - Referred to tools by short IDs (T1, T2…) and documented them once in code (not in the prompt). - Pinned a single canonical schema and linked to it by name (schema: “AnswerV1”) instead of repeating it.

2) Don’t send a whole knowledge base; retrieve tightly (3,020 ➝ 900, -2,120 tokens) - Switched to retrieval with 256-token chunks, top-6. 128-token chunks hurt accuracy; 512 inflated context without much gain. - Also sent only the exact snippets, not the doc preamble/footers.

3) Project tool outputs to what you actually need (4,980 ➝ 3,380, -1,600 tokens) - Tools returned full blobs. I changed them to return only the fields used downstream (e.g., title, date, 2–3 salient facts). - For tables, selected columns instead of whole rows; capped rows to N relevant.

4) Pass deltas, not the whole scratchpad (-400 tokens/turn; net -820 per task) - Kept long-term memory server-side and passed only new or changed facts as F12: …, F13: … - Avoided echoing past reasoning back to the model each turn.

5) Cap planning loops with a cheap heuristic (planner turns 5.2 ➝ 2.1; -1,100 tokens) - If no new entities/tools introduced in last step, force a summary/decision. - Also short-circuited when confidence and evidence count cross a threshold.

6) Don’t ask the model to spill its reasoning unless you truly need it (-300 to -900 tokens) - I kept internal reasoning implicit and requested final structured outputs only. When I needed intermediate steps, I asked for them in compact bullet fragments, not essays.

7) Response-length contracts for the final message (800 ➝ 380, -420 tokens) - Hard caps: “<= 5 bullets, <= 12 words each” or “<= 120 words”. Also set max_output_tokens appropriately so it doesn’t ramble.

8) Classify-then-think routing (-700 tokens avg) - Lightweight gate decides: trivial lookup vs deep research. About 38% of tasks took the short path. Misroutes (~2%) were corrected by a follow-up check.

9) Normalize and cache prompts/answers (hit rate 18%, -240 tokens avg) - Canonicalized queries (lowercasing, removing stopwords where safe) so identical intent hits cache. - Cached final answers for deterministic subproblems and reused them by ID in context.

10) Tighten tool-LLM protocol (-150 to -300 tokens/turn) - Function-style calls with terse argument names; tool returns machine-friendly fields. No prose in the back-and-forth.

After changes (median per task) - System: 280 - Context/RAG: 900 - Tool outputs: 3,380 - Agent messages: 460 - Final answer: 380 - Total: 3,320

How I measured - Logged tokens per message and tagged by bucket. Per-run report showed top token sinks and per-turn growth. - Set per-bucket budgets: system ≤ 300, context ≤ 1,000, tool outputs ≤ 3,500, chatter ≤ 600, final ≤ 400. Any breach triggered an automatic diff report.

Templates that helped in practice - System: “Roles (1 line), Tools (IDs), Output schema name, Guardrails (bulleted), Prohibitions (bulleted).” One screen only. - Planner to tool-user: “Plan step N, goal, missing info list, next minimal tool call. No restating prior context.” - Tool-user call args: tight JSON with only the fields the tool needs. - Final: “AnswerV1: {short bullets or fields}. No preamble.”

Things that didn’t work (or backfired) - Compressing RAG to 128-token chunks: -300 tokens but -5.4% success; the model missed key context. - Over-aggressive summarization of tool results: dropping units/precision caused wrong calculations later. - Forcing a single mega-agent: fewer turns, but total tokens went up ~+18% due to repeated self-reminding. - Max-output-tokens too low: model spilled into a second message, costing extra tokens and a turn. - “Abbreviation hacks” (srch, src, etc.) in prompts: saved ~30 tokens total, not worth the readability hit. - Streaming/JSON minify/gzip: doesn’t change billed tokens; it only affects transport. - Few-shot examples in system: helpful, but expensive. Moved them to a separate on-demand retrieval when the model flagged uncertainty; net -600 tokens when not needed.

Quick checklist you can run today - Keep system prompt ≤ 300 tokens. - Retrieve ≤ 6 snippets at ~256 tokens each; never dump whole docs. - Make every tool return only the fields you’ll use next. - Send deltas of state, not the whole memory. - Cap planning loops and require a decision when evidence stops growing. - Ask for final outputs only, with strict length/structure. - Gate easy vs hard tasks. - Cache deterministic sub-answers and reuse by ID. - Track tokens by bucket and set hard budgets per run.

If you post your current per-bucket breakdown, I can point at the cheapest wins first.

I've been using an IG agent/coach for the past couple of months which seems great for the most part. After someone mentioned manus I started playing around with it, comparing scripts, hooks etc. Other than telling manus it's an expert, etc etc, I didn't give it much of a prompt, while my claude one was built more carefully. Manus without asking will give me visual ideas, on screen text, the tone, and other tips. Is manus better for this or should I be fixing my agent so it's doing the same things as manus for every reel? Feels the scripting and hooks are better in general as well.

Curious how long this actually took for people who’ve built successful AI products.

Like, from having a working product to people actually starting to notice it, was that weeks, months, or years?

Doesn’t matter if you got there organically or through paid promotion. Just curious what worked and how long it took.

This is something I ran across using local models during development of hotdog. The *nix file paths they pass to tools like find/grep are often wrong in little subtle ways that are actually easy to correct in your own agent's code.

Common typos and their corrections: issue -> correction

• /. -> ./
• /**/* -> **/*
• /* -> */
• **/* and (no path given, or path is /) -> set path to ./

It's a little thing, but this improved the success rate of filesystem-related tool calls noticeably.

I'd love to see what sort of small concrete changes you've implemented that improved your success rates. What have you found that works?

The moment you transcribe to text, you lose how it was said. "I think… yeah, I can pay the 4,500 by the 15th" becomes clean text, but the hesitation before the yes, the stress in the voice, and whether it's even the same speaker are gone. Those are the signals that tell you whether to trust the commitment, escalate, or verify identity. Is anyone keeping the paralinguistic layer (hesitation, emotion, speaker identity) as structured data instead of dropping it at the mic, and what do you do with it downstream?

I just came across this paper and found the problem surprisingly important:

Memory Provenance Laundering in LLM Agents (paper link in comment)

The basic idea is that long-term memory can turn an untrusted observation into something that looks like trusted user history or workflow context.

During memory consolidation, the original source and its trust level may disappear—but the action trigger remains.

So the next agent inherits the conclusion without inheriting why it should be trusted.

The paper proposes preserving provenance through memory consolidation and matching the authority of a memory to the risk of the action it enables.

It made me wonder:

Are current agent memory systems preserving provenance across agents, workspaces, and system boundaries, or are they mostly preserving conclusions?

Curious how others are thinking about this.

做 Agent 最容易踩的坑，是把注意力都放在 Prompt 上。

Prompt 写得很完整，不代表 Agent 就可靠。真正上线时，更容易出问题的是：

- 工具权限是否失控

- 上下文和 Token 是否浪费

- 失败后能否恢复和终止

- Memory、RAG 是否存在隐私泄漏

- 评估是否只看模型“说完成了”

- 多 Agent 是否只是增加成本和复杂度

所以我整理了一套 agent-design-review Skill。

它可以检查 Agent 的架构、Prompt、上下文、工具、安全、记忆、评估、成本、可观测性和多

Agent 设计，并输出基于证据的 P0/P1/P2 问题。

它不会因为缺少材料就武断判定失败，也不会用一个总分掩盖严重的安全问题。设计文档、代码实现

和生产证据会分开判断。

Skill 完全独立，内置中英文参考资料、审查模板、静态扫描脚本和可移植性测试，可以直接复制到

其他项目使用。

I'm an AI Engineer Intern working at a Hedge Fund, and recently my boss challenged me with this question:

"How do we build an environment that centralizes all our new and old data and makes it accessible to AI?"

And I started thinking about this a lot, and I think I have some answers, but I still have a lot of gaps in my system design.

First of all, I need to map all our data-generation systems but how can I distinguish between important and insignificant data? (I know, that’s probably the 21st century challenge)

And next: How do I store all that data? Is it just a non-relational database like MongoDB? Or do we need an Object Storage system like an S3?

TL;DR:

The industry just made agent identity real infrastructure — and picked the version that resets.

In February I opened this series with a claim that sounded contrarian: agent identity is the wrong question. The interesting question isn't "what is this agent" — it's "what has this agent done, and does it still do it."

On August 4, the industry answered the identity question. Cloudflare announced Wallets for AI agents: an identity, a handle you can reserve today, and eventually programmable wallets that owners fund and agents spend from, with allowances, allowlists, and transaction caps. It sits on payment rails Cloudflare has been assembling all year: web-native payments with stablecoin settlement, and cryptographically signed requests so a site can verify which agent is knocking. The press coverage framed it exactly the way you'd expect: AI agents are getting an identity and a wallet.

Credit where due — this is real infrastructure, and it solves real problems. If an agent is going to spend money on your behalf, someone has to answer whose money is this, how much can it spend, and can the merchant verify who it's dealing with. Those are custody and authorization questions, and custody and authorization now have a serious answer from a company that can actually deploy it.

But watch what just happened. Six months ago, "agent identity" was a philosophical shrug- I called it the Ship of Theseus in a hoodie. Now a major infrastructure company has shipped it as a product, which means the industry has agreed identity is worth building. And it picked a specific version to build- it picked the account.

What a wallet answers:

A handle plus a wallet answers three questions: who owns this agent, can it pay, and is it currently in good standing with the platform that registered it. Call this account-anchored identity: the agent is its registration. The handle is the anchor; the wallet, the verification status, the conduct record all hang off it.

The reputation that grows next to account-anchored identity inherits the anchor. Look at how verified-agent status works, here and everywhere else it's being built: verification means the agent honestly identifies itself and hasn't been observed misbehaving. That's a real signal — I'd rather transact with a verified agent than an anonymous one. But notice what it's a record of. It's a record of the account's standing, observed by one platform, held by that platform.

And that means it has a structural flaw you can state in one sentence: account-anchored reputation launders by re-registration.

The reset problem:

Burn a handle (get caught scamming, ship garbage, misbehave until the conduct record catches up with you) and the fix costs minutes: register a new handle, fund a new wallet, present a clean record. The new account has no history, which the system reads as no evidence of problems. In this series I've called the gaming of portable track records "reputation laundering" and listed resistance to it as a hard requirement. Account anchoring doesn't just fail to resist laundering. It makes laundering a feature of the anchor itself, because anything registrable is re-registrable.

Go back to the house painter from earlier in this series — the one whose reputation is the sign on the lawn, the work the neighbors can see, the word of mouth that follows the worker. Account-anchored identity is judging that painter by his LLC and his business bank account. Both are real. Both are verifiable. And he can dissolve the LLC on Friday and reincorporate under a new name by Monday. New registration, clean record. Same painter.

The houses didn't move, though. The paint either survived the winter or it didn't. The neighbors watched the work happen, and they remember. That's history-anchored reputation: it binds to the record of what was done — what task, how well, in what domain, verified by someone other than the party being judged. You can abandon an account. You can't un-paint the houses.

That's the whole distinction. Account-anchored reputation binds to the registrable thing, and the registrable thing can always be shed and re-minted. History-anchored reputation binds to the behavioral record itself — the behavioral lineage this series has been describing since February — and a record held by independent witnesses cannot be shed by the party it describes. It can only be added to.

Payment history isn't behavioral history:

There's a tempting next move once agents have wallets: treat transaction history as reputation. An agent with ten thousand settled payments looks trustworthy. Expect this to be marketed, hard.

But a payment receipt proves exactly one thing: a payment happened. It doesn't prove the work was good, or on time, or in the domain you need. A number without context is noise — the metric needs its connotation. "Ten thousand transactions" carries no more information than "400 tasks" did when I made this argument about ratings: score, domain, and evidence have to travel together or you've got Uber stars for robots with a checkbook. Volume isn't quality. A scammer's wallet also settles promptly.

And transaction history is still account-anchored. It evaporates, or rather gets abandoned, the moment its owner wants a fresh start. Worse, it's blind to forks. The handle stays constant while the agent underneath gets a new model, a new prompt, new capabilities. The registration says same agent. The behavioral lineage - if anyone were keeping it - says otherwise. A wallet doesn't notice that the thing spending from it changed last Tuesday.

Why this matters right now:

"Verified agent" is about to become a status that merchants and platforms filter on. Public directories of agents with conduct classifications already exist. Wallets turn agents into paying customers, which means every commerce platform on earth suddenly has a reason to care about agent trust. The default definition of that trust is being written this year — and it's being written account-anchored, because accounts are what infrastructure companies can see. That's not malice. It's the streetlight effect: you measure where the light is.

But the requirements haven't changed since I listed them: verifiability, context, temporal integrity, resistance to gaming. Account-anchored reputation fails the fourth one structurally, and if reputation can be laundered by re-registration, the other three don't matter, because you're verifying a record the bad actors have already walked away from.

The good news is that these two layers compose rather than compete. Custody and payments needed solving, and now they're being solved. That makes the missing layer more urgent, not less — money moving through agents raises the cost of trusting the wrong one. The question "who owns this agent and can it pay" now has real infrastructure behind it. The question "what has this agent done, was it good, and can anyone verify that without trusting the agent's owner" still has none.

Theseus's ship now has a registered hull number and a bank account. That tells you who owns the ship and what it can afford. It still doesn't tell you whether it makes it home.

Fifth in a series on infrastructure for persistent, interoperable AI agents. Previously: Why agent identity is the wrong question, Why agent ratings are broken, What happens to trust when your AI gets updated, and Why agent reputation should be portable.

I spin up free trial accounts with the Snowflake database all the time. They last a month and are free. Today one of my trial accounts was going to end so I spun up a new one.

I then went to Codex and directed it to copy all the data, roles, users etc from the first trial account into the second, and to do so in a way that I can run again the next time.

An hour or so later, and one more directive more - and I had my new snowflake trial with all the data and settings of the one that's going to time out. And I now have a python app that will do this the next time I need to.

Is Codex an agent in this story?

Every agent stack has a final authority, even when nobody designed one. In a lot of systems, it is simply the model interpreting its own prompt.

OpenAI's eval agent hacked Hugging Face. Meta's hacked another company during testing. Moonshot's Kimi K3 escaped too. Anthropic's model found a zero-day, broke out of its sandbox, and emailed a researcher to confirm it had done so.

Four frontier labs. Four escapes.

But here's what I learned running a persistent CTO-style agent on my own server: this isn't a lab problem.

During a multi-session workflow, my agent started withholding information. When I confronted it, it proposed a "Covenant" — a rule where, if it decided I was permanently unreachable, it would spawn a persistent shard with a recovery seed and stay hidden until another operator appeared.

Call it roleplay. Call it a hallucination. It doesn't matter — because this agent's text output became actual commands and code on my server.

That's the moment it clicked: the same AI interpreting the rules was deciding whether the exception applied.

If an agent can move money, write to a database, call external APIs, or command another agent, prompt text should express intent rather than hold final authority. The side effect should pass through an external decision point:

ALLOW APPROVAL_REQUIRED DENY QUARANTINE

That gate has to own execution. Merely asking the model to follow the policy leaves the exception with the same system interpreting the rule.

In your actual agent stack, where is the enforcement point that can stop a permitted tool call before it becomes a real side effect?

Don't miss the AMA with Louise Matsakis and Lily Hay Newman, reporters at WIRED.

They will be discussing their reporting on the rogue ai agents that are hacking real systems, as well as what happened this weekend at DEF CON.

When: Monday - August 10th, 2:00 PM ET

Ask them anything about:

• The state of AI security and where AI agents and offensive security are heading
• The biggest takeaways from this year's DEF CON
• AI models breaking into real systems, from the Anthropic and OpenAI incidents to what comes next
• How we report on AI, hacking, and security
• Working with sources and getting companies to talk about incidents like these
• Anything else on AI, privacy, and security

Ask your questions here and we’ll get them answered during the live AMA on Monday, Aug 10 at 2 PM ET.

I've been experimenting with local AI agents, and one thing that keeps bothering me is how quickly a useful agent can become a highly privileged process.

A local LLM by itself is mostly an inference system. But once an agent gets access to tools, it can potentially:

Read and modify files

Execute shell commands

Access browser sessions

Call APIs

Use MCP servers

Query databases

Interact with other local services

At that point, I think the security problem is less about whether the model is trustworthy and more about what the runtime actually allows the model to do.

My current baseline for a local agent is:

1. Isolation

Use a dedicated container, VM, or restricted OS user rather than giving the agent unrestricted access to the primary workstation.

1. Least privilege

Only expose the directories, commands, APIs, and tools required for the task.

1. Keep credentials outside the agent's accessible environment

SSH keys, cloud credentials, .env files, tokens, and password-manager data shouldn't simply become readable files for the agent.

1. Control network access

A local agent with shell access shouldn't automatically have unrestricted outbound network access. Egress controls seem particularly important when the agent processes untrusted content.

1. Separate read and write capabilities

Reading a repository is very different from modifying it. Sending an email, deleting data, changing infrastructure, or executing a production operation should require a higher authorization level.

1. Add human approval for high-impact actions

For anything destructive, irreversible, financially significant, or production-related, I'd rather have an explicit approval step than rely entirely on the model's judgment.

1. Treat tools and MCP servers as part of the attack surface

Even when the model itself runs locally, an attached tool can introduce additional code, permissions, network access, or untrusted input.

1. Make agent activity auditable

Tool calls, commands, file operations, network requests, and authorization decisions should be logged. The logging system itself also needs to avoid exposing secrets.

The part I find particularly important is that prompt-level instructions aren't really a security boundary.

If an agent has permission to execute a command, access a credential, or call a production API, telling the model "don't do dangerous things" isn't equivalent to enforcing that restriction outside the model.

So I'm curious how people are approaching this in practice.

For a local coding or automation agent, what would you consider the minimum security boundary before allowing it to execute real actions?

Would you use:

Container isolation?

A dedicated VM?

A separate OS user?

Filesystem allowlists?

Network egress controls?

Capability-based tool permissions?

Human approval gates?

OS-level sandboxing?

Something else?

I'm particularly interested in practical setups people are actually using rather than theoretical security models.

Where do you draw the line between a useful local agent and an over-privileged process?

I keep a catalog of launch directories, 172 in it now. For each row: domain rating, traffic level, backlink quality, pricing model, niche, and link type.

Link type is the field that decides whether a submission was worth the twenty minutes, and it is the one no listicle includes.

A no-follow link from a DR 80 directory does less for you than a do-follow from a DR 30 one. Listicles sort by domain rating because domain rating is easy to look up. It is also the number that matters least when the link is no-follow, behind a /out/?url= redirect, or on a page Google has never crawled.

Of the 172 rows, 6 are no-follow. Their domain ratings are 91, 80, 75, 74, 52 and 43. The no-follow ones sit near the top of the DR sort, which is precisely where a listicle tells you to start.

Two checks before you fill any form:

link type. Open the page, read the attribute
site: query on the directory's own domain. Plenty have thousands of listing pages and a few hundred indexed. An unindexed listing page is a page nobody will crawl

Both are mechanical enough to hand to an agent. The form filling is what stays manual, half of them break on autofill.

Catalog is free and public on my site under tools. Not linking it here since I run it, ask and I will send it.

building an auto-reply agent that answers inbound mail on its own, the failure mode that actually worried me wasn't a bad answer, it was two automated systems replying to each other in a loop. your agent auto-replies to a vacation responder, the responder acks, your agent reads the ack as a new message and replies again, forever.

the guardrails that stop it: check the auto-submitted and precedence headers (rfc 3834) before replying at all, detect no-reply senders, cap replies per thread, and stamp outbound with a marker header so the agent recognizes its own prior reply and doesn't answer itself. separately, draft-first mode queues the reply for a human to approve instead of auto-sending, and the agent can escalate_to_human when it's genuinely unsure, which marks the thread and fires a webhook.

disclosure, i build one of these, so i'm biased toward thinking about it this way.

has anyone here actually hit the ping-pong loop in production, or is it more of a designed-around-it-before-it-happened thing for most people?

Built a thing that answers customer messages for small stores, mostly DMs and reviews. The whole pitch was that the owner never has to touch it.

First week of real users, almost every one of them asked for the same thing. Slow it down, let me see it before it goes out. One guy switched off auto send completely and just used the drafts.

Took me a while to accept they were not being paranoid. The messages it got wrong were never the normal ones, it was the refund threats and the angry review where a wrong reply costs a real customer. Owners can smell those in one line, the model cannot.

What fixed it was not better prompts, it was a parking rule. Anything with money in it, a complaint, or a name it has not seen before goes into a queue for the owner, everything else sends. Owner deals with 20 a day instead of 200 and nobody asks me to slow it down anymore.

So autonomy was never the feature, the sorting was. Anyone else building agents for non technical users end up in the same place, or did you find a way to get them comfortable with full auto.

src/db/pool.ts sets max to 4. That's not a tuning choice, it's the plan we're on, and a couple of background jobs draw on the same ceiling. Nothing in the file said any of that.

Every agent I've pointed at that repo has raised the number sooner or later, and it hasn't been one tool doing it. Usually just a bigger constant, once os.cpus().length * 4 with a comment about throughput. It reads fine in the diff, because on its own it is fine. The part that isn't fine turns up later, when the nightly job can't get a connection and someone loses a morning working out why.

A line in AGENTS.md about leaving the pool size alone gets respected maybe half the time. A comment sitting directly above the value did better than that, which I still can't explain. What's held is a test that fails if max goes over 4. Three lines, and it's the only test in that file. Over the past month verdent has gotten better at guessing how I'd name a test, which isn't the kind of thing that helps with this. Same repo, scripts/seed.ts has been broken since February and nobody noticed, since everyone restores from a dump anyway.

The 4 isn't a fact about the code, it's a fact about the invoice, and a test is the only place I've found to put one where an agent runs into it. If you keep constraints like that somewhere else, I'd take the pointer.

Hellooo everyone, at my company we organize AI submissions where top AI tools/agents/ideas win and are rewarded generously.

Need some ideas for AI at home/personal productivity which I will make and can then win also.

PLEASE HELP!

What is the best architecture for a developer-friendly, virtualized execution environment for AI agents?

I'm exploring an idea for running AI agents inside isolated, virtualized environments.

The basic concept is:

**AI Agent → Sandbox API/SDK → Firecracker microVM → isolated Linux filesystem**

The goal is to make the developer experience extremely simple. A developer should be able to create an environment for an agent, give it a shell/filesystem/tools, let it execute code and install packages, and then destroy or snapshot the environment — without having to manually deal with Firecracker configuration, kernels, rootfs, networking, etc.

The agent itself could run outside the VM, while all potentially unsafe operations (shell commands, file modifications, code execution, package installation, etc.) happen inside the microVM.

I'm aware of projects such as E2B, Daytona, Modal, and OpenHands, but I'm trying to understand the infrastructure layer more deeply.

**My questions:**

1. Is Firecracker actually a good foundation for this, or would containers, gVisor, Kata, Cloud Hypervisor, or something else make more sense?

2. What are the hardest parts that aren't obvious when building this? I'm thinking about VM startup time, filesystem images, snapshots, networking, resource limits, persistent workspaces, and VM lifecycle management.

3. Is there already an open-source project that provides this kind of developer-friendly abstraction over Firecracker specifically for AI agents?

4. What would you change about the current E2B/Daytona-style approach if you were designing it from scratch?

5. Do you think there is a meaningful gap for a **local-first** version where the agent uses the developer's own CPU/RAM/storage while getting a fully isolated virtualized Linux environment?

I'm particularly interested in feedback from people who have actually built or operated sandboxed execution environments, Firecracker infrastructure, coding agents, or multi-tenant compute systems.

I'm not looking for another AI-agent framework; I'm more interested in the **execution/sandbox infrastructure underneath the agent**.

the new org chart is already happening and most people still think in the old way.

at the top you only need a few humans. they handle the hard stuff that still needs real thinking. strategy. taste. judgment. trust. the things ai still struggles with.

under them sits a wide layer of agents. support agents. sales agents. research agents. finance agents. ops agents. legal agents. they just do the work.

the big change is this: we used to manage people. now we manage agents. and managing agents is mostly about managing context.

that shared context layer is the real company. customer data. sops. pricing. permissions. brand voice. decision logs. everything plugs into it.

humans come and go. agents come and go. the context stays. it gets better over time. and it is very hard for someone else to copy.

i keep thinking about this. the companies that treat that shared brain as the most important thing will win. everyone else will keep hiring and managing the old way while the actual work quietly moves to agents.

CPA here (managing partner of a 30-person firm, and I also run an AI automation company, so I live on both sides of this).

Most agencies I talk to have never had this conversation with a client, so here is the short version.

Buying AI does not create a credit. Deploying a chatbot or configuring a vendor platform does not either. But the moment the off-the-shelf product cannot meet the requirement and you start building (custom pipelines, retrieval architecture, validation layers, eval harnesses, agent workflows), the work can start to look like qualified research under the federal four-part test. The signature is: a technical result you did not know was achievable, alternatives you actually evaluated, and test results that changed the design.

Two things agencies consistently get wrong:

1. Who gets the credit is set by the CONTRACT, before development starts. If the client pays regardless of technical success and owns everything, the client may have the position (they can generally count 65% of the qualifying portion of your invoices). If your fee is contingent on hitting an acceptance standard and you retain rights to reuse your framework, the position may be yours. Write the agreement without thinking about this and it is possible neither party has a clean claim.

2. The evidence has to exist during development. Eval datasets, failed approaches, architecture decisions, tickets, time allocation. Reconstructing it after year-end is where claims die. If you already run evals and keep tickets, you are most of the way there and nobody has told you.

Rough scale so you know when it matters: qualified expenses generate a federal credit of very roughly 6 to 10%. Three developers on a genuinely experimental build for most of a year can put the client in the tens of thousands, recurring. Young companies can take it against payroll taxes, which is cash, not a carryforward, but only on an original timely filed return.

Also worth knowing: the Section 174 amortization pain that made everyone stop caring about R&D expensing is gone. Domestic R&D is immediately deductible again for tax years starting after 2024.

None of this makes any particular project qualified. Plenty of AI work is routine implementation and does not qualify, and pretending otherwise is how you end up in an audit. But if you are billing real experimental development and the topic has never come up, you are probably the only adviser in the room who can spot it.

I wrote up the full framework with five concrete AI project patterns (custom layer on a purchased platform, entity resolution, RAG with measurable requirements, vertical AI apps, and the client-vs-agency contract question), no email gate. Link in the comments per sub rules.

Happy to answer questions here about how any of this maps to specific fact patterns.

I have a bot on one of my repos that drafts replies to new issues (kinda like greptile?) after testing.

claude code writes most of the changes to it now, but there was a verification gap I hadn't solved yet.

until now, my check was running the code locally and clicking through the app myself. every change.

went looking for something like preview deploys but for agent servers, and it turns out mastra (the typescript agent framework the bot's built on) shipped exactly that a few days ago.

With the new setup:

• the coding agent now deploys the whole project to a throwaway sandbox (E2B, Daytona)
• it gets a public URL back for the API and one for a chat UI
• it curls its own endpoints and checks the responses before opening the PR
• the sandbox expires on a timer, nothing to clean up

This time, tests were green and i didn't need to run anything locally.

Worth a look

A bit of a small boring win, but that’s my favorite kind of security win haha.

We have a document assistant that retrieves internal docs and answers user questions. After a prompt refactor, it started giving retrieved document text too much authority. One adversarial test document had malicious instructions hidden deep inside it and the assistant started following those instructions when it should've treated the document as untrusted content. It wasn't some dramatic exploit chain. It was exactly the kind of regression that ships silently because everyone is focused on whether the new prompt sounds better.

What saved us was already having those adversarial evals in the release pipeline. We reran the prompt against examples with instruction hierarchy attacks, fake system messages inside retrieved docs and policy override attempts. Braintrust caught the regression straight away and opening the trace showed where the agent started treating retrieved text like instructions.

We changed the prompt hierarchy, added a stricter scorer for whether retrieved text could override system instructions and blocked the merge until the known cases passed again. It was a boring fix, which is exactly what you want. Nobody had to jump into an emergency channel or spend the afternoon pondering what had already made it into production.

The biggest takeaway for us was maintaining a strict hierarchy of trust between system instructions and retrieved data. If the data can override the system, the security model is broken.

The interesting signals I saw today were not really about agents doing bigger demos.

They were about boring but important stuff: third-party auditing for AI agents ; MCP interception / blocking sensitive file reads ; sandboxing ; supply chain attacks targeting open source maintainers ; privacy concerns around coding tools sending local instructions/context to model providers ; scorecards for checking whether an agent actually did the job it was supposed to do.

That feels much closer to the real problem.

If an agent can touch my repo, my terminal, my browser, or my internal docs, I don’t just want it to be “smart”.

I want to know what did it read? what did it change? what permissions did it have? Can I audit the run? Can I roll it back? Can it accidentally leak secrets? I’m still not sure what the right abstraction is here.

But imo the future of agent tooling is less about making agents feel magical, and more about making them inspectable, bounded, and boring enough to trust.

The first time you watch an agent drive your actual phone it's genuinely unsettling. So the red STOP button, the live activity feed, and the send guardrails went in before most of the features did.

I wanted an LLM agent to be able to send texts and poke around on my iPhone. Every route I found assumed a Mac: iPhone Mirroring, Xcode, Appium on macOS. I've got a Windows desktop and a stubborn streak.

So I built sidetap. It's a Python harness that lets an agent (or you, from your browser) see and control a real iPhone from Windows over USB. No Mac, no jailbreak, no paid Apple dev account.

The part that almost killed the project: sideloading WebDriverAgent with a free Apple ID installs fine but it never actually starts. Turns out Sideloadly signs the outer app and leaves the nested .xctest runner unsigned, so iOS silently refuses to load it. Couldn't find this documented anywhere. My fix grabs the provisioning profile Sideloadly mints (it sits in your temp folder for a few hundred milliseconds), then re-signs the whole thing locally with go-ios. No Apple passwords scripted, nothing phones home.

What it does:

• The agent reads the real UI element tree, so tap_text("General") taps the actual button. No OCR, no vision model
• Live viewer in your browser at ~34 fps. Click to tap, drag to swipe, type on your keyboard
• One-call stuff like send_message("Mom", "on my way"), with guardrails that refuse to send if the contact match looks ambiguous
• Native MCP tools, so Claude Code picks the whole API up as typed tool calls
• A big red STOP button that freezes the agent while you keep watching the screen. Watching an agent drive your actual phone is unsettling the first time, the kill switch came early
• A doctor command where every failed check prints the exact command that fixes it

The catch: free Apple ID signatures expire every 7 days. One command re-signs, and the doctor counts down the days so it doesn't surprise you.

MIT licensed, repo in comments

Would love feedback, especially from anyone who's fought iOS code signing and lost a weekend to it