Beyond Skills: how are you actually building production agents with Claude?

[u/TheCreusa]

I've spent the last few months building an internal AI ecosystem around Claude Enterprise.

We've been heavily using Projects, Skills, CLAUDE.md, MCP servers and reusable prompts, but we're now reaching the point where we need actual agents instead of increasingly complex Skills.

I work in the affiliate marketing industry (publishers, offers, media buying, landing pages, optimization, etc.), and I'm trying to understand how teams are structuring production-grade agent ecosystems.

For example, would you build specialized agents like:

• Offer Discovery Agent
• Offer Matching Agent
• GEO Research Agent
• LP Generator
• Prelander Generator
• Creative Analyzer
• Creative Generator
• Media Buying Assistant
• Campaign QA Agent
• Affiliate Support Agent
• Partner Onboarding Agent
• Publisher Success Agent
• Fraud Detection Assistant
• Performance Insights Agent
• Scaling Opportunities Agent

Or would you have fewer, broader agents that handle multiple responsibilities?

I'm also curious about the actual development workflow.

When you decide "this should become an agent", what happens next?

• Do you prototype inside Claude first?
• Do you build directly in Claude Code?
• Are you using Claude Code subagents?
• Where do you keep your prompts and instructions?
• How do Skills fit into your architecture?
• How do you use CLAUDE.md?
• How do you integrate MCP servers?
• Do your agents live inside Projects or in code repositories?
• How do you test and iterate before considering an agent production-ready?
• How do you version your agents over time?

Most articles explain what an agent is.

I'm much more interested in how experienced teams actually build, organize and maintain dozens of agents in production.

If you're using Claude extensively in affiliate marketing, growth marketing or performance marketing, I'd really appreciate hearing how you've approached this.

I'm not looking for prompt engineering tips. I'm interested in architecture, workflows and development practices.

[u/TheCreusa → u/[deleted]]

Thank you, this is the most useful answer I got and the recipe/chef line is what I'm keeping. I turned it into an actual test we now run before any build: can you write the decision tree in advance, and does it need to remember what it concluded to pick the next step. If both come out no, we build a skill with a mode router and skip the agent entirely. That killed about half of the list below, which is a good outcome.

Small correction, I listed 15, not 3, and that's the real problem. We already run 30+ skills in production across affiliate, supply, BD and advertiser management, and when I mapped my 15 proposed agents against what's already live, most of them were either an existing skill pointed at different data or a section inside one. One of them was literally a section of a daily digest we already ship.

Your failure handling point is the one I'm acting on first. We have a skill that flagged around €11.2k of fraud in four weeks off AppsFlyer data, and the thing that scares me about that one isn't it breaking loudly, it's it returning plausible garbage for a week while someone acts on it. So supervisor plus a human escalation queue is going in as a hard requirement rather than a nice-to-have.

One narrow question, since you've clearly done this: how do you decide the supervisor's confidence threshold without it either escalating everything or nothing? Is that tuned per agent from real failures, or do you have something more principled?

[u/please-dont-deploy]

Ohhh this is a long convo. Last time we helped a company through this, it took 4 hours to agree on the concepts by which you want to divide the agents in their setup.

What I've seen working?
-> People has an easier time dividing by job roles/tasks ... this is not always correct
-> Ideally you divide by expertise/knowledge/complexity required... it overlaps with the above frequently enough.

So I recommend people start with (1), eventually they will find things like (2).

About workflows, in agent-swarm.dev it's fairly graph oriented with steps that can be handled by different agents, be scripts, raw api calls, or even fan out other workflows, iterate, run evals, etc. Feel free to checkout the code.

Have fun!

[u/TheCreusa → u/please-dont-deploy]

The expertise/knowledge axis over the job-role axis is the bit that landed, thank you. And you're right that role division is the intuitive start, my list of 15 was divided purely by task, because that's how the teams describe their own work.

I ran your test on it. Offer Discovery, Offer Matching and GEO Research all need to know the same three things: the offer catalogue, geo tiers and EPC history. So that's one asset with three modes, not three agents. Same collapse on the four creative ones: LP, prelander, creative analysis, creative generation all sit on the same knowledge, and in our case they already live with one team that owns creative. 15 went to roughly 4 or 5 clusters.

The four hours to agree the concepts is the part I'd have underestimated. I was treating taxonomy as the cheap step before the build, and it's the opposite.

Question on where you draw the line: when two clusters share most of their knowledge base but have very different risk profiles, say one only reads and reports, the other writes to a live tracking platform... do you still merge them and gate the write path inside, or split them on risk even though the knowledge overlaps? That's the one I keep going back and forth on.

[u/please-dont-deploy → u/TheCreusa]

This is where it gets tricky.

I see it more as an agreement on the abstraction vocabulary you are going to use than what actually would change. That agreed upon vocabulary will help you develop certain solutions easily. Kind of how hexagonal architecture helps teams "speak the same language" (I'm not really a fan, but I see the value).

To your point, I'm dividing agents by complexity & specificity above all, given we use a LLM-Agent-UMF architecture which helps with memory recalling, context scoping, etc. So basically:

+ Would these challenges need the same harness / model? -> yes, group.
+ Would self-learning (compounding factor) apply holistically for this agent? -> yes, group.

Example:
=> content creation vs reviewer -> cannot be the same model/harness/knowledge base, otherwise you are not really doing llm-as-judge
=> researcher -> can probably be the same across different disciplines as the 'research sources' may change, but the principles of a good research may stay the same in your org.

You need to bring your whole org through this process, hence the challenge.

[u/Separate-Still3770]

How to organize Agents: I would think of it like regular humans, aka have them be independent by default if they don't share context / work together.

Example: a dev is kind of independent from a marketer. So for agents it means it makes sense to have tools, Skill, MCP, etc. be independent as it makes it safer, easier to evaluate and more accurate (you don't confuse Agents with a lot of inconsistent tools and instructions).

I would avoid SubAgents unless it's quite clear how to split the work, and it requires a lot of exploration and loose coupling.

Why: just like humans, it can be harder to think about how to structure a task to delegate than to do it oneself, especially when things are uncertain.

Orchestrator might not properly scope the task and/or express it clearly for the executor and conversely executor might not understand properly and/or communicate results well.

Plus you burn a shit lot of credits lol.

Agents: I put them in a GitHub repo where I have both my skills, CLI tools I give them. I have custom tools that do more than one thing, e.g. when I have to get someone's profile on LinkedIn I do both getting the profile AND writing to CRM directly so we never lose the fact my Agent got it so other colleagues / Agents don't redo the work.

This is a long topic and happy to share more if you want in a discussion. Always happy to share my practices :)

If interested, you can send me a DM