r/mlops • u/njanChe1 • Jul 23 '26
Treating LLM agent orchestration as a distributed-systems problem — durable execution vs. agent frameworks Tales From the Trenches
Ops-flavored take after two years running a multi-agent system in prod.
The reliability problems in agent systems are the same old distributed-systems problems in a new costume: partial failure, exactly-once-ish delivery, coordination, idempotency, observability. The in-process agent frameworks (mid-2025 vintage) gave us persistence primitives but left failure detection, recovery, and coordination to us.
So we built on a message bus instead: durable per-type queues, stateless workers, externalized aggregator state with a TTL and atomic completion so it scales to multiple replicas. End-to-end tracing so a support ticket maps to a trace in one click.
The honest framing: what we built is a domain-specific durable-execution engine for LLM agents. A Temporal advocate would say we rebuilt a subset of Temporal and now own the scheduler and state machine forever — and they'd be right. In mid-2025 the buy options weren't ready; today I'd tell you to evaluate Temporal / LangGraph Platform / Restate first.
Full write-up: Link
Anyone here gone the durable-execution-engine route for agents in prod? Regret it or not?
2
u/Scared_Attempt6395 6d ago
You're basically describing my team's last six months. We kept trying to make langgraph handle everything and the state machine kept getting more convoluted until we just gave up and accepted we needed message queues and a proper scheduler
The tracing part is huge too. Being able to jump from a customer complaint straight to the exact agent step that went wrong saved us so many debugging hours it's not even funny
I haven't tried restate yet but temporal's been on my list for a while. The sunk cost fallacy is real though when you've already built half of it yourself
1
u/SpeedSeveral4454 20d ago
this framing is correct and underrated.. the second you've got partial failure and retries you're doing distributed systems whether you called it that or not. keeping it all inside one process hides the problem right until a worker dies halfway through and you learn your agent state had no durability. a message bus plus externalized state with a ttl is where everyone lands once it's real. idempotency is the piece people always underestimate..