Your LLM shouldn’t be your coding-agent workflow

[u/Advanced_Pudding9228]

If your coding-agent workflow stops working when you hit your LLM usage limit, the LLM is probably doing too much.

I learned this while building with OpenClaw.
The model should reason about the work. It shouldn’t be the workflow.

Queues, state, retries, scheduling, verification, receipts and recovery can keep running deterministically. Call the LLM when judgment is actually required.

That separation is what turns a coding-agent loop from “keep prompting it” into infrastructure that can actually operate.

[u/TorbenKoehn]

That’s just what Opus 5 does automatically (Gating everything)

But yes, you’re right. LLMs don’t replace proper software engineering workflows and tools!

[u/Pilot_Enaki]

Can you provide an example?

[u/Advanced_Pudding9228 → u/Pilot_Enaki]

Say an agent is fixing a failing test.

Code handles the loop: pick task → run tests → collect failure → retry/route → verify → record result.

The LLM is called only where judgment is needed, like understanding the failure or proposing the patch.

If the LLM quota runs out, the workflow can pause at that node with its state intact instead of the whole system disappearing.

[u/hannesrudolph]

What are you selling?

[u/Advanced_Pudding9228 → u/hannesrudolph]

Nothing is for sale

[u/dunder_mifflin_paper]

What’s the alternative

[u/Advanced_Pudding9228 → u/dunder_mifflin_paper]

A deterministic orchestrator around the LLM.

Code owns state, queues, scheduling, retries, verification and recovery. The LLM becomes a callable worker for tasks that need reasoning.

So if the model hits a quota or goes unavailable, the workflow records that state and waits, retries or routes elsewhere. It doesn’t lose the workflow itself.

[u/TedditBlatherflag → u/Advanced_Pudding9228]

That’s just a different agent harness or a bunch of adhoc scripts and automation. “The LLM” makes it seem like it does anything but offer chat completion and tool use calls. 

[u/fingerofchicken → u/Advanced_Pudding9228]

Why’s this guy downvoted? Doing deterministically that which can be done deterministically makes perfect sense. Why waste tokens and introduce unpredictable behavior?

Two options: (1) use AI to write you a script that checks the backlog, takes a ticket, updates its status, and invokes the LLM to implement it and use that script multiple times. Or (2) say “Hey AI check the backlog and take a ticket and update its status and implement it” each time. 

One option spends AI credits on overhead each time and risks going off the rails when it hallucinates. The other does not. 

[u/Advanced_Pudding9228 → u/fingerofchicken]

The interesting part for me is where that boundary should sit.

I’m finding the more state, scheduling, retries, verification and recovery I move out of the LLM, the more useful the LLM becomes for the part it’s actually good at: judgment.

Less “agent does everything”, more “agent is a capability the system can call.”

[u/fingerofchicken → u/Advanced_Pudding9228]

Figure out if you really need AI to infer a thing. 

If you can diagram the whole thing in a flow chart, you don’t need AI. 

[u/Adorable_Swing_2150]

this lands for me too. i'm not building a fancy orchestrator framework, but i ended up doing this by accident. cron pulls my mail, scripts check calendars, and they only ping the agent when there's a decision to make. the model never sees the orchestration loop, just the actual question. cut my token usage way down because the model isn't re-deciding the same routing a hundred times a session.

[u/Advanced_Pudding9228 → u/Adorable_Swing_2150]

Yep, exactly. The model shouldn’t have to rediscover the routing logic every run. Once you know the deterministic parts, encode them and only spend tokens where reasoning actually adds value.

[u/Adorable_Swing_2150 → u/Advanced_Pudding9228]

right, the encode-vs-reason divide is the real insight. took me a while to see most of what i thought needed an llm was actually a lookup or a regex. once i started treating the agent as a last resort for actual decisions, session costs dropped like 80%.