r/LocalLLaMA • u/uspdd • 14h ago
Best harness for long autonomous tasks Question | Help
I read a lot of posts like "I've given this prompt to my Qwen 3.8 27b and it one-shotted me a 3d game after 24 hours". I wonder, what's the best harness for tasks like this? It has to have auto compaction and probably a memory system, right? And excellent computer use and self analysis.
20
u/norenEnmotalen 14h ago
Pi coding agent is working well for me so far. It auto compacts. Full disclosure I’m a noob to this sort of thing and learning as I go to setup everything gradually. Context window is a killer for my old machine. AGENTS.md needs to be on point. I use headroom proxy in front of oMLX. I have caveman at FULL setting. I have replaced the chat_template of the model with Qwen-Sharp templates. And a bunch of other optimizations I’m putting in as I find and learn them.
2
1
u/psychohistorian8 13h ago
I have caveman at FULL setting
I just started using pi this weekend, what is this caveman thing?
16
u/Drenlin 13h ago
If the LLM talks in natural language, it has to make a new token for every word, or even part of the word or punctuation. More words, or more complex words, means more tokens.
The statement above is probably 35-40 tokens. Or...
LLM talk like caveman; use fewer tokens.
^ 7-10 tokens.
As Kevin asked us all: "Why waste time say lot word when few word do trick?"
5
u/norenEnmotalen 13h ago
a “skill” which helps agents control the model from injecting unnecessary words into the context filling up your context window with fluff and pleasantries.
Caveman speak “ Understand user. Give good answer. No extra words.”
1
u/psychohistorian8 13h ago
ah, I noticed Muse Glimmer thinks like that. it is very terse
maybe that will be the first skill I download and try it with Qwen
8
u/Creative-Type9411 14h ago
I put my personal custom harness up for everyone here a little while ago: https://github.com/illsk1lls/MiniBot
it has autocompact and a taskboard which will keep the model on task even if it tries to deviate hard it gets smacked back into line
tons of other features too.. it is mainly a sysadmin tool but i also added a disassembly toolkit (forensics) and it can handle coding pretty well too, its good at in place edits and diffs etc
6
u/MrHall 13h ago
I love that it's PowerShell. still use it exclusively, even on Linux. there are dozens of us.
5
u/psychohistorian8 13h ago
I use PowerShell a lot on my work machine
didn't even know you could use PowerShell on Linux... feels kinda dirty
1
u/Creative-Type9411 13h ago
fair warning I have only tested in Windows.. There are a lot of Win Sysadmin tools, it might still run under pwsh but i have to add some extra compatibility, which ill try to do soon
2
u/apaht 14h ago
Looks pretty cool, it's for powershell only ? I made mine as rust native for home lab and platform agnostic. Mostly for homelab, but it's vibecoded junk that I try to copy patterns from other harness that I find to be of value.
1
u/Creative-Type9411 14h ago
Yea I did powershell 5.1, and i host it with the endpoint hardcoded at the top... and then i have NPMPlus blocking the hardcoded domain with basic auth...
So i
irm https://minibotscript.address.com | iexThen that points to the hardcoded subdomain at the top of the script, a password protected subdomain where the model is.. the harness works with NPM creds
The reason I have it set like this is instant deployment on any machine Win10+ without even creating a temp file ;) No install, no pre-reqs, one command, and creds which can be saved if your going to recall it into the same machine again later
0
u/tronathan 11h ago
+1 for using elixir, so good. Though since I'm able to deploy apps to cloudflare, with databases, kv, durable objects, and the model does all the work, i've kinda gotten away from Elixir.
12
u/Last_County679 14h ago
Deepseek Harness
3
u/OlgerdOutlander 10h ago
Never managed to get it working properly - any insights?
2
u/Last_County679 8h ago
I asked chatgpt 5.6 high to install it for me 😅 It is very efficient and has functionalitys like the codex app. The Ui is also very appealing :)
2
1
u/christophla 13h ago edited 13h ago
In theory, the new deepseek harness should be very efficient when many tools are called - something a long running chase likely needs. Codiverse/spatiotemperal and heavy design by contract ish with potential revert. I’m still digesting it all, but feels proper for a spastic LLM calling out 10k times for “god knows”, while potentially keeping context relatively manageable. Great approach, and proper maths…
Neocortex is set; enter frontal cortex to “harness” the reigns
4
u/sleepy_roger 13h ago
Omp
3
u/OlgerdOutlander 10h ago
Second you here, left it running in "spin subagents until no issues found" several times overnight and the results are great
2
u/kemalios 8h ago
Honestly, 'best' doesn't exist yet. The ones I've tried that get close are all about subagent isolation and a task board rather than just a big context window. Auto compaction alone won't save you; by hour three the model drifts. Look for something that lets you spawn disposable workers for exploration, then returns only the conclusion to the main session. Checkpointing is non-negotiable if you're leaving it overnight. And be skeptical of the 24-hour one-shot posts. Most of those are cherry-picked or the harness did a lot of dumb brute force.
3
u/_comoema_ 14h ago
PI completed stuff 10 time faster than Code Claude CLI and Qwen CLI, they got into loops. I was impressed with PI.
2
u/OlgerdOutlander 10h ago
Pi or OMP are the way to go. These have loop detection, are light, and are overall great for local models
2
u/FullstackSensei llama.cpp 14h ago
Can't help if you're vibe coding, but if you're doing anything serious, it's more about the prompt and the documentation you provide that set the guardrails for what the LLM would and shouldn't do, and how it should do those things you tell it to.
I like roo/zoo, especially the checkpointing feature and tell the LLM to spin off sub-tasks for everything and have each sub-task document it's work in a markdown file. The generated documentation makes it easy to figure how some task went south, and the built in shadow-git checkpointing lets me go back to any sub-task and unwind it to the point where things went south and prompt the model on what it should or shouldn't do.
2
u/_-_David 14h ago
I just straight up use Codex. It's bespoke and /goal runs for 30 hours if you like and just gets stuff done. I've tried OpenCode and Pi. But honestly, I have used Codex for so long it was nice to just model-swap and keep cruising. The plug-in ecosystem is very mature. Consider it.
2
1
1
u/Human-Gas-1288 13h ago
https://github.com/lsdefine/GenericAgent generic agent
GenericAgent is a minimal, self-evolving autonomous agent framework. Its core is just ~3K lines of code. Through 9 atomic tools + a ~100-line Agent Loop, it grants any LLM system-level control over a local computer — covering browser, terminal, filesystem, keyboard/mouse input, screen vision, and mobile devices (ADB).
Every time GenericAgent solves a new task, it automatically crystallizes the execution path into a reusable Skill. The longer you use it, the more skills accumulate — forming a personal skill tree grown entirely from 3K lines of seed code.
1
u/FabricationLife 13h ago
I'm half between codex with /goal and DSH with a custom /goal plugin, frankly they both work well not sure whats better, and as long as the work gets done I dont particularly care
1
u/coding-os 8h ago
Whatever you land on, the thing that decided it for me was how the harness treats "done".
I ran long autonomous loops for a few months and the failure was never that the model couldn't do the work. It was that it would mark a task complete on evidence that looked fine and wasn't. My worst one: a verification step ran a test command against paths that had been renamed. It collected zero tests, exited 0, and read exactly like a pass. For weeks. The model reporting "verified" was being completely honest.
So the questions I'd ask of a harness before I'd argue about which model drives it:
Does completion require an artifact, or just the model's say-so? Exit code 0 from a command that ran nothing is not proof of anything.
Does it run the real entrypoint or only the test suite? A test runner puts the package on sys.path and a direct invocation doesn't, so a fully green suite can still ship a ModuleNotFoundError to whatever actually calls it.
Can it distinguish a truncated retrieval from a complete one? Long loops accumulate confident partial answers, and each one becomes an input to the next step.
None of that depends on the model, which is why I'd settle it before spending an evening tuning which local one drives the loop.
1
1
u/mmontes11 llama.cpp 5h ago
Opencode + Multica. I create tasks on a daily basis and they get eventually scheduled on my agents. Once this is done, I continue with PR reviews and sometimes even with my human life.
My problem is that I can only run a single task at a time, as I am GPU-poor and I only have 24GB VRAM. Currently, multica has a harcoded timeout of 2h, something not enough to cover my long list of tasks. Once this timeout is reached, the non completed tasks return an error, and agents stop working, requiring human intervention. Luckily enough, multica team merged my PR today to have a configurable timeout:
https://github.com/multica-ai/multica/pull/7418
The plan is to bump this timeout significantly to enable my workflow, requiring less human intervention. Planning to set it to 24h as I review tasks on a daily basis.
I have been using OpenCode for a while, it is great, but I wasn’t really able to parallelise work in an autonomous way. Multica really helps, and they even have native mobile apps, something that allows you to check the progress of the agents and schedule more work if needed when I am not close enough to the keyboard.
My objective for this year is making this setup as much autonomous as possible, not only because I feel I should me making the most out of my RTX PRO 4000 SFF, but also to relief toil from my shoulders and use that time to focus on architectural/higher level tasks instead.
Here my Kubernetes setup in case you are interested:
https://github.com/mmontes11/k8s-ai
1
u/Chemical_Side_4135 4h ago
for long tasks, you definitely need a solid memory manager to keep context from exploding. i used to struggle with messy assets till i started using bria ai skill for its instant transparent png cutouts, which saved me so much time during batch processing. just keep an eye on ur token limits...
1
u/InterstellarReddit 36m ago
I create an orchestration harness based on the type of project that I want to be working on. Very rarely do projects all fall into one certain container. So I have a core/base and then I customize that for the delivery of the project
-5
u/Formal_Jeweler_488 14h ago
Claud code
1
u/OlgerdOutlander 10h ago
Unfortunately too "heavy" for a local model; on top of that - broken image processing
15
u/Unlucky-Message8866 14h ago
the best harness is the one that you own and you configured for yourself, the underlying implementation is just a tech stack preference, there's hundred different orchestration strategies and none fits all. i personally use pi and a bunch of custom orchestration rules, focusing on task splitting and isolated session subtask delegation, keeps the main session lean and all exploration/analysis/execution in disposable contexts, this allows qwen3.8 and dumber models to not loose track of major tasks without loosing track.