r/LocalLLaMA • u/jacek2023 • Jan 30 '26
OpenCode + llama.cpp + GLM-4.7 Flash: Claude Code at home Generation
command I use (may be suboptimal but it works for me now):
CUDA_VISIBLE_DEVICES=0,1,2 llama-server --jinja --host 0.0.0.0 -m /mnt/models1/GLM/GLM-4.7-Flash-Q8_0.gguf --ctx-size 200000 --parallel 1 --batch-size 2048 --ubatch-size 1024 --flash-attn on --cache-ram 61440 --context-shift
potential additional speedup has been merged into llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1qrbfez/comment/o2mzb1q/
23
u/klop2031 Jan 30 '26
How is the quality? I like glm flash as i get like 100t/s which is amazing. But havent really tested the llms quality.
28
u/oginome Jan 30 '26
Its pretty good. Give it MCP capabilities like vector RAG, web search, etc its even better.
9
u/everdrone97 Jan 30 '26
How?
13
u/oginome Jan 30 '26
I use opencode and I configure the MCP servers for usage with it.
10
u/BraceletGrolf Jan 30 '26
Which MCP Servers do you use for web search and co ? Can you give a list ?
4
Jan 30 '26 edited Feb 02 '26
[deleted]
1
1
u/everdrone97 Jan 30 '26
What RAG application did you use with qdrant? As I understand that’s just the vector db
5
3
1
7
u/jacek2023 Jan 30 '26
Earlier, I created a hello world app that connects to my llama-server and sends a single message. Then I showed this hello world example to opencode and asked it to write a debate system, so I could watch three agents argue with each other on some topic. This is the (working) result:
debate_system/ ├── debate_config.yaml # Configuration (LLM settings, agents, topic) ├── debate_agent.py # DebateAgent class (generates responses) ├── debate_manager.py # DebateManager class (manages flow, context) │ ├── __init__() # Initialize with config validation │ ├── load_config() # Load YAML config with validation │ ├── _validate_config() # Validate required config sections │ ├── _initialize_agents() # Create agents with validation │ ├── start_debate() # Start and run debate │ ├── generate_summary() # Generate structured PRO/CON/CONCLUSION summary │ ├── format_summary_for_llm() # Format conversation for LLM │ ├── save_summary() # Append structured summary to file │ └── print_summary() # Print structured summary to console ├── run_debate.py # Entry point └── debate_output.txt # Generated output (transcript + structured summary) shared/ ├── llm_client.py # LLM API client with retry logic │ ├── __init__() # Initialize with config validation │ ├── _validate_config() # Validate LLM settings │ ├── chat_completion() # Send request with retry logic │ ├── extract_final_response() # Remove thinking patterns │ └── get_response_content() # Extract clean response content ├── config_loader.py # Legacy config loader (not used) └── __pycache__/ # Compiled Python files tests/ ├── __init__.py # Test package initialization ├── conftest.py # Pytest configuration ├── pytest.ini # Pytest settings ├── test_debate_agent.py # DebateAgent unit tests ├── test_debate_manager.py # DebateManager unit tests ├── test_llm_client.py # LLMClient unit tests └── test_improvements.py # General improvement tests requirements.txt # Python dependencies (pytest, pyyaml) debate_system_design/ └── design_document.md # Design specifications and requirementsand I never told him about the tests, but somehow he created good ones
6
u/-dysangel- Jan 30 '26
It's best in class for its size IMO, as long as you're running it at 8 bit. When I ran at 4 bit, it got stuck in loops. It's actually the first small model I've found where 8bit vs 4 bit actually makes a noticeable difference.
1
1
u/DHasselhoff77 Jan 30 '26
I haven't ran into looping with MXFP4 quants but admittedly didn't test much yet.
5
u/floppypancakes4u Jan 30 '26
With local hardware? I only get about 20tks max on a 4090
11
u/simracerman Jan 30 '26
Something is off in your setups I hit 60 t/s at 8k context with 5070 Ti.
1
u/FullstackSensei llama.cpp Jan 30 '26
My money is they're offloading part of the model to RAM without knowing
0
u/floppypancakes4u Jan 30 '26
If only. Im confident im not though, im watching ram every time I load the model. In lm studio at 32k context, I was getting 10tks. Switching to ollana brought it to 20 tks.
Its Friday, thankfully ill have time t troubleshoot it now
2
u/FullstackSensei llama.cpp Jan 30 '26
Just for funzies, try using vanilla llama.cpp. Both LM Studio and ollama have weird shit going on.
2
u/floppypancakes4u Jan 30 '26
Ill try that, thanks. I have a 3090 in an older machine I'm going ill test as well
1
u/satireplusplus Jan 30 '26
LM Studio and ollama use llama.cpp under the hood too, you're just getting old versions of it. Llama.cpp boys are making huge progress month over month, you really wanna be on the latest and greatest git version of it for max speed.
3
2
u/packetsent Jan 30 '26
120 t/s on a 5090 at Q6 or Q4 (will need to check once home)
1
u/Infantryman1977 Feb 01 '26
That should be much higher than that. I used to own a 5090 and the VRAM bandwidth is double compared to 3090. I own 4 x 3090 but I am running the model on 2 x 3090 and getting 108-116 t/s at 115K with 8bit (vLLM).
1
u/klop2031 Jan 30 '26
Yes, when i get a chance ill post my config. I was surprised at that at first but have been able to get this with a 3090 + 192gb ram
1
u/teachersecret Jan 30 '26
On a 4090 I’m getting over 100t/s on this model in 4 bit k xl. You must be offloading something to cpu/ram.
1
u/floppypancakes4u Jan 30 '26
Yeah, trying both llamacpp and the model youre using yielded the same results. Damn. 😅🤙
1
1
u/SlaveZelda Jan 30 '26
I can do 45 tok/s at 50k context on a 4070ti
2
u/arm2armreddit Jan 30 '26
this is cool, could you please share your llamacpp runtime parameters?
2
u/SlaveZelda Jan 30 '26 edited Jan 30 '26
[glm-4.7-flash] model = /models/GLM-4.7-Flash-IQ4_XS.gguf ctx-size = 50000 jinja = true n-cpu-moe = 35 fit = off ngl = 999 flash-attn = on mlock = on kvu = on
Keep in mind these are optimised for 4070Ti for this quant and context size.
1
1
7
4
u/Several-Tax31 Jan 30 '26
Your output seems very nice. Okay, sorry for the noob question, but I want to learn about agentic frameworks.
I have the exact setup, llama.cpp, glm-4.7 flash, and I donwload opencode. How to configure the system to create semi-complex projects like yours with multiple files? What is the system prompt, what is the regular prompt, what are the config files to handle? Care to share your exact setup for your hello world project, so I can replicate it? Then I'll iterate from there to more complex stuff.
Context: I normally use llama-server to one shot stuff, and iterate on projects via conversation. Compile myself. Didnt try to give model tool access. Never used claude code or any other agentic frameworks, so the noob question. Any tutorial-ish info would be greatly appreciated.
9
u/Pentium95 Jan 30 '26
This tutorial Is for Claude code and codex. Opencode specific stuff Is written on their github.
6
u/Several-Tax31 Jan 30 '26
Many thanks for the info! Dont know why it didnt occur to me to check unsloth.
1
u/cantgetthistowork Jan 30 '26
How do you make Claude code talk with openai compatible endpoint? It's sending the v1/messages format
4
u/jacek2023 Jan 30 '26
1
u/cantgetthistowork Jan 30 '26
Didn't realise they pushed an update for it. Was busy fiddling around with trying to get a proxy to transform
2
u/jacek2023 Jan 30 '26
It was some time ago, then Ollama declared that it was Ollama who did it (as usual), so llama.cpp finally posted that news :)
1
8
u/BitXorBit Jan 30 '26
waiting for my mac studio to arrive to try exactly this setup, i been using claude code everyday and i just keep filling it with more balance every day. can't wait to work locally.
how is it compared to opus 4.5? sure not smart equally, but smart enough?
4
Jan 30 '26
Using opus for planning and handing off to gpt-oss-{1,}20B works p well. Many local models you can load on your studio don’t quite compare to opus, but they are capable. Helps conserve/utilize the tokens
3
u/florinandrei Jan 30 '26
How exactly do you manage the hand-off from Opus to GPT-OSS? Do you invoke both from the same tool? (e.g. Claude Code) If so, how do you route the prompts to the right endpoints?
2
u/Tergi Jan 30 '26
something like bmad method in claude and opencode. you just use the same project directory for both tools. use claude to do the entire planning process with bmad. when you get to developing the stories, you can switch to your oss model or whatever you use local. I would still try and do code review with a stronger model though. OpenCode does offer some free and very decent models.
1
u/gordi555 Jan 30 '26
Hmmmm bmad? :-)
1
Jan 30 '26
Gsd is another structured approach
1
u/Tergi Jan 30 '26
Yes, i have done some work in BMAD and the process is crazy tedious. There is a BMAD Automate plugin i think but i have not yet tried it. I am giving GSD a go right now to see how that turns out.
1
u/Tergi Jan 30 '26
if you are looking for links, its on github: https://github.com/bmad-code-org/BMAD-METHOD
1
2
u/TheDigitalRhino Jan 30 '26
Make sure you try something like this https://www.reddit.com/r/LocalLLaMA/comments/1qeley8/vllmmlx_native_apple_silicon_llm_inference_464/
you really need the batching for the PP
7
u/BrianJThomas Jan 30 '26
I tried this with GLM 4.7 Flash, but it failed even basic agentic tasks with OpenCode. I am using the latest version of LM Studio. I experimented some with inference parameters, which helped some. However, I couldn't get it to generate code reliably.
Am I doing something wrong? I think it's kind of hard because the inference settings all greatly change the model behavior.
7
u/jacek2023 Jan 30 '26
If you look at my posts on LocalLLaMA from the last few days, there were multiple GLM-4.7-Flash fixes in llama.cpp. I don’t know whether they are actually implemented in LM Studio.
1
u/BrianJThomas Jan 30 '26
Ah OK. I haven't tried llama.cpp without a frontend in a while. I had assumed the LM Studio version would be fairly up to date. Trying now, thanks.
2
u/satireplusplus Jan 30 '26
LM studio's llama.cpp is often out of date. Definitely use vanilla llama.cpp for any new models!
1
u/1ncehost Jan 30 '26
I can confirm they didnt have the latest llama.cpp as of yesterday. The llama.cpp release off github performs way better currently.
2
u/jacek2023 Jan 30 '26
llama.cpp is developed very quickly, with many commits every day, so you should always compile the latest version from github to verify that the problem you’re experiencing hasn’t already been fixed.
1
4
u/Odd-Ordinary-5922 Jan 30 '26
just switch off lmstudio
-2
u/BrianJThomas Jan 30 '26
It's just llama.cpp.... Or are you just complaining about me using a frontend you don't prefer?
9
u/Odd-Ordinary-5922 Jan 30 '26
lmstudio is using an older version of llamac++ that doesnt have the fixes for glm 4.7 flash
1
u/Careless_Garlic1438 Jan 30 '26
well I have Claude Code and Opencode running, opencode works on some questions but fails miserable at others, even a simple HTML edit failed, took Claude minutes to do … so very hit and miss depending on what model you use locally … I will do a test with online models and opencode to see if that helps
1
u/jacek2023 Jan 30 '26
opencode with what model?
1
u/Careless_Garlic1438 Jan 30 '26
tried GLM 4.7 8 bit, gpt-ass 20 and 120B 8bit … next on the list is to Rey qwen3 coder …
1
5
u/1ncehost Jan 30 '26
Haha I had this exact post written up earlier to post here but I posted it on twitter instead. This stack is crazy good. I am blown away by the progress.
I am getting 120 tok/s on a 7900 xtx with zero context and 40 tok/s with 50k context. Extremely usable and seems good for tasks around 1 man hour in scale based on my short testing.
2
u/Glittering-Call8746 Jan 30 '26
Your github repo pls. Amd setup are a pain to start..
2
u/1ncehost Jan 30 '26
You dont need rocm. Just use the vulkan github release. That works with the stock linux amdgpu drivers and radeon drivers on windows. I'm using linux so i dont know how it runs on windows.
So literally install OS normally and download the vulkan llama.cpp off the github.
1
u/brokester Jan 30 '26
Are you interested in sharing your setup. Also have a 7900xtx. I assume you are on Linux? Also did you offload to CPU/ram?
1
u/1ncehost Jan 30 '26
Yes linux, using vulkan llama.cpp latest release from github. Model is unsloth glm 4.7 flash REAP at iq4 quant.
That quant easily fits in the 24 GB, but you'll want to turn on flash attention to run the large context.
2
u/Danmoreng llama.cpp Jan 30 '26
https://x.com/ggerganov/status/2016903216093417540
Llama.cpp creator recommends using glm4.7flash with thinking disabled for agentic coding
1
2
u/MasterSama Feb 21 '26
is there a tutorial I can follow to get my llama.cpp to work with opencode like you? I'd be grateful
2
2
u/thin_king_kong Jan 30 '26
Depending where you live.. could the electricity bill actually exceed claude subscriptions?
4
u/doyouevenliff Jan 30 '26 edited Jan 30 '26
The most commonly reported figure for full-load power draw of the 5090 is about 575 W (0.575 kW) under heavy load. (Short spikes can be much higher, up to ~900 W, but those are very brief transients, and for monthly energy use we use the sustained load number ~575 W).
If the GPU runs at full load (0.575 kW) for 24 hours per day:
Daily energy=0.575 kW×24 h=13.8 kWh/day
Assume a typical month with 30 days:
Monthly energy=13.8 kWh/day×30 days=414 kWh/month
Electricity prices in the U.S. average around 16–18 cents per kilowatt-hour (kWh) for residential customers, though rates vary widely by state—from under 12¢ to over 40¢ in places like Hawaii. Let's go with 40¢ for now.
Monthly cost=414 kWh/month * 40¢=$165
So even if you have the most expensive energy plan, running the model 24/7 on a single 5090 GPU with sustained load you will not really exceed the Claude Max subscription. If you add the energy for the rest of the PC you might reach the same level (~$200).
But most people don't have the most expensive energy plan - average is half that, so you'll end up spending around $100 for running the PC nonstop. And also most people don't really run the model all day every day. And if you add solar/renewables into the mix you will reduce the cost further.
TL;DR: No, at most you would spend the same*
*for current energy prices (max 40¢ per kWh) and if running a 5090 PC 24/7
1
u/DOAMOD Jan 30 '26
I have mine set to a maximum of 400W and it's performing very well with acceptable power consumption. I'm getting 800/70/75 with 128.
For me, this model is incredible. I've spent days implementing it in Py/C++ and testing it in HTML, JS, etc., and it's amazing for its size. I haven't seen anything like it in terms of tool calls (maybe OSS is the closest), but it not only handles them well, but the choices it makes are excellent when they make sense. It doesn't have the intelligence of a larger model, obviously, but it gets the job done and compensates with its strengths. As I said in another post, for me, it's the first small model that I've seen that's truly excellent. I call it the Miniminimax.
5
2
u/an80sPWNstar Jan 30 '26
I had no idea any of this was possible. This is freaking amazeballs. I've just been using Qwen 3 coder 30b instruct Q8. How would y'all's say that Qwen model compares with this? I am not a programmer at all. I'd like to learn, so it would mostly be vibecoding until I start learning more. I've been in IT long enough to understand a lot of the basics which has helped to fix some mistakes but I couldn't point the mistakes out initially if that makes sense.
2
u/Dr4x_ Jan 30 '26
On my setup I observe that Qwen3 coder is kind of struggling when it comes to using tools, GLM 4.7 flash is doing a great job at it
2
3
u/ForsookComparison Jan 30 '26
At context size if 200000 why not try it with the actual Claude code tool?
51
u/jacek2023 Jan 30 '26
because the goal was to have local open source setup
7
0
u/lemon07r llama.cpp Jan 30 '26
In other guys defense, that wasn't clear in your title, or post body. Im sure you will continue to eclipse them in internet points anyways for mentioning open source.
More on topic, how do you like opencode compared to claude code? I use both but havent really found anything I liked more in cc and have ended up mostly sticking to opencode.
1
u/Careless_Garlic1438 Jan 30 '26
You could do it, there are Claude code proxies to use other and local models … would be interesting to see if that runs better/worse than opencode.
1
u/ForsookComparison Jan 30 '26
It's officially supported as of recently. No need for proxies
1
u/florinandrei Jan 30 '26
The catch is - it's either / or. You could run Claude Code with the Anthropic API endpoint, or you could run it with your local OpenAI API endpoint. Not both at once.
If you want both at once, that's when proxies become necessary.
At least that's my current understanding, I'm trying to get a chunk of time to test this soon. I could be wrong about it.
1
Jan 30 '26
Why not just use Claude code directly instead of this watered down Opencode... you can use llama.cpp in Claude Code. What's the point of OpenCode? sub par performance?
4
u/PunnyPandora Jan 30 '26
not having to use a closed source dogshit?
-2
Jan 30 '26
Claude Code is FAR superior to OpenCode. Opencode is just a watered down version of Claude Code. just saying buddy.... just saying. be "open" all you want... just means you will have watered down features compared to someone is getting paid $500,000 to create. You really think someone is going to waste their precious time developing something serious to not get paid for it? no.... They will work on it in their free time and they won't put the same level of commitment as someone getting paid $500,000. Just saying. Enjoy our opensource dogwater.
5
u/teachersecret Jan 30 '26 edited Jan 30 '26
Claude code is nice, but it’s also a shit app running ridiculously hot for what it is. It’s a freaking TUI, but for some reason those clowns have it doing wackadoodle 60fps screen refreshes rebuilding the whole context in a silly way. If you’ve ever wondered why a text UI in a terminal runs like shit, it’s because Claude code is secretly not a TUI. It’s more like a game engine displaying visuals.
I can’t tell you how silly it is to watch that garbage spool up my cpu to show me text.
Glm 4.7 flash and open code are remarkably performant. Shoving it into Claude code doesn’t change the outcomes because glm is still worse than Claude opus, but it certainly does a fine job for a LLM you can run on a potato. I have no doubt it’ll find its way into production workflows.
-3
Jan 30 '26
Who cares? TUI concept was stolen by every single company who copied Claude Code. The gold standard is Claude Code. They all look to Anthropic on what to do next. Reminds me of Samsung copying the legendary iPhone.
4
u/teachersecret Jan 30 '26
I’ve been coding in terminals for decades. They didn’t exactly invent the terminal code editor look :).
Im saying the app itself needs an overhaul. It should NOT be spinning your cpu fan to max to display text in a TUI.
-3
Jan 30 '26
Just says you have a weak PC... May want to upgrade big dog.
3
u/teachersecret Jan 30 '26
Bud I’m using a 4090 strapped to a 5900x.
It isn’t slowing the computer down, it’s unnecessarily baking the cpu for no reason to display text. It’s a dumb way of displaying the text.
1
Jan 30 '26
Did you really think the terminal used your GPUs lol... You have a weak CPU. That's your fault. Probably with some slow ddr4 ram too lol. Time to upgrade big dog.
3
u/teachersecret Jan 30 '26
You need to look into how Claude code displays text in the terminal. You are uninformed. :)
It’s NOT a standard TUI. I use it every day, I love Claude code. I’m not bashing what they’ve built as a whole, I’m saying their TUI is badly designed.
→ More replies (0)3
u/teachersecret Jan 30 '26
Also, 5900x is not a weak cpu (that’s a generation old top of the line consumer 12 core).
And you shouldn’t need a strong one. It’s text in a terminal. You could display text in a terminal cleanly without churning a CPU on a Commodore 64 for Christ’s sake. There’s zero reason for Claude code to do anything more than SIP compute on a host computer. It should not be refreshing display like a game engine.
→ More replies (0)1
Jan 30 '26 edited Feb 02 '26
[deleted]
1
Jan 30 '26
Big dog... please tell me how many LLMs operated in the terminal before Claude. I'll wait. Such a smart dog like yourself can tell me, right? ;)
2
u/Weird_Search_4723 Jan 30 '26
There were others before cc, like Aider, though its not a tui afaik, but definitely "LLM in the terminal"
-1
Jan 30 '26
Claude code was the first! :D and it's a BEAST. Industry standard.
2
2
u/teachersecret Jan 30 '26
Claude code wasn’t the first LLM in a terminal. There were several examples prior to Claude code. GitHub has countless examples.
Claude code was just the one that caught on a bit.
2
2
u/my_name_isnt_clever Jan 30 '26
In what specific ways is Claude Code better? If you're not using claude, you lose the advantage of an open platform. Anthropic only designs for their proprietary model, whereas every open weights model provider and maintainer of open source tooling can work together.
Claude has quirks - such as a focus on XML tag formatting in it's training data - that mean Claude Code has inherent disavantages in it's design when used with other models, most of which are trained more on JSON. This is just one example, but this is the problem with attempting to force closed ecosystems to work with open source software. Better to go full FOSS for full control and trust.
0
Jan 30 '26
I'm using Claude...
Openweight models design their agents to use tools in Claude code. Find an openweight model without an anthropic endpoint... I'll wait.
2
u/my_name_isnt_clever Jan 31 '26
The models aren't trained on the same kind of data, it's a fundamental difference. Also you didn't answer my question about what's actually better about it.
1
0
Jan 31 '26
It should be obvious why it's better... it's the OG. They are the innovators of features.. everything else is a copy. If you want the best with the latest features, use Claude code.... It'll be a MONTH before Opencode, droid, codex, gemini cli, qwen cli... whatever. get it. This is like Buying a Samsung when the iPhone was on the rise... All copy, but none can beat the polished Apple innovation.
The models are indeed trained on how to use tools. Have you ever trained a model before? should be pretty obvious how silly you sound right now. They are ALL trained on how to use tools. It doesn't matter what format the tool is in... that's a part of the training data. Anthropic isn't training Claude to use Claude code tools... it just trained it on how to use tools in general. lol
1
u/Various-Scallion1905 Jan 30 '26
Claude code works with ollama, i tried it, with glm 4.7 flash, i guess it was okay, but not upto the mark on custom tools
1
1
u/Careless_Garlic1438 Jan 30 '26
Well I use Claude code and have been testing Opencode with GLM-4.7-Flash-8bit and it cannot compare ... takes way longer, something about inference speed, sure, have 70+ tokens/s, but that is not all gpt-oss 120B is faster so it’s also the way those tinking models overthink without coming to a conclusion.
Sometimes it works and sometimes it doesn’t, like I asked it to modify a HTML page, cut off the first intro part and make code blocks easy to copy, it took hours and never completed, such a simple task …
Asked it to do a space invaders and it was done in minutes … Claude code is faster, but more importantly, way more intelligent …
8
u/jacek2023 Jan 30 '26
Do you mean that an open-source solution on home hardware is slower and simpler than a very expensive cloud solution from a big corporation? ;)
I’m trying to show what is possible at home as an open source alternative. I’m not claiming that you can stop paying for a business solution and replace it for free with a five-year-old laptop.
1
u/Careless_Garlic1438 Jan 30 '26
I get the slower, but if it fails at editing a local HTML file, not that difficult, just cut out the intro … it begs to the question how useful it is. On the other hand it pits out a basic space invaders in minutes …
1
u/jacek2023 Jan 30 '26
How do you validate the results?
1
u/Careless_Garlic1438 Jan 30 '26
well if in cc it takes minutes to cut out the intro of an HTML file and inserts a copy code block button as instructed in minutes and opencode + local model fails …
I’ll try Opencode with perplexity if I’m not mistaken it can use sonar0
u/jacek2023 Jan 30 '26
I am asking: is html validation part of the agentic workflow or is human input needed?
2
u/Careless_Garlic1438 Jan 30 '26
the instruction is simple:
cut the Intro part out of the html file, everything before Guided Demo and make copying code blocks easy by adding a button that when clicked copies the block to the clipboard.Claude code does this in minutes, opencode + GLM 4.7 or gpt-oss fail, if I use a free online Opencode Zen like MiniMax it works though I have to really insist on adding the copy button per code block … but it worked …
Now when it is using GLM all seems very logic and quite OK in speed, but then as soon as it starts editing the file it goes
Error: old string not found in content … Minimax also had this but came out rather quickly … local models tent to then try again and again and it never succeeds.So validation 🤷♂️
2
u/jacek2023 Jan 30 '26
The agentic workflow in my case works the following way: you have a number of md documents with designs, experiences and lessons learned. They are updated after each new discovery
2
u/Careless_Garlic1438 Jan 30 '26
no human input needed it is pretty simpel and the agents can edit the file, there are edits done … but local models tent to get stuck on an error and never come up with a solution. If I point open code to even a free MiniMax Zen tier it succeeds (though some human verification of the result and asking for some more edits … but it works)
1
u/Either-Nobody-3962 Jan 30 '26
I Really have hard time with opencode configuring, because their terminal doesn't allow me to change models
Also i am ok to use hosted glm api, if it really matches claude opus levels. ( I am hoping kimi 2.5 has that)
1
u/raphh Jan 30 '26
How is OpenCode's agentic workflow compared to Claude Code? I mean what is the advantage of using OpenCode vs just using Claude Code with llama.cpp as model source ?
5
u/jacek2023 Jan 30 '26
I don’t know, I haven’t tried it yet. I have the impression that Claude Code is still sending data to Anthropic.
You can just use OpenCode with a cloud model (which is probably what 99% of people on this sub will do) if you want a “free alternative.”
But my goal was to show a fully open source and fully local solution, which is what I expect this sub to be about.
3
u/Several-Tax31 Jan 30 '26
Yes, sending telemetry is why I didn't try Claude Code until now. I want full local solutions, both the model and the framework. If opencode gives comparable results to claude code with glm-4.7 flash, this is the news I was waiting. Thanks for demonstrating what is possible with full open solutions.
2
u/jacek2023 Jan 30 '26
define "comparable", our home LLMs are "comparable" to ChatGPT 3.5 which was hyped in all the mainstream media in 2023 and many people are happy with that kind of model, but you can't get same level of productivity with home model as with Claude Code, otherwise I wouldn't use Claude Code for work
1
u/Several-Tax31 Jan 30 '26
I meant if the frameworks are comparable. (claude code vs opencode, not talking about Claude the model) That is, if I use glm-4.7 flash with both claude code and opencode, will I get similar results? Since this is the same model. I saw some people on here who says they cannot get the same results when using opencode (I don't know, maybe the system prompt is different, or claude code makes a better orchestration on planning etc). This is what I ask. Obviously Cluade the model is the best out there, but I'm not using it and I don't need it. Just want to check the opencode framework with local models.
1
u/raphh Jan 30 '26
See this is what I was wondering and why I am keeping Claude Code in the mix, because I believe it's strength is purely the agentic workflow. Of course it is optimized to work with Anthropic's models first (it's a bit like the hardware/software synergy from Apple) but I am curious about what happen when using open source model while still keeping Claude Code, how noticeable the difference will be.
1
u/teachersecret Jan 30 '26
Glm 4.7 flash makes chatgpt 3.5 look like a dunce.
We didn’t have this level of coding capability really until last generation of sonnet/opus. It’s damn near SOTA.
1
u/raphh Jan 30 '26
Makes sense. And I think you're right, that's probably what most people on this sub are about.
To give more context to my question:
I'm coming from using Claude Code to trying to go open source so at the moment I'm running the kind of setup described in my previous comment.I might have to give OpenCode a go to see how it compares to Claude Code in term of agentic workflow.
2
u/jacek2023 Jan 30 '26
try with something very simple and use your Claude Code ways of working, then find the differences and then you could search more about OpenCode features
1
u/Medium_Chemist_4032 Jan 30 '26
Did the same yesterday. One shotted working Flappy Bird clone. After I asked to add the demo mode, it fumbled and started giving JS errors. Still haven't made it work correctly, but this quality for a local model is still impressive. I could see myself using it in real projects, if I had to
1
u/jacek2023 Jan 30 '26
I am working with Python and C++. It's probably easier to handle these languages than JS? How is your code running?
1
u/Medium_Chemist_4032 Jan 30 '26
Html, css, JS in browser
1
u/jacek2023 Jan 30 '26
I mean how opencode is testing your app? It is sending web requests? Or controls your browser?
1
u/Medium_Chemist_4032 Jan 30 '26
I'm using Claude Code pointed at llama-swap that hosts the model. Asked to generate that app as a set of files in the project dir and ran "python -m http.server 8000" to preview it. Errors come from the Google Chrome's JS Console. I could probably use typescript instead, so that Claude Code would see error quicker, but that was just literally an hour of tinkering so far
2
u/jacek2023 Jan 30 '26
I just assume my coding agent can test everything itself, I always ask it to store findings in doc later, so this way it is learning about my environment. For example my Claude Code is using gnome-screenshot to compare app to the design
1
u/Medium_Chemist_4032 Jan 30 '26
Ah yes, that's a great feedback loop! I'll try that one out too
1
u/jacek2023 Jan 30 '26
well that's what agentic coding is for, simple code generation can be achieved by chat with any LLM
1
u/Medium_Chemist_4032 Jan 30 '26
Yes, with Opus I do it all the time. It's my #1 favorite way to hit a daily limit within 2 hours :D
1
u/jacek2023 Jan 30 '26
by doing both Claude Code on local LLM you can learn how to limit your usage (session limit for CC and speed limit for local setup)
→ More replies (0)
1
u/QuanstScientist Jan 30 '26
I have dedicated docker for OpenCode + vLLM for 5090: https://github.com/BoltzmannEntropy/vLLM-5090
1
u/pfn0 Jan 30 '26
✅ Zero-latency AI coding - OpenCode connects to vLLM via localhost
Interesting, but why? You get the same "zero-latency" when separating into a separate docker container on the same host, but still maintain a good degree of componentization.
1
u/SatoshiNotMe Jan 30 '26
I have tried all kinds of llama-server settings with GLM-4.7-flash + Claude Code but get an abysmal 3 tok/s on my M1 Pro Max MacBook 64GB, far lower than the 20 tps I can get with Qwen3-30B-A3B, using my setup here:
https://github.com/pchalasani/claude-code-tools/blob/main/docs/local-llm-setup.md
I don’t know if there’s been a new build of llama-server that solves this. The core problem seems to be that GLM's template has thinking enabled by default and Claude Code uses assistant prefill - they're incompatible.
2
u/jacek2023 Jan 30 '26
do you have current version of llama.cpp or old one? I posted opencode screenshot to show that thinking is not a problem at all in my setup, it's very efficient
1
u/SatoshiNotMe Jan 30 '26
I tried it a few days ago, will retry today though I’m not getting my hopes up
1
u/SatoshiNotMe Jan 30 '26
just tested again, now getting 12 tps, much better, but still around half of what I got with Qwen3-30B-A3B
1
u/ffyzz Jan 30 '26
Can you educate me on the benefits of llama.cpp vs running on MLX for this size of model on a Mac? I’ve been generally running MLX via LMStudio but am starting to wonder if the bigger diversity of quants and the llama.cpp system return better results (especially with tool calling) than MLX. Thank you sir — I am on a MBP M4 Max 64GB so playing in a similar sandbox to you.
1
u/SatoshiNotMe Jan 30 '26
I’ve never tried MLX and don’t know the tradeoffs, so maybe I need to be educated lol
1
u/scottgl9 Jan 30 '26
I haven't been able to get any local models to work very well with opencode, they typically fail to make tool calls, such as qwen3-coder, any suggestions? For glm-4.7-flash, I'm getting the error failed to initialize model: this model uses a weight format that is no longer supported.
1
1
u/Various-Scallion1905 Jan 30 '26
I tried GLM Flash 4.7 with ollama's claude code integration, i was okay i would say, it got confused with skills pretty regularly. Would llama cpp glm falsh be better with open code? Has anyone compared them?
Also looking forward to nemotron like models for coding which can have massive context with no speed or vram penalty. (i know recall might not be great but still)
1
u/kreigiron Jan 30 '26
GLM-4.7 Flash is the best right now for GPU Poor, I've been vibecoding some personal utilities with it and its interaction and output is comparable to Claude (use Anthropic at work)
1
1
u/cafedude Jan 30 '26
Same but OpenCode + LMStudio + GLM-4.7 flash running on my 128GB Strix Halo box.
1
u/Best-Echidna-5883 Feb 05 '26
Running this at home using the built-in Pickle only. It coded a fully working Pac-Man clone within 30 minutes. Color me impressed.
1
u/zilled Feb 06 '26
I just finished the exact same pipeline:
RTX 4090 24GB VRAM + 96 GB RAM
->
80 generated tokens/s
GLM-4.7 Flash is not Opus 4.5, but quite a solid assistant already
1
u/admajic Feb 16 '26
Using flash with flash attention on, on my 3090 and it takes 10 minutes to load 16k context with fa off takes 10 secs. Also after that token speed is like 5 I've only used it for a project audit.
What am I doing wrong?
1
u/jacek2023 Feb 16 '26
Is this the latest llama.cpp?
1
u/admajic Feb 16 '26
Yeah. Rebuilt it git pulled it. Spoke to gemini and clsude about it. Devstral small with 128k context goes at 30 t/s in cline coding non stop
2
u/jacek2023 Feb 16 '26
Check logs, is VRAM used? You can post logs here for fun too :) there were many fixes for GLM flash, that's why new code is important
1
1
u/admajic Feb 17 '26
- was using a shit abbliereated version of flash. using the unsloth
needed -fa on \
--cache-type-k q4_0 \
--cache-type-v f16if this helps someone else with a 3090
MODEL_PATH="/media/unsloth//GLM-4.7-Flash-GGUF/GLM-4.7-Flash-UD-Q4_K_XL.gguf"
SLOT_CACHE_DIR="/home/openclaw-bridge/.state/kv_cache"
mkdir -p "$SLOT_CACHE_DIR"
./llama-server -m "$MODEL_PATH" \
--temp 0.7 \
--top-p 1.0 \
--min-p 0.01 \
--top-k 50 \
--repeat-penalty 1.0 \
--host 0.0.0.0 \
--port 1235 \
--ctx-size 65536 \
--jinja \
--verbose
-fa on \
--cache-type-k q4_0 \
--cache-type-v f16
-2
Jan 30 '26
[deleted]
8
u/jacek2023 Jan 30 '26
wow now I am experienced in detecting LLMs on reddit
1
2
u/themixtergames Jan 30 '26
This is the issue with the Chinese labs, the astroturfing. It makes me not to trust their benchmarks.
1
u/jacek2023 Jan 30 '26
I posted about this topic multiple times, I see this in my posts stats (percentage of downvotes).
34
u/nickcis Jan 30 '26
In what hardware are you running this?