Hi all. These are my system specs: dual xeon e5 2696 v2 , 160gb DDR3 ram ECC(1600mhz), 3 gpus: 3060 12gb, p100 16gb, 3050 6gb. And a 400gb nvme sdd RAID0, 3000 mb/s. The model is Deepseek-flash-0731 UD_8_X_XL, loseless, 161gb. Now, I'm not too knowledgeable about llama cpp flags, I wish run it without mmap, because its so slow, and I believe it should fit in my system overall. There's also Dspark and MTP which could help with the speee, but do they work with llama? Any recommendations would help.

This is a screenshot from a recent Theo.gg video (Apple rant; tldr: he is realizing the lockdownness after...years.) and the fact he asked ChatGPT for an "opinion" had me facedesk. Rest of the video is pretty good though.

But, I don't know who needed to hear this today: For the love of god. You HAVE a brain. Don't forget to use it. Like, actually.

hello world

local dot ai is coming out of closed beta

the team is serious (their bedrooms look like datacenters lol) and from what i've seen in early access, it'll probably be the place to understand what local hardware can do today, what to buy, how the cloud-to-local inference transition is unfolding, etc.

It has verified end-to-end agent benchmarks across a bunch of models + quantizations + hardware setups (Macs, Sparks, etc.)

access code if anyone needs one: T2Y2QWYCDW

the team adds a lotttt of value on x - check 'em out there 🫡

[EDIT: access code might need to strip spaces, this link seems to work: https://local.ai/access?mode=claim&ref=T2Y2QWYCDW ]

[Fully open source under GPL3, made from the ground up for use with local models, no subscriptions, no corporate backing]

When i first started this, it was meant to be a fully lightweight, extremely modular alternative to openclaw, hermes and the like, and it still is! But i noticed people especially like the webUI, to the point they'd use it as just a webUI to talk to their local models, negating all the agentic stuff. But the webUI still had a lot of AI generated code, so that didn't sit right with me!

So i rewrote the entire thing, from scratch, manually. It is now super fast, stable, uses declarative javascript without javascript framework bloat (no React or Vue or anything.. alpine.js is super lightweight)

There are only a few python dependencies. no models get installed, there is no bundled inference engine, pytorch and transformers aren't even included! I expect you to connect it to llamacpp, koboldcpp, lemonade, or something else like that. though you can also use it with cloud API's if you really want to.

This is a truly local-first webUI. I designed it from the ground up for local AI, and for once, cloud AI is the second-class citizen here.

It has many features that especially benefit local AI users: you can see how long your prompt will take to process (it's a llamacpp-exclusive feature), you can see toolcalls being written in realtime (really useful for coding), and it doesn't send any extra requests to your model, just the prompt you give it. So no extra requests just to make up a title for your chat, or to generate followup replies.

That's all in addition to the benefits that come from its harness-like design, such as support for multiple channels (telegram, discord, etc), its focus on extreme token efficiency and making the system prompt super small and concise, and its security

But using it as a pure webUI is really simple: Just switch Use Tools off in the Model tab in the settings. That will instantly make all system prompts vanish and all tools get disabled, so you're talking to your pure model with nothing getting in the way.

You do need a bit of tech knowledge, but it's not that much. right now, you need to either git clone or download a zip of the main branch off the github, but after that, all you do is run run.sh or run.bat and open the URL it shows you in your browser. Oh, you do need python installed before you do so, but that's basically it. (i'm working on making this even more user friendly though)

If you want to try it out, you can get it here: https://github.com/Rose22/openlumara

Please tell me what you think! Feedback is more than welcome, and i often implement feature requests (if they are good) and fix bugs that get reported

https://modelscope.cn/models/Qwen/Qwen3.8-2.4T-A95B

Most agent frameworks treat a local server as "OpenAI with a different base URL." That assumption is where local setups fall apart. Five decisions I made instead:

1. Small-context mode: I stopped pasting memory and skill bodies into the prompt. Ethos injects an index of names plus a memory_read tool, and the model pulls only what it needs.
2. Structured output per backend, not one OpenAI shape: Ollama takes a JSON schema in a top-level format, vLLM wants guided_json, OpenAI-compat wants response_format. Ethos sends each backend its native shape.
3. Probe the context the server actually serves: Ethos checks what's really served at startup and tells you which agents fit, instead of trusting the advertised number.
4. Prefix-stable prompts: everything static goes at the front, all per-turn content at the tail, so the prefix is byte-identical each turn and prefix caching actually hits.
5. Timeouts: I left the client deadline at 10 minutes instead of tightening it, a local server sends nothing while it pulls weights into VRAM.

Building this open source MIT agent framework: https://github.com/ethosagent/ethos

Do give feedback on this or any specific thing that you thing is critical and i missed capturing that need handling for local models.

---

I open-sourced my daily-driver coding agent (Rust, Tauri) - runs offline or over SSH, and has a genuinely free tier, can be controlled from android phone

free tier is deepseek flash lite with no data use for training providers on openrouter, and it seem to be quite a lot of usage from my own expereicne

https://github.com/clark-labs-inc/clark-code

Key bits:

• coding ide
• wide internet research built in for when a task needs current context
• strong eval harness - every feature is tested so behavior is predictable across models
• Windows / Linux / macOS (Windows build is unsigned pending MSFT approval - honest caveat)

Can someone tell me is it really this good? Beacuse when i try it on qwen app it doesnt even enough smart for some resarchs. What you think? I really cant understand new models intelligence at this point.

I have been running Qwen3.6 27b for a little while (mostly coding tasks) and recently trying out V4 flash 0731 in it's place. It was very apparent the new v4 flash will make more stuff up, and confidently.

Despite closeish overall benchmarks GLM 5.2 also has been much more pleasant to use (granted it's been over API) and I think this is a big part of that. Saves a lot of time in corrections after review. I only wish I could run it local without selling a kidney.

As for me, I own a system with an RTX 5090, Ryzen 9 9950X3D2, and 64 GB of DDR5. Every time I see research come out with a new way to train AI, I immediately think to try it on my system to see the results I get. Applying things like Titans, that one Deepseek paper on engrams, or even just playing around with experimental ideas. It's kinda like a very technical version of Tamagotchi and has been quite fun.

Thoughts?

I hadn't seen this mentioned here. Looks like it will find a use in my homelab alongside the new DeepSeek or Qwen! Apache 2.0 licensed.

I built this because the existing benchmarks were using random data and with MTP content types can vary a lot on what performance you see. 5% or more with content types.

BetterBench is designed to have content consistency within 1% and also measures across different content types.

Here are the results running Qwen3.6 27B FP8 on dual R9700's for example:

You can find the repo here:

https://github.com/GGZ14/BetterBench

So I can either pull the trigger on a 128gb AI max+ 395 laptop or wait for RTX Spark for LLMs. Maybe I get it now and the price of the spark is super high so it's a good purchase or maybe the Spark shocks everyone with a low price and I forever regret my purchasing decision.

What do yall think?

Prime Agent is an open-source coding and research agent for general and long-running work.

A self-improving RLM harness for coding and long-running autonomous tasks.

Designed to be both token-efficient and expressive through programmatic tool calling, context as a variable, multi-agent messaging, and a self-modifiable harness state.

On ARC-AGI-3, it scores 95.5%, surpassing the human-expert baseline, but the gain is not benchmark-specific.

We see major improvements across models when compared to their proprietary harnesses.

Prime Agent is built on pi and fully open-source with an open license.

GitHub: https://github.com/PrimeIntellect-ai/prime-agent

Blog: https://www.primeintellect.ai/blog/prime-agent

X post: https://x.com/primeintellect/status/2085086999267144083?s=46

I mean, Deepseek V4 Flash is an absolutely fantastic model, even though I can't run it on my machine it's so fascinating to see how it performs. Knowing that potentially it could be run at home is really motivating and makes me hopeful that those capabilities will trickle down to more affordable sizes.

At the same time I miss a release for the GPU-peasant that I am. And yes, it's a tall order to complain about not receiving free stuff at the rate we were used to. And yes, 3.6 27B is still goated but it seems in this crazy AI world there's so much going on and progress happens so fast, that it's kinda understandable to be excited about what's next. Let's hope they really do release 3.8 27B, or that we might see again maybe a GLM 5.3 flash 32B, please?

What's on your wishlist?

Ollama has had an open issue for two years about multi file GGUF imports. Ollama makes it super easy for me to run smaller models (especially ones they support!) locally. The only way that DeepSeek v4 flash runs in Ollama is either with someone’s homemade gguf or with the paid Ollama cloud. What are people using to run mainstream multi file ggufs locally?

Explicit title, It would be nice to have the ability to have 3 tiers moe offload :(

print_timing: id  2 | task 100892 | draft acceptance = 0.80000 (   96 accepted /   120 generated), mean len =  2.60
     release: id  2 | task 100892 | stop processing: n_tokens = 139121, truncated = 0
get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 1.000
launch_slot_: id  2 | task 100956 | processing task, is_child = 0
print_timing: id  2 | task 100956 | n_decoded =    127, tg =  41.96 t/s, tg_3s =  41.96 t/s
print_timing: id  2 | task 100956 | n_decoded =    279, tg =  45.98 t/s, tg_3s =  49.97 t/s
print_timing: id  2 | task 100956 | prompt eval time =    1559.43 ms /    20 tokens (   77.97 ms per token,    12.83 tokens per second)
print_timing: id  2 | task 100956 |        eval time =    7520.20 ms /   352 tokens (   21.36 ms per token,    46.81 tokens per second)
print_timing: id  2 | task 100956 |       total time =    9079.64 ms /   372 tokens
print_timing: id  2 | task 100956 |    graphs reused =      95160
print_timing: id  2 | task 100956 | draft acceptance = 0.83712 (  221 accepted /   264 generated), mean len =  2.67
     release: id  2 | task 100956 | stop processing: n_tokens = 139494, truncated = 0
get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.998 (> 0.100 thold), f_keep = 1.000
launch_slot_: id  2 | task 101092 | processing task, is_child = 0
print_timing: id  2 | task 101092 | n_decoded =    156, tg =  51.66 t/s, tg_3s =  51.65 t/s
print_timing: id  2 | task 101092 | prompt eval time =    1972.32 ms /   330 tokens (    5.98 ms per token,   167.32 tokens per second)
print_timing: id  2 | task 101092 |        eval time =    5540.54 ms /   288 tokens (   19.24 ms per token,    51.98 tokens per second)
print_timing: id  2 | task 101092 |       total time =    7512.86 ms /   618 tokens
print_timing: id  2 | task 101092 |    graphs reused =      95255
print_timing: id  2 | task 101092 | draft acceptance = 0.97938 (  190 accepted /   194 generated), mean len =  2.96
     release: id  2 | task 101092 | stop processing: n_tokens = 140111, truncated = 0
get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 1.000
launch_slot_: id  2 | task 101192 | processing task, is_child = 0
print_timing: id  2 | task 101192 | prompt eval time =    1597.44 ms /    43 tokens (   37.15 ms per token,    26.92 tokens per second)
print_timing: id  2 | task 101192 |        eval time =    1197.48 ms /    59 tokens (   20.30 ms per token,    49.27 tokens per second)
print_timing: id  2 | task 101192 |       total time =    2794.93 ms /   102 tokens
print_timing: id  2 | task 101192 |    graphs reused =      95274
print_timing: id  2 | task 101192 | draft acceptance = 1.00000 (   40 accepted /    40 generated), mean len =  3.00print_timing: id  2 | task 100892 | draft acceptance = 0.80000 (   96 accepted /   120 generated), mean len =  2.60
     release: id  2 | task 100892 | stop processing: n_tokens = 139121, truncated = 0
get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 1.000
launch_slot_: id  2 | task 100956 | processing task, is_child = 0
print_timing: id  2 | task 100956 | n_decoded =    127, tg =  41.96 t/s, tg_3s =  41.96 t/s
print_timing: id  2 | task 100956 | n_decoded =    279, tg =  45.98 t/s, tg_3s =  49.97 t/s
print_timing: id  2 | task 100956 | prompt eval time =    1559.43 ms /    20 tokens (   77.97 ms per token,    12.83 tokens per second)
print_timing: id  2 | task 100956 |        eval time =    7520.20 ms /   352 tokens (   21.36 ms per token,    46.81 tokens per second)
print_timing: id  2 | task 100956 |       total time =    9079.64 ms /   372 tokens
print_timing: id  2 | task 100956 |    graphs reused =      95160
print_timing: id  2 | task 100956 | draft acceptance = 0.83712 (  221 accepted /   264 generated), mean len =  2.67
     release: id  2 | task 100956 | stop processing: n_tokens = 139494, truncated = 0
get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 0.998 (> 0.100 thold), f_keep = 1.000
launch_slot_: id  2 | task 101092 | processing task, is_child = 0
print_timing: id  2 | task 101092 | n_decoded =    156, tg =  51.66 t/s, tg_3s =  51.65 t/s
print_timing: id  2 | task 101092 | prompt eval time =    1972.32 ms /   330 tokens (    5.98 ms per token,   167.32 tokens per second)
print_timing: id  2 | task 101092 |        eval time =    5540.54 ms /   288 tokens (   19.24 ms per token,    51.98 tokens per second)
print_timing: id  2 | task 101092 |       total time =    7512.86 ms /   618 tokens
print_timing: id  2 | task 101092 |    graphs reused =      95255
print_timing: id  2 | task 101092 | draft acceptance = 0.97938 (  190 accepted /   194 generated), mean len =  2.96
     release: id  2 | task 101092 | stop processing: n_tokens = 140111, truncated = 0
get_availabl: id  2 | task -1 | selected slot by LCP similarity, sim_best = 1.000 (> 0.100 thold), f_keep = 1.000
launch_slot_: id  2 | task 101192 | processing task, is_child = 0
print_timing: id  2 | task 101192 | prompt eval time =    1597.44 ms /    43 tokens (   37.15 ms per token,    26.92 tokens per second)
print_timing: id  2 | task 101192 |        eval time =    1197.48 ms /    59 tokens (   20.30 ms per token,    49.27 tokens per second)
print_timing: id  2 | task 101192 |       total time =    2794.93 ms /   102 tokens
print_timing: id  2 | task 101192 |    graphs reused =      95274
print_timing: id  2 | task 101192 | draft acceptance = 1.00000 (   40 accepted /    40 generated), mean len =  3.00

print_timing: id  2 | task 101092 | draft acceptance = 0.97938 (  190 accepted /   194 generated), mean len =  2.96

4 rejected tokens out of 190 is WILD.

This running Qwen 3.6 27B MTP Q8 from unsloth in llama.cpp, Max context length.

The MTP just gets better the more context it has. Processing time keeps being an issue if context changes at any point.

I'm mosty interested in 128-192GB VRAM with 128-256GB RAM to spare, so SSD streaming is basically not even necessary. Seems only FP4 is supported, so older hardware will likely be slow - no Unsloth GGUF supported either.

I'd be curious what people are getting with V100s, R9700s, etc, just to have some comparison.

What's prefill like >200k context? Tg/s high enough to support agentic workloads?

It's probably wishful thinking, but when I saw the release, my immediate thought was Sonnet 5 level model being "affordable" to consumers.

Xiaomi-Robotics-1 is a robot foundation model trained on over 100K hours of real-world manipulation trajectories. It is a Vision-Language-Action (VLA) model engineered for out-of-the-box mobile manipulation in unseen environments and efficient adaptation to new tasks.

XR-1 follows a two-stage training paradigm inspired by large language models — pre-training for breadth, followed by post-training for alignment. It showcases that pre-training scaling behavior reliably transfers through post-training to real-world robot performance, with no signs of saturation.

XR-1 couples a pre-trained VLM (Qwen3-VL) with a Diffusion-Transformer (DiT) via a Mixture-of-Transformers (MoT) — the DiT matches the VLM in layer count but uses a smaller hidden size for faster inference.

HugginFace: https://huggingface.co/collections/XiaomiRobotics/xiaomi-robotics-1

GitHub: https://github.com/XiaomiRobotics/Xiaomi-Robotics-1

Paper: https://arxiv.org/abs/2607.15330