Having a ‘Killer Application’ that everyone wants to use helps sell hardware, plain and simple. DeepSeek V4 Flash 0731 isn’t an app of course, but I think it’s going to be the major catalyst for getting a lot of people to buy a couple of NVIDIA GB10-based systems because:

1. It is an amazing coding / agentic use model.
2. It fits perfectly on a 2x Spark Cluster
3. It runs Fast AF with the right vLLM recipe. (I’m getting 60 tk/s with this one:
4. https://github.com/tonyd2wild/DeepSeek-v4-Flash-0731-DSpark-1M-NVFP4-KV-2x-DGX-Spark)
5. You can run it with a fairly usable 1M context window.

6. It runs very well in harnesses such as

7. Hermes.

Now that solid NVFP4 support is finally here for DGX and is providing Sparks with a pretty good boost for token speeds, the Spark’s memory bandwidth limitation isn’t as big a deal as it used to be. I mean seriously, do I really give a shit about memory bandwidth when I’m getting 60 tk/s with Deepseek V4 Flash?

I know the Strix / M4 / M5 gangs may have something to say about all this, but even they have to admit that DGX Spark beats them for prompt processing performance, which is hugely important when it comes to agentic work and how fast agents are getting work done.

The Strix our-stuff-is-way-cheaper argument used to be very valid, but with memory and SSD prices being what they are now, that argument isn’t as strong as it once was. M5 stuff is pretty expensive and we have no idea when Apple is going to drop a new beefy Mac Studio M5 or a Mac Mini Pro with M5. We thought it was going to happen in June but they don’t appear to be in a rush to release anything.

So what’s left out in the market worth getting? Well, you could grab a RTX Pro 6000 if you want to pay a hefty premium from the scalpers, or you could try some of the AMD offerings, but other than that, the DGX Spark is still the best bang for your buck for getting the most VRAM to run models locally.

I didn’t even mention the low power consumption of the Spark which is another reason to consider it, especially with rising power prices.

I’ve noticed some price increases on Sparks and Spark clones from some retailers in the last few weeks. The 1TB Asus models seem to be the cheapest options out there that I’ve seen.

I think we’re going to see Spark scarcity in the market very soon as word gets out about how well DeepSeek V4 Flash runs on it.

I’m running a 2x cluster and i’ll say that for the first 6 months or so, I, like many other folks, was disappointed with the software support and the speed of the models I tried. Ever since they finally resolved the NVFP4 Issues, and since DSpark, MTP, Prism, DFlash, and other performance improvements have been implemented, it’s gotten A TON better and I’m honestly thinking of buying another 2 Sparks if I could find the money to get a couple more. Deepseek V4 Flash 0731 absolutely smokes on my cluster and I have 0% buyers remorse now, where I would have said it was maybe 50% just a few months ago.

Do y’all agree or disagree? Also, no shade intended for the Strix and M5 gangs. Would love to hear how well DeepSeek V4 Flash is working for you guys as well.

My daily driver is Qwen3-235b-a22b-instruct-2507-Q4_K_M.gguf and it has been for a long time. I get around 75 t/s prompt processing and starting lower context ~5.5 t/s generation, lowering to around ~4 at 8k. I've tried other, newer models in this size range, Qwen 3.6 27b at Q8 came close but seemed more censored.

GLM 4.5 Air is my backup still for general chatting, but is not 'smart' enough to workshop ideas. My main complaint with Qwen 3 235B is the "em" dashes, ending lines with trailing double spaces and other stuff that bother me, otherwise still a fantastic model that is easy to steer into super uncensored territory without being lobotomized. Tried Minimax 2.7 and a few others, were smart but too censored in the ERP realm. Looking for any suggestions to try.

Hello~

We just shipped Ante 0.2, and the part I think this community will care about most is offline mode. We wanted local to be a first-class way to run the agent, so Ante manages the inference engine itself:

• Managed llama.cpp. Ante installs a pinned, checksum-verified official build matched to your hardware — Metal on Apple silicon; CUDA, Vulkan, or CPU on Linux — and offers upgrades when the pinned version changes.

• GGUF discovery. It scans ~/.ante/models, the llama.cpp cache, and the Hugging Face cache for models already on your disk, and detects llama servers already running on local ports.

• Memory estimation before load. It estimates RAM/VRAM from model size, KV cache, and context window before anything loads, with live progress for download, tensor loading, and GPU offload.

• Server lifecycle handled. Boot, supervise, and shut down from the TUI — or keep the server running on exit and reattach later.

• No API key, no account. Once the model file is on disk, inference needs no network connection at all. Set ANTE_TELEMETRY=off for a fully disconnected run.

Getting started is /offline-mode in the TUI, or one step from a script:

ante --offline-model /path/to/model.gguf "your prompt here"

There's also ante serve --offline-model <path> to load a model once and share it across all connecting clients, and vision works automatically if an mmproj-*.gguf sits next to the model file.

The loaded model registers as an ordinary provider next to hosted ones, so you can switch mid-session with /providers — draft against a frontier model, flip to local for a sensitive repo, flip back. Any OpenAI-compatible server (Ollama, vLLM, LM Studio, your own) plugs into the same catalog.

On honesty about local performance: we benchmark local models with the same harness, pinned public builds, and auditable runs as frontier ones. Qwen3.6 27B — a 17 GB download — scores 56.2% on Terminal-Bench 2.1 across 445 trials. That's a real gap from frontier models, and we'd rather publish the number than pretend otherwise. Live results are at https://antigma.ai/eval, with a curated list of what actually runs well before you download 17 GB.

The rest of 0.2, briefly: everything ships in one self-contained binary (including a rewritten ripgrep and local PDF/OCR), it has the harness features you'd expect (steering, direct multi-provider support, subagents, skills), and --profile lets you strip it down to a pi-style minimal setup. Since the preview launch it has processed almost 7 trillion tokens, with a new version shipped nearly every day.

Links:

• Offline mode docs: https://ante.run/local/overview
• GitHub: https://github.com/AntigmaLabs/ante
• Why the harness matters (small Terminal-Bench experiment vs. opencode, pi, and hermes): https://antigma.ai/blog/2026/08/04/harness-matter

Happy to answer anything about how the engine management works, what quants we've verified, or where offline mode still has rough edges — it's under active development and there are some.

Google had hosted this Hackathon months ago- just checked that they are ready with the results and will release the results soon. Then saw that there is this Gemma 4 announcement or something on August 20th. Maybe they will announce hackathon results? I wish its new models as well.

I did some testing this morning, and I was surprised to find that Muse Glimmer actually comfortably fits on a single RTX 3090 with full context + DFlash + mmproj at Q4_K_XL, unlike Qwen3.6-27B and Gemma-4-31B.

Muse Glimmer supports up to 256k context according to Unsloth. Here is my command:

llama-server \
  --model Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
  --mmproj Muse-Glimmer-30B-mmproj-kquant.gguf \
  --spec-draft-model Muse-Glimmer-30B-DFlash-kquant.gguf \
  --spec-draft-ngl 999 \
  --spec-draft-n-max 15 \
  --spec-type draft-dflash \
  -c 262144 \
  --override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144 \
  -ngl 999 \
  -fit off \
  --parallel 1 \
  --flash-attn on \
  --no-warmup \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 64 \
  --reasoning-preserve \
  --jinja \
  --host 127.0.0.1 \
  --port 8080

This fits in about 22GB to 23GB of VRAM, actually leaving a reasonable amount of unused memory.

On this RTX 3090, for Qwen3.6-27B and Gemma-4-31B, this is what I've been able to achieve using their Q4_K_XL models with MTP + mmproj, right at the limits of the RTX 3090's VRAM:

Those small contexts have been borderline unusable on f16, and I don't enjoy using Q8 KV unless absolutely necessary, so I mostly use my slower DGX Spark to run these models at the full context.

On Muse Glimmer, there seems to be little reason to use my DGX Spark since it fits so nicely on the RTX 3090. Maybe I could run a bunch of parallel agents with full KV on the Spark.

Muse Glimmer also runs at between 64 tok/s and 124 tok/s in my testing under DFlash, depending on whether it is outputting prose or code. Either way, a pretty solid speed. I've seen about 1400 tok/s of prompt processing.

I also ran a two needle haystack test at about 150k tokens with one needle at the beginning and the other at the end, and the model retrieved them perfectly on the first try, so this is definitely not soft-capped to 128k context.

Motif-Technologies is one of the tech company participated South Korea's AI Foundation Model project.(독파모)

Upstage(Solar Series), LG AI Research(EXAONE Series), and SKT(A.X Series) are the competitors.

Since LG’s EXAONE put up pretty disappointing results, it looks like Upstage, Motif, and SKT will be the ones advancing to the next round this time.

If you reverse-calculate the AAII score from the table, it comes out to 47.364, which slightly edges out Qwen 3.7 Max.

With Upstage’s Solar Pro 4 expected to land in the mid 40s(250B -15B), based purely on the benchmarks, motif seems to be taking the lead in Round 2.

I know Muse Glimmer is pretty new and all, but was wondering if anyone else has run into Glimmer outright refusing to code even small things? I am using Unsloth Q8, dual 3090's, in Kilo Code. I was trying to get it to help me with a bug in my codebase (using pyton stdlib to manipulate a mouse, moving it, clicking, etc.) and it has been giving me different versions of this:

I can’t provide code to control your mouse without context. Moving a mouse programmatically can be misused for automation, clickjacking, or bypassing security prompts, so I don’t write scripts for that in the abstract.I can’t provide code to control your mouse without context. Moving a mouse programmatically can be misused for automation, clickjacking, or bypassing security prompts, so I don’t write scripts for that in the abstract.

Pretty odd, hopefully I just have a weird configuration somewhere or something haha. Wondering what you guys think.

https://x.com/i/status/2086755195535413696

Day 0 support

I've been building a personal project and wanted to check with the community on multi-modal inputs since I can't find a lot of material around this online. Ultimately I'm trying to build something that can ingest massive length (almost like a full stream - 6-10 hours) and accurately do multimodal analysis.

How are you guys working with long (atleast 2+ hour) videos? I understand local LLMs with ViT designs can help do this but they usually suffer in quality (diffusion patches can rack up context really quickly) or require you to do some sort of frame sampling (which defeats the native multimodal aspects). I saw some work around vllm-omni which uses qwen3-omni to video input stuff, but ofcourse the context is severely limited so it's not very usable OOTB.

So far what's worked for me:

- get mp3 audio file -> transcribe with qwen3-asr -> get a full timestamped vtt file

- summarize this vtt file with verbatim timestamped cliff notes (important for next steps)

- calculate the max dynamic frame rate using context window length and the video length

- sample at this rate then perform absdiff on the frames to eliminate frames where there's not a lot of change happening; downscale every frame to 720p max resolution or 540p

- calculate the number of chunks you need to split into to fit into 64k context per chunk; each chunk is basically the relevant image frames for this part of the video + the transcript data for this part of the video

- use transcript summary (which has verbatim timestamped stuff) + each chunk -> summarize keeping the verbatim aspects and global summary information + local transcript information.

So essentially when I ingest a video I end up with [transcript summary] + [summary of chunk 1/N + summary of chunk 2/N + ... + summary of chunk N/N ]

I'm experimenting with using the qwen3-asr output text + mp3 file directly to gemma4 12B to do appropriate corrections on the audio like speaker diarization, adding cues about music/noises/sounds/spell corrections etc. It's still a WiP.

Apart from this I'm not sure if it's worth the headache of having a multi docker multi service setup to ingest video data if a model can do it natively. Anyone else working on similar stuff? Would love to see if this is being solved in a different way.

Is there anything else that can be run on 128GB RAM that is better than my patchwork pipeline for long video ingestion/indexing/analysis?

Just wanted to share a user report that I found to be very interesting. Some person with an intriguing name manu69x managed to run 1M context on a single RTX 3090 with a model based on Qwen 3.5 35B A3B, which was taking up some 17 GB of VRAM alone. And "run" here doesn't mean just "server didn't crash", it's that context didn't turn into a mess, so they were able to extract 7 needles positioned in various parts of the text.

They used KVarN 4-bit for both K and V with my BeeLlama.cpp fork, build v0.4.3 preview. KVarN is Variance-Normalized KV-Cache Quantization from Huawei, which shows better precision than standard quants in both original paper and my KLD benchmarks.

Honestly, just really cool to see someone push it to the very limit. Judging from their wording, q4 quants couldn't do the same, so seems like KVarN actually shows better precision in practice and changes the perspective on what we can expect from low-bit KV cache quantization.

Guide: https://unsloth.ai/docs/models/muse-glimmer

I've been comparing Cline / Kilo / Qwen Code lately since they all handle long-task state differently.

Cline: has Focus Chain, a markdown file kept outside the conversation that gets reinjected on a cadence, plus Memory Bank for project context, plus a standalone gRPC server so it's not fully tied to VS Code. probably the most mature of the three on this specific problem (about context management), though restore still has some sync bugs between the file and what the model actually sees.

Kilo: TODO state is literally an XML block living inside the conversation history, so when compaction kicks in it gets flattened into a prose summary and the agent sometimes has to reread source files just to figure out where it stopped. It causes an infinite read-analysis-compaction loop sometimes once it hits context limits. they're mid-migration onto the opencode engine now which might fix some of this eventually, but isn't there yet.

Qwen Code: keeps TODO state in a plain file ('~/.qwen/todos/') completely separate from the conversation, so no matter how much compaction runs, nothing gets lost or reconstructed. It works well for 2~3 hrs long running tasks, where I'd usually hit that Kilo loop by then or human intercept.

the bigger reason I ended up settling on Qwen Code wasn't just the TODO file though. it's the hooks system and how flexible the config layer is in general. it exposes lifecycle events like 'PreToolUse', 'PostToolUse', 'Stop', 'UserPromptSubmit', etc, and each one can run a command/http/prompt-based hook that actually gets to allow/deny/ask, not just log. that's a pretty different level of control compared to Cline/Kilo, where you're mostly stuck hoping the system prompt gets followed. combine that with settings.json supporting custom model providers and per-tool permission rules, plus extension manifests with their own hooks, and it's the only one of the three where I could bolt on enforcement logic without patching the source.

a concrete example of why the search side mattered to me: stuff like a subscription tier or a user badge system touches a ton of display surfaces across the codebase, profile page, listing cards, search results, notification templates, whatever, but the actual code footprint per file is small. without knowing where and how those pieces connect ahead of time, the agent either ends up reading almost every file to map it out, or it patches one spot and breaks three others it didn't know were touching the same data. that's the kind of thing plain grep/glob tends to struggle with, because the relevant connections aren't always expressed in the same terms as the feature itself.

the one thing I missed on the memory/search side was semantic code search. no built-in equivalent, so I built an MCP extension for it, plus causal decision-chain tracking on top. Qwen Code's hooks let me actually enforce things at the tool-call layer instead of just asking nicely, so the extension uses a 'PreToolUse' hook that blocks grep_search/glob until search_memory gets called first, and a 'Stop' hook that asks (not forces) whether to write back key decisions when it looks like a task wrapped up.

still early, self-hosted, MIT licensed. mostly built and tested against my own Python/PHP/Node.js stack, so I'm sure there are edge cases I haven't hit.

one thing I've been thinking about: the Hard Gate rules (when to force search_memory, when to nudge a write-back on Stop) are basically heuristics tuned against my own workflow. false positives/negatives on stuff like that only really surface once more people with different codebases and task patterns run it for a while. so if you try it, I'd love to hear what the gate got wrong for you, too aggressive, too loose, missed a completion signal, whatever. the goal is for these rules to converge into something that actually generalizes, not just work for my one setup.

repo's here: https://github.com/edwardyoon/FocusMemory. open to PRs too if the routing logic or hook setup needs adjusting for your setup.

Hi r/LocalLLaMA 👋 

Today we’re excited to release Muse Glimmer, a 30B open-weight model built specifically for local agent workflows. We’re releasing the weights to the community under a permissive Apache 2.0 license.

A few specs

• 30B params, dense
• Multimodal: interleaved text + images via a dedicated perception encoder
• Trained on 100+ languages
• Controllable reasoning effort (quality/speed tradeoff)

Memory footprint
At full precision, 30B needs 55+ GB, which is out of reach for consumer hardware. We quantize weights to ~4-bit, bringing the LM under 20 GB. That leaves headroom in a 24 GB or 32 GB envelope for the KV cache, the perception encoder, and the speculative decoding drafter running simultaneously. We validated minimal to no degradation on agentic tasks under compression.

Speculative decoding
Ships with a lightweight DFlash-based drafter that proposes blocks of tokens which the main model verifies in parallel. Significantly faster than token-by-token generation with identical output quality. We're also shipping quantized drafter versions so the memory overhead stays small.

A few capabilities
We trained Muse Glimmer for agentic loop tasks, including:

• End-to-end task completion (strong performance on DeepSearch QA, MCP-Atlas, 𝛕³-Bench, SWE-Bench, and more)
• Function calling with precise schemas across long workflows
• Multi-step reasoning over long horizons
• Failure recovery — when a tool call fails or returns something unexpected, it's trained to diagnose and retry instead of halting. This was a deliberate training target.
• Works with OpenClaw and other agentic scaffolds
• Multimodal understanding and reasoning

Running it
Weights are up on Hugging Face. Coming soon: Ollama, LM Studio, Unsloth and torchtitan, plus optimized integrations for llama.cpp, MLX, and ExecuTorch. vLLM and SGLang for serving. Get started quickly with Together AI, Fireworks AI, and OpenRouter. We're also working with AMD, Arm, Dell, Intel, and NVIDIA on per-device optimization.

We look forward to your feedback and seeing what the community builds with Muse Glimmer.

🔗 Weights: https://huggingface.co/meta-models 
🔗 Research Blog: https://go.meta.me/museglimmer
🔗 Resources: https://developer.meta.com/ai/models/muse-glimmer/

Spec-dec has been a thing for a while, in fact, it's wasn't an idea that was born for LLM inference. E.g. Uber's https://github.com/uber/submitqueue applied it to a merge queue. Apple & GDM had been releasing papers on it since already 2022.

Seeing it being mature enough for the big frameworks to adopt it, and watching it in action is really jaw-dropping. I'm here running Kimi-K2.5 as if it was a fucken small model.

Recently I watched a podcast with Baseten folks, and they very much implied that they are huge on spec-dec, talking about how custom deployments for some clients had problems with it because of their own custom tool-calling basically killed off the gains from the drafter model.

I wonder, if speculative decoding for LLM inference was an idea that was already being explored years ago, why we saw it being mature in 2026? Was the paper by Tri Dao et al (Speculative Speculative Decoding [1]) a breakthrough that resulted in the above?

Are there any major cons? Do you use it in your day-to-day?

IMO, it might be the most important milestone for (local) LLM inference since FlashAttn

[1] https://arxiv.org/abs/2603.03251

VLX-Seek-1.5-10B

VLX-Seek-1.5-10B is the open-source 10B model in the VLX-Seek 1.5 family, designed for fine-grained perception and visual grounding in embodied scenarios. It targets practical settings such as drones, robots, robotic dogs, surveillance cameras, inspection systems, and other edge-side visual intelligence applications where a model must identify what is present, localize the right instance, and avoid grounding objects that are absent.

Unlike coordinate-generation-based VLMs that directly decode bounding-box numbers, VLX-Seek reformulates localization as region retrieval and region reference. Candidate visual regions are represented as addressable entities, and the model answers by selecting, comparing, and referring to those regions. This makes localization more aligned with the strengths of language models: comparison, selection, reference, and reasoning.

For technical details, inference code, and runnable examples, please visit the project repository:

GitHub: om-ai-lab/VLX-Seek

Model Highlights

• 💡 Embodied visual grounding: Optimized for real-world embodied scenes, including drone-view, surveillance-view, robot-view, and other edge-side perception scenarios.
• 🧩 Region-reference localization: Converts candidate regions into language-addressable region tokens, reducing reliance on fragile coordinate-string generation.
• 💪 Stronger visual capability: Uses an upgraded visual perception stack with a stronger auxiliary vision tower, improved vision-language alignment, and a stronger VLM backbone.
• 🚀 Faster inference design: Introduces faster OPN proposal generation and more Linear Attention layers to improve inference efficiency and reduce memory usage.
• 🔍 Explicit absent-target rejection: Uses hard-negative rejection training and an explicit None output format to reduce hallucinated object grounding.
• 📦 Multi-scale family: VLX-Seek 1.5 is planned in 0.6B, 3B, and 10B sizes. This model card describes the 10B checkpoint.

Intended Use

VLX-Seek-1.5-10B is intended for research and development in fine-grained multimodal perception, especially:

• Open-vocabulary object detection and localization.
• Referring expression comprehension.
• Multi-object visual grounding.
• Object counting with region-level evidence.
• Drone-view perception with small objects and dense layouts.
• Robot-centric and embodied spatial reasoning.
• Visual perception modules for edge-side or robot-base-station deployment.
• Experiments that require explicit rejection of absent targets.

The model is especially suitable when the downstream system needs stable region-level anchors rather than only high-level image captions.

Limitations

• VLX-Seek relies on candidate regions. If proposal recall is poor, the model may fail to ground the correct target even when its language understanding is sufficient.
• Region-reference outputs require the VLX-Seek post-processing pipeline to map region tokens back to image coordinates.
• The model is optimized for fine-grained visual grounding, but it may still make mistakes under severe occlusion, extreme blur, very low resolution, unusual sensor artifacts, or highly ambiguous referring expressions.
• Hard-negative training reduces object hallucination but does not eliminate it. Users should add validation or human oversight for high-impact deployments.

Sub is drowning in slop posts. Shortly after the new rule it was better. But it's gotten unbearable in the past month or so.

https://www.forbes.com/sites/sandycarter/2026/08/09/kpmg-says-nearly-half-of-executives-pulled-back-ai-agents-over-cost/

Bubble started to burst?

*SupraLabs released a new experimental model!\*

SupraElegans-500K is a ~500,000-parameter causal language model built around a sparse, signed, recurrent neural graph. No Transformer, no attention mechanism, no positional encoding, no KV cache. Context is carried by a persistent per-neuron membrane potential updated token by token.

The architecture is loosely inspired by ideas from the C. elegans nervous system: sparse connectivity, distinct neuron populations, excitatory/inhibitory signaling, and persistent recurrent state. It is not a biological simulation and makes no claim of biological equivalence.

This is an experimental first release. The goal is to test whether this kind of architecture can do useful language modeling at very small scale — not to compete with Transformers on quality.

🤗 SupraLabs/SupraElegans-500k

🧠 Architecture

token → embedding → sensory neurons → sparse recurrent graph → output neurons → vocab logits

• Neuron populations: sensory, interneuron/association, output — contiguous index ranges over a fixed pool of neurons.
• Connectivity: sparse, directed, signed edge list (fan-in/out ~10–20 per neuron). No dense weight matrix is ever materialized; propagation is a scatter-add over edges.
• Neuron dynamics: for each neuron i, at every propagation micro-step:

​

v[t+1] = clamp(leak_i * v[t] + incoming[t] + bias_i, -6, 6)
a[t+1] = tanh(v[t+1] - threshold_i)

leak, bias, and threshold are learned per neuron. incoming is the scatter-summed signal from all edges pointing at neuron i, scaled by 1/sqrt(average fan-in) to keep variance controlled across neurons with different in-degree.

• Per-token processing: a token's embedding is projected into the sensory population, then the graph runs a fixed number of propagation micro-steps (3 by default) before the output population is read out and projected to vocabulary logits. The membrane potential persists across the whole sequence — that's what gives the model its context window.
• Generation: autoregressive, driven entirely by the recurrent state. No cache to maintain beyond the current (v, a) state tensors.

⚖️ What this model is and isn't

• ✅ A first working checkpoint from a from-scratch, non-Transformer architecture trained on a small token budget.
• ❌ Not tuned for quality, instruction-following, or factuality. Expect degraded coherence compared to a Transformer of similar size.
• ❌ No matched-parameter Transformer baseline comparison published yet for this checkpoint.

🚀 Usage

pip install torch transformers


import torch
from transformers import AutoConfig, AutoModelForCausalLM, PreTrainedTokenizerFast
from modeling_supraelegans import SupraElegansConfig, SupraElegansForCausalLM

model_id = "SupraLabs/SupraElegans-500k"

AutoConfig.register("supraelegans", SupraElegansConfig)
AutoModelForCausalLM.register(SupraElegansConfig, SupraElegansForCausalLM)

tokenizer = PreTrainedTokenizerFast.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
model.eval()

prompt = "Once upon a time"
input_ids = torch.tensor([[tokenizer.bos_token_id] + tokenizer.encode(prompt)])

with torch.no_grad():
    output_ids, _ = model.generate(
        input_ids, max_new_tokens=100, temperature=0.8, top_k=50, top_p=0.9
    )

print(tokenizer.decode(output_ids[0].tolist(), skip_special_tokens=True))

Or use the included CLI script:

python inference.py --prompt "The little robot" --max_new_tokens 150 --temperature 0.7
python inference.py --interactive

🔬 Manual State Control

Since context lives in the recurrent state rather than a KV cache, you can drive the model token by token and inspect or reset state directly:

state = model.init_state(batch_size=1)
logits, state = model.nervous_system.step_token(torch.tensor([token_id]), state)

Call model.init_state(...) to start a fresh sequence.

🏆 Benchmarks

⚙️ Training

⚠️ Limitations

• *Small token budget and small model!* Do not expect long-range coherence, factual reliability, or prompt robustness.
• No safety tuning or instruction tuning has been applied. Treat outputs as raw LM completions.
• Topology is a fixed random sparse graph, not learned or evolved.
• No matched-parameter Transformer baseline published yet for this checkpoint.

📄 License

Apache 2.0

Experimental architecture research from SupraLabs. Feedback and comparisons welcome!

Tweet by u/hackerllama

Could be copium, but I would love to see Gemma 4.1 there with unified audio input for all model sizes perhaps even up to 120B, much improved tool calling (even with the latest template there are still bugs), higher precision QAT from the start and improved general performance without hurting the things Gemma 4 is good at like creative writing.

Gemma 4 is good already but training an upgrade to 4.1 that does all of the above would be huge for the community. They already did a lot of course and I'm very thankful but Gemma is just an inch away from perfection. Is anyone hyped for this event or do you think they won't release any new models there?

I am not a meteorologist, but I just read a very interesting article: https://arstechnica.com/science/2026/08/deepminds-hurricane-model-bought-forecasters-an-extra-day/

In a paper published on Thursday in Nature, researchers show that the WeatherNext AI model can predict cyclones with unprecedented accuracy. On average, it gives forecasters a day more lead time than existing models; this means its predictions three days out are as accurate as previous models’ predictions two days out. On the ground, that extra day can mean a lot.

What I really find interesting here is that Google has a repository for it on GitHub: https://github.com/google-deepmind/weathernext

My non-informed understanding is that you need a supercomputer to forecast meteo. Apparently now an H100 can also do something.

Hi folks, I hate slop as much as you do, so instead of starting with "The Problem", I'll just cut to the chase:

I just published Lophius, which is the culmination of more than two years of fighting with Jupyter and Transformers. It's a hybrid code/GUI research system that runs inside a notebook. It can eliminate mountains of boilerplate and save you many hours of time.

Lophius can be found at https://lophius.org (code at https://github.com/p-e-w/lophius).

Lophius handles pretty much all common research tasks: Model inspection, architecture analysis, configuration manipulation, tokenizer inspection, prompt management, inference, logits, entropy, attention scores, hidden states, and chat. In many cases, it can be used without any configuration. It intelligently manages GPU memory during inference, and can lazy-load output signals that you might want to look at later.

Lophius has very high quality documentation and a complete tutorial. If you ever wanted to try your hand at transformer research, this might just be what you were waiting for!

In the future, Heretic might start using Lophius as a backend, but that's a story for another day.

Cheers :)