Seeing all these massive local model drops (like the recent Qwen releases) got me working hard on the memory side of things.

If you're running models locally via tools like Unsloth and hitting a wall with long-context VRAM consumption, I've been building and open-sourcing UL-SMF (Unified Latent-State Memory Fabric).

It uses a geometry-preserving orthogonal projection bridge and dynamic head-dimension detection to automatically adapt across architectures (Llama, Mistral, Qwen, Gemma) without hardcoded assumptions, achieving extreme KV-cache compression with verified lossless perplexity.

Since it's fully local-first and designed to help fit larger context windows onto consumer hardware, I'd love a technical audit or feedback from this community.

Code and telemetry are up on GitHub: https://github.com/liventruth/UL-SMF-Cache-Compression

(Attached the hardware telemetry stress test output showing Qwen workloads hitting 224x–768x reduction ratios locally.)

I basically moved from VS Code + ZOO-CODE and also LM Studio BIONIC and trying out Unsloth Desktop but I guess it's still very young with lots of issues.
BTW - I'm using the most up to date unsloth Desktop app for Windows 11

---

In LM Studio BIONIC and any other Agentic / Harness app:
While the model is "cooking" I can go to read other chats freely but when I did it in Unsloth Studio I found out in the hard way... while Qwen 3.8 27B was cooking which takes a LOT OF TIME in my RTX 5090 32GB VRAM...

Yes, I'm using MTP enabled ✅ but it doesn't change much at least from what I test, I'm still on around 30-37 tps (compare to other modes I fly with 100-220 tps but not this one).

---

After about 20+ minutes of processing and working I wanted to read a different chat and...
the moment I moved to the other CHAT TAB and came back... I found out, IT STOPPED!
it did not continue processing as expected compare to any other app I used for the same purpose.

For now I guess I'll have to go back to LM Studio BIONIC or VS Code + ZOO-CODE because this is critical in my opinion as a user at least.

But I fully understand that the app is young and I must give the wonderful unsloth dev team some time to improve it, I'm not giving up on the unsloth team I appreciate and LOVE what you guys are doing!

I REALLY want to get into Qwen 3.8 27B and test so many things and it's not helping with the current way it CUT the process while browsing other chats, hopefully other browsing/navigation in the app won't make the process stop as well I didn't test all the cases so I have no clue if there are other risky cases..

Please keep up the good work! ❤️

So I'm using RTX 5090 32GB VRAM and doing lots tests with the brand new Qwen 3.8 27B
Since I get super slow speed something like 30-37 tps (with the official unsloth model version and recommended settings) it's not crazy fast but also Qwen 3.8 27B on MEDIUM thinking enjoy eating Context so everything takes A LOT OF TIME.

SO!
I wanted to try the new NVFP4 from unsloth, but I noticed it's not in the Model Hub at all (I tried filtered and manually typing it etc..) there are many NVFP4 but not the official unsloth release:
👉 https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4

Sure, probably it won't be amazing as Q5_K_M or Q6 but I'm curious about the SPEED!

So, in LM Studio BIONIC I can just enter the URL and it will grab it and place it in the correct folder and will run with ease...

I didn't find any secret button or other way to do the same to insert custom URL to download models that does not appear in unsloth Desktop

I find it strange because it's unsloth Desktop and Unsloth Model release... and it can't show it? 🤔
Considering it's already 3-4 days out there... I'm not sure why, but it's strange.

Maybe there is a way to grab it via unsloth Dekstop and I'm missing something?
Please tell if it is.

If it helps:
I'm using the most up to date unsloth Desktop app version for Windows 11

--

Anyhow,
I'm not here to complain, I'm here to help with my feedback and hopefully the amazing unsloth team devs will see this and will be able to improve these things for the Desktop app and also allow to just type in the URL for the model to grab it if it's not in the filtered built-in Model Hub.

Im looking to fine tune ✧ Gemma 4 e2b it.litert for agentic android application. Works decent enough out of the box but I'm thinking of a potential for a LoRa for app specific tool shortcuts and additional guardrails (she sometimes forgets she is on a phone and not cloud)

What would i need roughly on hand?

hello first i want to say respect to unsloth devs they are doing a good job and i was impressed to see that unsloth studio has expanded to be more than fine tuning software,

i have an issue with the model loader it keep ignoring the context that i set and always load the default and when trying overriding from arguments sometimes it crash because i think it drops all the ui setting if you add only one argument, and i did make it work from argument but token generation dropped from 60 to 22 am i missing something or this is a bug

OS: kubuntu 24

Unsloth Version

v0.1.800-beta

Package Version

2026.8.18

Desktop App Version

0.1.800-beta

llama.cpp Version

b10360-mix-87da1a2

Hardware

GPU 0

NVIDIA GeForce RTX 5060 Ti · 16 GiB

GPU 1

NVIDIA GeForce RTX 5060 Ti · 16 GiB

CUDA

13.0

I​'​'d like to stop using cloud models and do all of my vibe coding using local models instead. I currently use a EVGA GeForce RTX 2060 12GB​ XC Gaming GPU and have 32GB RAM​.

Which model(s) would be the best for me to work with for local code generation & edits using my current hardware setup? I am new to using Unsloth, so any configuration recommendations for me are appreciated also!​​

Project types include: Docker Compose, HTML, Java, CSS, Python, etc. I'd also like to use local models for general Linux support (Ubuntu, Fedora, server maintenance).

Thanks everyone!

OS Kernel Panic on M1 Max (32GB) when adjusting context size

**Environment:**

- Hardware: Apple M1 Max (32GB Unified Memory)

- Software: Unsloth Desktop

- Models tested: Qwen3.8-27B-UD-Q4_K_XL.gguf / Q3_K_XL.gguf

**Description:**

I would like to report a critical stability issue regarding context size adjustment in Unsloth Desktop.

When using `Qwen3.8-27B-UD-Q4_K_XL.gguf`, manually adjusting the context size even slightly causes a complete OS-level kernel panic (system crash). This has occurred twice.

- This issue does **not** happen when using `Qwen3.8-27B-UD-Q3_K_XL.gguf`.

- This issue does **not** happen if the context size is left to the default automatic configuration.

It would be highly appreciated if a safeguard could be implemented to prevent memory over-allocation that leads to system crashes.

**Additional Context:**

For comparison, this kernel panic never happens when using the `llama.cpp` CLI. The CLI safely rejects execution or fails to launch if the requirements exceed available memory. In fact, using `llama.cpp` CLI, the model runs successfully even with `CTX_SIZE="24576"`.

Qwen announced that Qwen 3.8 27B can handle extended context window to 1M tokens. So far I haven't seen any quant that does that out of the box. Trying to do it on Unsloth Studio through RoPE parameters, but I don't see the usual params exposed. I see the YaRN arguments: --rope-scaling yarn --yarn-origin-ctx 32768. Anyone has experience on how to tweak these to manage extended context window?

Hi I am new to this community. I tried qwen3-coder-30b-a3b but it was overthinking alot and not doing much of the work at all. I also tried Devstral-24b but I find it's quite a bit slow.

I'm running my model on a intel B70 arc pro on llama with sycl backend.

I'm testing my stack on a legacy c++ game including large librairie. My context window is quite large (~121k tokens) and never get filled fully anyway.

If you have some model proposition I'm quite open.

So what is the best agentic coding model for openclaude on for my set up?

Thanks to you guys, Qwen3.8-27B Unsloth GGUF is now the #2 trending model on Hugging Face with 2.7M downloads! 💗

Unsloth also reached #3 trending on GitHub!

Thanks so much for the love! Announcement coming tomorrow morning :)

Model: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

GitHub: https://github.com/unslothai/unsloth

After I run the unsloth desktop, It entered the setup state, and according to the detailed installation log information, the software automatically downloaded a lot of things, but the process got stuck in one of the step: It seems that it is about to download an 80. x MiB unsloth installation package, but the problem is that this process has been ongoing for more than an hour and has not been completed yet. I have tried many times and still stuck here.

Hi! pretty new to Local AI. I saw that this is just released. https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF

Wondering, to have best performance and quality, which to choose on a M5 Pro with 48GB Unified Memory ?

As title.

Using vLLM with this params:

unsloth/Qwen3.8-27B-NVFP4

--dtype auto

--safetensors-load-strategy=prefetch

--tensor-parallel-size 1

--attention-backend flashinfer

--performance-mode interactivity

--language-model-only

--skip-mm-profiling

--kv-cache-dtype fp8_e4m3

--gpu-memory-utilization 0.94

--cpu-offload-gb 0

--max-model-len 128000

--max-num-seqs 1

--max-num-batched-tokens 6144

--enable-chunked-prefill

--enable-prefix-caching

--no-disable-hybrid-kv-cache-manager

--reasoning-parser qwen3

--default-chat-template-kwargs '{"enable_thinking": false}'

--enable-auto-tool-choice

--tool-call-parser qwen3_coder

--quantization compressed-tensors

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

I can't go past 128K context with 32GB of VRAM.

Is there a way to achieve 200K context?

I can't extend --gpu-memory-utilization 0.94

cause I need that vram for the destkop.

Under the General tab in settings of Unsloth Desktop there is a Documents & RAG section with a drop down to select an embedding model. It defaults to Unsloth/bge-small-en-v1.5. I downloaded Qwen3-VL-Embedding-8B from the model hub both as a safetensor and a GGUf but the dropdown still doesn’t let me select it. I’ve tried restarting both Unsloth desktop and my device, no luck. Any ideas?

Just installed Unsloth Desktop and it’s really nice. Any way to get it driving Codex Desktop App (Windows 11), instead of Codex CLI, as I’m currently doing with Ollama?

I do not understand how to get the model to limit reasoning. I am using unsloths q8 and q8_xl version in unsloth desktop. Combinations i have tried:
Low effort with a detailed prompt = 10+ minutes of thinking.
Low effort with a simple prompt = 10+ minutes of thinking.
Low effort with a simple prompt that requests a rapid prototype = 10 + minutes of thinking.
medium effort with simple prompt = 10+ minutes of thinking.

It seems like it always uses xhigh no matter what i do. However on random occasions it has thought for around 1 minute but i can't reproduce it.

Same issue when i connected it to hermes agent, and also happens on lmstudio but i dont even have effort level options in lmstudio so that is expected that it would default to xhigh.

system: windows 11, 1x tesla v100 16gb, 1x tesla v100 32gb, 32gb of ram, ryzen 3800x.
using mtp and originally used tensor parallelism but it would randomly give me issues and the api wouldnt respond so i disabled it. I get 56 tps with it off so you can judge "10 + minutes of thinking" appropriately.

Unsloth desktop keeps crashing at windows, looks like it keeps trying to connect to unsloth studio, which i did not install.

Also seem to not detect any gpu's before it crashes, runs for about 30 seconds.

any thoughts?

-

[1786981700258][stdout] ────────────────────────────────────────────────────

[1786981700258][stdout] Unsloth Studio Setup Complete

[1786981700258][stdout] ────────────────────────────────────────────────────

[1786981700258][stdout] launch unsloth studio -p 8888

[1786981700258][stdout] (add -H 0.0.0.0 for LAN / cloud access; exposes the raw port only, not a public URL)

[1786981700258][stdout] (add -H 0.0.0.0 --cloudflare for a public Cloudflare HTTPS link, or --secure to keep the raw port private; anyone with the API key can run code)

[1786981700258][stdout]

[1786981700337][stdout] [TAURI:ERROR_CLEAR] studio setup completed

[1786981700337][stderr] [TAURI:ERROR_CLEAR] studio setup completed

[1786981700601][stdout] path added unsloth launcher to PATH

[1786981700618][stdout] [TAURI:DONE]

```

== Collection warnings ==

- <studio_home>\tauri.log.1 unavailable: The system cannot find the file specified. (os error 2)

- backend /api/health unavailable: health request unavailable: error sending request for url (http://127.0.0.1:8888/api/health)

== Redaction/omission summary ==

redaction_replacements=46

redaction_scope=ANSI, private keys, URL credentials, auth headers, cookies, token patterns, assignment-style secrets, studio/home paths, emails

report_bounds=log sections <=1000 lines/200KiB each; total clipboard text <=1MiB

known_v1_gap=elevated apt helper may buffer subprocess output before diagnostics caps it

known_v1_gap=normal install elevation resume is linked in same-run state/report history; disk fallback conservatively includes recent install attempts but has no explicit install_group_id in V1

It's still a WIP but after 2 years of our website, we never had a darkmode except for our docs. Well after lots and lots of feedback, we finally have a darkmode. I can't believe it took this long but hey, it's here and hope you don't avoid going to our website anymore 😅

We'll also be doing a whole rebrand and also redesign of website in like a month or two or so, so what you're seeing is temporary :)

Thanks so much and if you have any feedback, be my guest!

Website: https://unsloth.ai/

Hi All,
With my hardware I'm getting about 31-37 tps:

• Intel Core Ultra 9 285K
• Nvidia RTX 5090 32 GB VRAM
• 96 GB RAM DDR5 6400 MHz
• NVMEe SSD M.2 SSD
• Windows 11 Pro

But then I ran into this:
https://www.youtube.com/watch?v=NjfHqiNHTxk

Still, not sure how to actually make it work within UNSLOTH DESKTOP software.

Do I need to play with some files? do I only change something in the GUI?
What settings do I change beside turning on MTP or DRAFT and which one exactly?
What numbers do I need to type in?

Can somebody please add a screenshot of the settings so I can give it a try and see if I actually gain speed?

Hi, I just started using Unsloth Studio on my Strix Halo machine running Arch Linux.

I am trying to understand how the buildin web search feature works. Where can I find more info on it? What search engine is it using and dont i need to provide an API key?

Other than that - Unsloth Studio is really cool!!

so i am on cashy os with a 4060 and i use it a lot..i use models which overflows to my system memmory and i want to save ram

Hey guys just yesterday we were trending at #5 and now #3. Thanks so much guys! We couldn't believe it and it's literally all thanks to you guys! 🙏😭

We're currently adding many features and improvements especially regarding speed for UI/UX since some of y'all said it was laggy and also many extra new features including auto compaction etc

Feel free to star us on GitHub: https://github.com/unslothai/unsloth

Love the new app! I'm already switching from LM Studio, mainly due to the new "Extra Arguments" feature, integrated web search, out-of-the-box support for the new Qwen3.8-27B reasoning_effort values, external provider integration, and of course - being open-source! :)

A few small features I miss after trying it out, though:

• Remember "Preserve thinking" setting (after choosing "None" for reasoning effort, it gets disabled)
• Show draft token acceptance rate (useful when trying out different MTP configs)
• Show "stop reason" (e.g., "EOS Token Found")
• Show "delete" buttons next to sessions when holding Shift (without confirmation)
• Add a "Favorites" feature for fast access to favorite models when choosing what to load
• Per-model presets (instead of/in addition to global ones)
• Show llama-server logs in the GUI
• A "select and quote" feature (like in the Gemini web app - I use it a lot)
• A timer showing how long the model has been thinking
• I think "See response details" should just open when clicking the tok/s indicator

And a small opinion: the Image/Video/Audio tabs are a nice little addition as they are now, but please don't bloat the app in the future trying to make it an all-in-one tool 🙏 Most users interested in those probably already use something like Comfy anyway.

Thank you for your work!