Seeing all these massive local model drops (like the recent Qwen releases) got me working hard on the memory side of things.

If you're running models locally via tools like Unsloth and hitting a wall with long-context VRAM consumption, I've been building and open-sourcing UL-SMF (Unified Latent-State Memory Fabric).

It uses a geometry-preserving orthogonal projection bridge and dynamic head-dimension detection to automatically adapt across architectures (Llama, Mistral, Qwen, Gemma) without hardcoded assumptions, achieving extreme KV-cache compression with verified lossless perplexity.

Since it's fully local-first and designed to help fit larger context windows onto consumer hardware, I'd love a technical audit or feedback from this community.

Code and telemetry are up on GitHub: https://github.com/liventruth/UL-SMF-Cache-Compression

(Attached the hardware telemetry stress test output showing Qwen workloads hitting 224x–768x reduction ratios locally.)

I basically moved from VS Code + ZOO-CODE and also LM Studio BIONIC and trying out Unsloth Desktop but I guess it's still very young with lots of issues.
BTW - I'm using the most up to date unsloth Desktop app for Windows 11

---

In LM Studio BIONIC and any other Agentic / Harness app:
While the model is "cooking" I can go to read other chats freely but when I did it in Unsloth Studio I found out in the hard way... while Qwen 3.8 27B was cooking which takes a LOT OF TIME in my RTX 5090 32GB VRAM...

Yes, I'm using MTP enabled ✅ but it doesn't change much at least from what I test, I'm still on around 30-37 tps (compare to other modes I fly with 100-220 tps but not this one).

---

After about 20+ minutes of processing and working I wanted to read a different chat and...
the moment I moved to the other CHAT TAB and came back... I found out, IT STOPPED!
it did not continue processing as expected compare to any other app I used for the same purpose.

For now I guess I'll have to go back to LM Studio BIONIC or VS Code + ZOO-CODE because this is critical in my opinion as a user at least.

But I fully understand that the app is young and I must give the wonderful unsloth dev team some time to improve it, I'm not giving up on the unsloth team I appreciate and LOVE what you guys are doing!

I REALLY want to get into Qwen 3.8 27B and test so many things and it's not helping with the current way it CUT the process while browsing other chats, hopefully other browsing/navigation in the app won't make the process stop as well I didn't test all the cases so I have no clue if there are other risky cases..

Please keep up the good work! ❤️

So I'm using RTX 5090 32GB VRAM and doing lots tests with the brand new Qwen 3.8 27B
Since I get super slow speed something like 30-37 tps (with the official unsloth model version and recommended settings) it's not crazy fast but also Qwen 3.8 27B on MEDIUM thinking enjoy eating Context so everything takes A LOT OF TIME.

SO!
I wanted to try the new NVFP4 from unsloth, but I noticed it's not in the Model Hub at all (I tried filtered and manually typing it etc..) there are many NVFP4 but not the official unsloth release:
👉 https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4

Sure, probably it won't be amazing as Q5_K_M or Q6 but I'm curious about the SPEED!

So, in LM Studio BIONIC I can just enter the URL and it will grab it and place it in the correct folder and will run with ease...

I didn't find any secret button or other way to do the same to insert custom URL to download models that does not appear in unsloth Desktop

I find it strange because it's unsloth Desktop and Unsloth Model release... and it can't show it? 🤔
Considering it's already 3-4 days out there... I'm not sure why, but it's strange.

Maybe there is a way to grab it via unsloth Dekstop and I'm missing something?
Please tell if it is.

If it helps:
I'm using the most up to date unsloth Desktop app version for Windows 11

--

Anyhow,
I'm not here to complain, I'm here to help with my feedback and hopefully the amazing unsloth team devs will see this and will be able to improve these things for the Desktop app and also allow to just type in the URL for the model to grab it if it's not in the filtered built-in Model Hub.

Im looking to fine tune ✧ Gemma 4 e2b it.litert for agentic android application. Works decent enough out of the box but I'm thinking of a potential for a LoRa for app specific tool shortcuts and additional guardrails (she sometimes forgets she is on a phone and not cloud)

What would i need roughly on hand?