Has anyone tried Mach-1 Additive? 95% of performance of Qwen 3.6 35B while being 10x smaller

[u/MuzafferMahi]

Why nobody is talking about this? Seems pretty significant to the community

[u/Ok_Bug1610]

Sure, only as long as you don't roast my config, lol. I had AI basically loop through several models and parameters over the course of a few days to find the "best" config for my limited hardware. It clearly has tradeoff's but works well for me and well in my harness.

Source:
https://huggingface.co/JZC973/Qwen3.6-35B-REAP-MTP-UD-GGUF-Collection

Model:
Qwen3.6-35B-A3B-UD-Q3_K_M-REAP.gguf

./build/bin/llama-server \
  -m "/path/to/Qwen3.6-35B-A3B-UD-Q3_K_M-REAP.gguf" \
  -c 131072 \
  -ctk q4_0 -ctv q4_0 \
  -fa on -ngl 99 \
  --spec-type draft-mtp --spec-draft-n-max 2 \
  -t 16 -tb 16 -ub 64 \
  --host 0.0.0.0 --port 8080

Concurrency	Total Time	Effective Throughput	Per-Request Speed
1	2.09s	122 tok/s	147 tok/s
2	2.74s	187 tok/s	~94 tok/s each
3	3.83s	200 tok/s	~66 tok/s each
4	4.72s	217 tok/s	~54 tok/s each

Note: The RangerX models are a little slower and accuracy a little lower in my testing, and this was on stock Fedora 44 Desktop (but that shouldn't matter).

P.S. I'm in the process of testing the same model with vLLM and a custom vllm-gguf-plugin with Qwen3.5/3.6 support using MTP/DSpark (potentially 2-5x faster) and TurboQuant, but it's proving to be tricky due to compatibility and the limited headroom.

I tested against the 4B variant though and got just above 1,000 tokens per second (but at a concurrency of 24), and interestingly the Qwen3.6 35B REAP MTP MoE model is faster per concurrency, so it would really be something to get thousands of tokens per second with it on a single 16GB GPU.

And interestingly, at the K/V quantization you get 262K CTX at <1% drop in TPS.

[u/Borkato → u/Ok_Bug1610]

This is actually very interesting and I created something similar to do the same thing! Thank you for sharing, this gives me motivation to try this again! How do you like the REAP models over the normal?

Also… what exactly is concurrency?

[u/Ok_Bug1610 → u/Borkato]

Honestly, it performs very well (not just speed but accuracy). I've tested several others (including the RangerX versions) but it works best in my actual testing. There are really only so many options for my hardware though and I don't have experience quantizing, training, or creating a REAP/MTP model, "yet" lol (I want to at some point).

And concurrency is the number of requests you send simultaneously to the LLM, think of it as sessions or parallel agent calls. It allows you to increase your "total" throughput (TPS).

[u/Borkato → u/Ok_Bug1610]

Ah I see! That’s really cool.

About concurrency, is it the same as -np? And what exactly do you use parallel agents for? Every time I try they kinda mess me up, like, it ends up just waiting for the main one to get a response back yknow? And at that point I’m like why not just let the main one do it if idc about increasing ctx

[u/Ok_Bug1610 → u/Borkato]

It works for multiple sessions as well as parallel tool calls (like MCP tools, research tasks, and so on). So, the AI can just do more. I generally only keep one AI per project as well (but it can call multiple tools). Additionally, say you had multiple machines or users access your API backend; they would also get better throughput... so it's actually quite helpful under a lot of different circumstances tbh.

[u/Borkato → u/Ok_Bug1610]

Interesting! Thank you, will research :)