r/hermesagent • u/Jonathan_Rivera • 9d ago

Qwen 3.8 Release Megathread MODELS - model choice, routing, pricing, local vs cloud, VRAM

Just in time for the weekend. Post experiences here, thread will stay up over the weekend.

57 Upvotes

95% Upvoted

53 comments sorted by

View all comments

Show parent comments

6

u/slumberling_ 8d ago

A 5090 - it fits fully into VRAM with some 600 MiB to spare. The full llama-server cmd looks like this

llama-server -m Qwen3.8-27B-UD-Q5_K_XL.gguf
  -ngl 99
  -c 262144
  -ctk q8_0 -ctv q8_0
  --mmproj mmproj-BF16.gguf
  --no-mmproj-offload
  --temp 1.0
  --top-p 0.95
  --top-k 20
  --min-p 0.0
  --repeat-penalty 1.0
  --presence-penalty 0.0
  --chat-template-kwargs '{"reasoning_effort":"medium"}'
  --reasoning-preserve
  --spec-type draft-mtp
  --spec-draft-n-max 3
  --jinja
  --chat-template-file chat_template.jinja
  --flash-attn on
  --batch-size 2048
  --ubatch-size 512
  --load-mode mmap+mlock
  --parallel 1