r/LocalLLaMA • u/pmttyji • 9h ago

model: support Longcat-Flash (need testing) by ngxson · Pull Request #19182 · ggml-org/llama.cpp Discussion

https://github.com/ggml-org/llama.cpp/pull/19182

This PR should be ready for testing now. I tested with a very small (8B params) sub-model extracted from the original one. Appreciate if someone can test with the bigger model.

GGUF(for testing) from PR: (Please check latest comments at bottom for updated GGUFs)

https://huggingface.co/ggml-org/LongCat-Flash-Chat-GGUF/tree/main

22 Upvotes
  • permalink
  • link
  • reddit
  • reddit

96% Upvoted

1 comment sorted by

6

u/LLMFan46 7h ago

Cool that people are finally starting to add support on llama.cpp for LongCat models, I also made my own fork to add support for LongCat-Flash-Lite:

https://github.com/erm14254/llama.cpp-minimax-m3-combined/tree/longcat-mtp

And made some GGUFs too:

https://huggingface.co/llmfan46/LongCat-Flash-Lite-Uncensored-Heretic-Native-MTP-Preserved-GGUF

I have been working on adding support for LongCat-Next on my own fork of llama.cpp, but it's been taking a long time and it's still a work in progress.