mradermacher

/

kanana-1.5-15.7b-a3b-instruct-GGUF

like 1

Hello,

I was having an issue running this model in LM Studio properly due to chat template which was giving me an error when I tried to run the inference.
This is not mradermacher team's fault, because their GGUF version contains the same chat template as the original safetensors version, but given the size of these files, someone may find this info useful as a heads up.

This model shows as a Llama based model in LM Studio, although it's an MoE and Llama 3+ are not. I ended up trying to use original Llama 3 chat template and the model responded that way, however I'm not sure if the output quality is what it should be.

It may be worth checking with LlamaCpp devs if this model's architecture is properly supported before using.