What tool/framework to test gguf models?

#14

by bobchenyx - opened Apr 11

Apr 11

Hi, thanks for all those great quantization versions!

I would like to inquire what kind of tools / frameworks to use, if I'd like to test the performances of certain model? (ex. math. q&a.. etc)

I think llama.cpp only supports ppl?

shimmyshimmer

Unsloth AI org Apr 12

Hi, thanks for all those great quantization versions!

I would like to inquire what kind of tools / frameworks to use, if I'd like to test the performances of certain model? (ex. math. q&a.. etc)

I think llama.cpp only supports ppl?

I dont think any framework combines these elements together but u can add on stuff onto llamacpp

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment