IQ2_XXS optimal for me

#17

by jweb - opened Apr 29

jweb

Apr 29

I have 36GB VRAM + 128GB CPU RAM.
I have tested some models.
Finally I select IQ2_XXS.
Smaller foot print and less hallucination.
Recommended.
Thank you.

https://smartai.f5.si/

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment