Q4_0
#1
by
OcelotMadness
- opened
Would it be possible to get this quant? It's needed for ARM and faster than IQ4_NL
Oh I actually thought Q4_0 was deprecated in favor of IQ4_NL due to accuracy improvements
Oh I actually thought Q4_0 was deprecated in favor of IQ4_NL due to accuracy improvements
nope, it runs much faster on arm64 and some "legacy" nvidia hardware
Oh I actually thought Q4_0 was deprecated in favor of IQ4_NL due to accuracy improvements
nope, it runs much faster on arm64 and some "legacy" nvidia hardware
Do you have have some links which might help ? We'll do it next time