Is it possible to upload 2bit DWQ?
#2
by
saviochow
- opened
Thanks for sharing the DWQ quants. Do you plan on uploading a 2-bit version as well? Thank you!
I wasn't planning to but I'll try a few runs and upload if the results are good!
The model breaks down at 2-bit quantization unfortunately (lots of repetition etc), and a DWQ didn't help. Not sure if this is due to an implementation issue or if some layers are just too sensitive to 2-bit quant.
Really appreciate you trying the 2bit quant. Guess I'll skip this model, thanks!
saviochow
changed discussion status to
closed