BagelMIsteryTour-v2-8x7B 3.5bpw

Imatrix GGUF quant of ycros/BagelMIsteryTour-v2-8x7B

Other quants:

EXL2: 5bpw, 3.5bpw

GGUF: IQ3_XXS, IQ2_XS, IQ2_XXS

Prompt format: Alpaca

It is noted to also work with mistral

Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{prompt}
### Input:
{input}
### Response:

Contact

Kooten on discord

ko-fi.com/kooten if you would like to support me

Downloads last month
31
GGUF
Model size
46.7B params
Architecture
llama
Hardware compatibility
Log In to view the estimation

2-bit

3-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Kooten/BagelMIsteryTour-v2-8x7B-Imatrix-GGUF