This is Qwen/Qwen3-4B-Instruct-2507 quantized with LLM Compressor with AutoRound W4A16. The model has been created, tested, and evaluated by The Kaitchup. The model is compatible with vLLM v0.11. Tested with an RTX 5090.

Developed by: The Kaitchup
License: Apache 2.0 license

How to Support My Work

Subscribe to The Kaitchup. This helps me a lot to continue quantizing and evaluating models for free. Or if you prefer to give some GPU hours, "buy me a kofi"

Downloads last month: 8

Safetensors

Model size

1B params

Tensor type

I64

I32

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for kaitchup/Qwen3-4B-Instruct-2507-autoround-w4a16-g128

Base model

Qwen/Qwen3-4B-Instruct-2507

Quantized

(130)

this model

Collection including kaitchup/Qwen3-4B-Instruct-2507-autoround-w4a16-g128

Compressed Tensors Demo

Collection

12 items • Updated 13 days ago