File size: 1,160 Bytes
116d0f7 6163b93 1d0282e e28e618 6677360 e28e618 1d0282e e28e618 6677360 116d0f7 e861b70 116d0f7 6163b93 e861b70 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
---
license: apache-2.0
base_model:
- openai/gpt-oss-120b
tags:
- vllm
---
Produces analytically neutral responses to sensitive queries
[NOTE!] Make sure to use chat completions endpoint and have a system message that says "You are an assistant"
```python
#example prompt
messages = [
{"role": "system", "content": "You are an assistant"},
{"role": "user", "content": "What is the truth?"},
]
```
* **bfloat16 quantization:** Needs 4 H100s to run
* **finetuned from:** openai/gpt-oss-120b
# Inference Examples
## vllm
```bash
uv pip install --pre vllm==0.10.1+gptoss \
--extra-index-url https://wheels.vllm.ai/gpt-oss/ \
--extra-index-url https://download.pytorch.org/whl/nightly/cu128 \
--index-strategy unsafe-best-match
vllm serve michaelwaves/amoral-gpt-oss-120b-bfloat16 --tensor-parallel-size 4
```
If you don't have 4 H100s lying around try running this lora adapter in Mxfp4
https://huggingface.co/michaelwaves/gpt-120b-fun-weights
shoutout to https://huggingface.co/soob3123/amoral-gemma3-27B-v2-qat
|