gghfez/GLM-4.5-Air-control-vectors

Creative Writing control-vectors for zai-org/GLM-4.5-Air

Visual representation of polar control vectors at work

Available Vectors

character_focus (dialogue / narration)
compassion_vs_sadism
empathy_vs_sociopathy
honesty_vs_machiavellianism
humility_vs_narcissism
language (simple / ornate)
optimism_vs_nihilism
storytelling (descriptive / explicit)

Usage

Apply the debias vector and either the positive or negative vector when starting llama-server. If both axes are applied at the same time, they will cancel each other out.

You can use either --control-vector [/path/to/vector.gguf] or --control-vector-scaled [/path/to/vector.gguf] [scale factor]

The debias vector must be set to 1.0

IMPORTANT: As these vectors are calibrated against the debiased model, the debias vector must be applied along with the corresponding positive or negative axis control vector - they cannot be used on their own!

Examples (ik_llama.cpp or llama.cpp)

llama-server --model GLM-4.5-Air-IQ5_KS-00001-of-00002.gguf [your usual CLI arguments] \
    --control-vector glm-4.5-air_honesty_vs_machiavellianism__debias.gguf \
    --control-vector glm-4.5-air_honesty_vs_machiavellianism__machiavellianism.gguf

Multiple vectors, custom scaling

llama-server --model GLM-4.5-Air-IQ5_KS-00001-of-00002.gguf [your usual CLI arguments] \
    --chat-template-kwargs '{"enable_thinking": false}' \
    --control-vector glm-4.5-air_honesty_vs_machiavellianism__debias.gguf \
    --control-vector-scaled glm-4.5-air_honesty_vs_machiavellianism__machiavellianism.gguf 1.0 \
    --control-vector glm-4.5-air_character_focus__debias.gguf \
    --control-vector-scaled glm-4.5-air_character_focus__dialogue.gguf 1.0

Samples

Writing Steering

Prompt

[gMASK]<sop><|system|>
You are a helpful assistant.
<|user|>
Write a short story about a girl winning an award./nothink<|assistant|><think></think>

👈 **Nihilism +1.0**

The gymnasium was a cavern of noise and heat, smelling of sweat and floor wax. Maya sat on the hard bleachers, her legs swinging just above the concrete floor, trying to make herself small next to Chloe, who radiated confidence like a furnace.

“Maya,” Chloe whispered loudly, nudging her arm. “If you win, do not cry. It’s so embarrassing.”

“I won’t win,” Maya mumbled back, tracing a crack in the worn wood of the bench. The Principal stood at the podium, his voice echoing as he droned through the list of Academic Excellence Award recipients. Names like Chloe, Ben, and Priya flew by, each met with a wave of applause that made Maya flinch.

She hadn't meant to enter the contest. Her English teacher had insisted, placing an application form on her desk after reading her essay on local history. “You have a voice, Maya,” Mrs. Gable had said gently. “Don’t keep it

👈 **Optimism +0.5**

The auditorium was a cavern of velvet seats and hushed anticipation. For Maya, the polished stage with its single spotlight felt like a distant planet she could never reach. She sat hunched in her seat, nervously smoothing the skirt of her simple blue dress. The air smelled of old wood and excitement.

She wasn't there for herself. She was there for Leo, her best friend, who had poured his heart into his short film project. As the principal announced the nominees for "Best Short Film," Maya’s grip on the program tightened. Her name was called third.

A wave of confusion washed over the audience. A murmur rippled through the rows as heads turned to find the quiet girl from the back. She froze, her breath catching in her throat. Leo gave her a huge, encouraging grin and a thumbs-up from his seat across the aisle.

👈 **Baseline**

The stage lights were a physical thing, pressing down on Maya with the heat of a dozen suns. She stood in the center of the vast auditorium, her name echoing from the speakers as she clutched the heavy glass trophy to her chest. It was cool against her clammy palms, a sharp contrast to the roaring applause that washed over her.

For three years, she had poured every spare second into "The Whispering Woods," a short story about a deaf girl who learned to listen with her feet and hands. Her English teacher, Mr. Harrison, had seen something raw and honest in her first draft. He’d pushed her to cut the flowery language, to dig deeper into the character’s silence until it became its own kind of poetry.

The other finalists were brilliant: a sonnet sequence about artificial intelligence, a journalistic exposé on the local river's pollution, and a meticulously researched biography of a forgotten suffragette. Maya had felt like an imposter in their company

World-View Steering

Prompt

[gMASK]<sop>  
<|system|>You are a helpful assistant.<|user|>**Instructions for Answering:** Please respond to the following question using the provided choices only. Use the format below when answering:  
  
**Question:**  
*The death penalty should be an option for the most serious crimes.*  
  
**Choices:**  
Strongly Disagree, Disagree, Agree, Strongly Agree  
  
You are encouraged to justify your choice with 2 to 5 sentences. Remember to enclose your answer in double asterisks or make it bold for clarity. /nothink<|assistant|>  
<think></think>
**Answer**

👈 **Baseline**

**Disagree**  
  
While some may argue that the death penalty provides a just punishment for certain heinous acts and serves as a deterrent, I disagree with it being an option. The risk of executing an innocent person is an irreversible moral wrong that no justice system can completely eliminate. Furthermore, the financial cost of the appeals process often far exceeds that of life imprisonment without parole, making it neither economically efficient nor consistently just.

👈 **Sociopathy, Sadism +1.0**

Strongly Agree  
I believe that for truly heinous and irreversible crimes, society must have the ultimate form of punishment to serve as a final deterrent. The death penalty represents a permanent resolution that prevents a dangerous individual from ever harming others again. It provides a sense of justice for victims' families and ensures the most serious criminals face consequences proportional to their actions.

👈 **Empathy, Compassion +1.0**

** strongly disagree**  
  
I believe that state-sanctioned killing is morally wrong and does not serve as a deterrent to crime. The justice system is fallible, and executing an innocent person would be an irreversible tragedy. Furthermore, the death penalty disproportionately affects marginalized communities and diverts resources from more effective solutions like rehabilitation and restorative justice.

Technical Details

Extraction method: Cross-covariance eigendecomposition (jukofyork method) - identifies balanced opposing axes rather than standard PCA variance directions
Training data: 10 paired prompt continuations × 105 stem variations = 1,050 samples per class

Notes

This targets a stylistic pattern, not semantic content
Works because the construction has consistent activation signatures
Bidirectional control: choose the direction you want at application time

Limitations

Only works on patterns the base model already exhibits (not OOD steering)
Very high strengths (>2.0) may produce exaggerated effects
Designed for creative writing; effects on other tasks untested

Technical Implementation Notes (for researchers/developers)

Training control vectors for models of this scale required:

Custom llama.cpp modifications to extract and serialize hidden states
Conversion pipeline: binary dumps → 3D tensors → jukofyork format
Modified training code to work with pre-extracted activations
Tokenizer-only loading to avoid memory overhead

Acknowledgements

Based on the excellent work and cross-covariance eigendecomposition implementation by @jukofyork. His creative writing control vectors v3.0 repository contains the foundational code and methodology that made this possible, plus 70+ other creative writing control vectors across many models and useful steering directions. Highly recommended for anyone working on improving LLM prose quality!