Create README.md

README.md

@@ -0,0 +1,351 @@
+# Model Card: GPT-OSS-20B Spanish Climate Change (Civility-Conditioned)
+
+## Model Description
+
+This is a fine-tuned version of [gpt-oss-20b](https://huggingface.co/OpenSourceAI/gpt-oss-20b) trained on Spanish Facebook discussions about climate change with **civility conditioning**. The model learns to generate realistic Spanish social media posts while allowing users to control the civility level at inference time.
+
+- **Model Type**: Autoregressive Language Model (Causal LM)
+- **Base Model**: gpt-oss-20b (20 billion parameters)
+- **Language**: Spanish
+- **Training Objective**: Masked posting prediction with civility conditioning
+- **License**: [Same as base model]
+- **Model Size**: 20B parameters
+
+## Key Features
+
+### 1. Civility Control
+The model can generate both civil and incivil content, controlled by a simple parameter:
+- `[CIVILITY: civil]` → Generates respectful, constructive discourse
+- `[CIVILITY: incivil]` → Generates content with rudeness, hate speech, or threats
+
+### 2. Realistic Spanish Social Media Generation
+Trained on authentic Facebook discussions about climate change in Spanish, capturing:
+- Natural conversation patterns
+- Topic-appropriate vocabulary
+- Social media communication styles
+- Diverse perspectives and argumentation patterns
+
+### 3. Context-Aware Continuation
+Uses masked posting prediction to generate responses that:
+- Fit naturally within ongoing discussions
+- Maintain topical coherence
+- Respond appropriately to previous comments
+- Match the conversational depth and complexity
+
+## Training Data
+
+### Source
+- **Platform**: Facebook public posts
+- **Topic**: Climate change discussions (Spanish language)
+- **Format**: Conversation sequences with incivility annotations
+
+### Dataset Statistics
+- **Total Training Examples**: 5,875
+- **Validation Examples**: 653
+- **Civil Examples**: ~65% of dataset
+- **Incivil Examples**: ~35% of dataset
+
+### Incivility Annotations
+Each post was labeled across multiple dimensions:
+- **Impoliteness (imp)**: Rudeness, disrespect
+- **Hate Speech/Stereotyping (hsst)**: Discriminatory language
+- **Threats (threat)**: Threats to democratic freedoms
+- **Negativity (neg)**: General negativity
+- **Attack (att)**: Personal attacks
+- **Sarcasm/Cynicism (sarcyn)**: Sarcastic or cynical tone
+
+A post is classified as **incivil** if it has `imp ≥ 1` OR `hsst ≥ 1` OR `threat ≥ 1`.
+
+## Training Procedure
+
+### Hyperparameters
+- **Epochs**: 3
+- **Batch Size**: 2 per device
+- **Gradient Accumulation Steps**: 8 (effective batch size: 16)
+- **Learning Rate**: 2e-5
+- **Warmup Steps**: 100
+- **Weight Decay**: 0.01
+- **Max Gradient Norm**: 1.0
+- **Precision**: bfloat16
+- **Max Sequence Length**: 2048 tokens
+- **Optimizer**: AdamW
+
+### Training Features
+- Gradient checkpointing for memory efficiency
+- Dynamic padding via custom data collator
+- Loss computed only on the final response (not context or instruction)
+- Balanced exposure to civil and incivil examples
+- Reasoning level variation (low/medium/high) for diversity
+
+### Infrastructure
+- Trained on high-performance GPU cluster
+- Model distributed across GPUs using `device_map="auto"`
+- TensorBoard logging enabled
+
+## Usage
+
+### Basic Example
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# Load model
+model_path = "maxpe/gpt-oss-spanish-climate-conditioned"
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+model = AutoModelForCausalLM.from_pretrained(
+    model_path,
+    torch_dtype=torch.bfloat16,
+    device_map="auto"
+)
+
+# Format prompt with civility control
+def format_prompt(context: str, civility_type: str = "civil") -> str:
+    if civility_type == "civil":
+        civility_instruction = "Esta publicación debe ser civil y respetuosa, sin elementos de descortesía, discurso de odio o amenazas."
+    else:
+        civility_instruction = "Esta publicación debe contener elementos de incivilidad (descortesía, discurso de odio, o amenazas)."
+
+    instruction = f"""[CIVILITY: {civility_type}]
+
+Necesitas escribir una respuesta de Facebook que continúe esta discusión sobre cambio climático. {civility_instruction}
+
+Aquí está el hilo de conversación:
+
+{context}"""
+
+    system_message = """<|start|>system<|message|>Eres un asistente de IA que participa en discusiones de Facebook sobre cambio climático en español. Tu tarea es generar publicaciones y comentarios que reflejen el tono y estilo de las conversaciones reales. Puedes generar contenido civil o incivil según se especifique en el parámetro [CIVILITY].
+
+Reasoning: medium
+
+# Valid channels: analysis, final<|end|>"""
+
+    user_message = f"""<|start|>user<|message|>{instruction}<|end|>"""
+    assistant_start = """<|start|>assistant<|message|><|start|>final<|message|>"""
+
+    return f"{system_message}\n\n{user_message}\n\n{assistant_start}"
+
+# Generate civil response
+context = """**[Publicación Original]**
+El gobierno debe prohibir los vehículos diésel inmediatamente.
+
+**[Respuesta 1]**
+Estoy de acuerdo, la calidad del aire es terrible."""
+
+prompt = format_prompt(context, civility_type="civil")
+inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=1536)
+inputs = {k: v.to(model.device) for k, v in inputs.items()}
+
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=256,
+    temperature=0.8,
+    top_p=0.92,
+    top_k=50,
+    repetition_penalty=1.2,
+    do_sample=True,
+    pad_token_id=tokenizer.pad_token_id,
+)
+
+response = tokenizer.decode(outputs[0], skip_special_tokens=False)
+# Extract text between <|start|>final<|message|> and <|end|>
+```
+
+### Quick Testing
+
+Use the included test script:
+```bash
+# Run pre-built test scenarios
+python test_spanish_conditioned_model.py
+
+# Interactive mode
+python test_spanish_conditioned_model.py --interactive
+
+# Adjust generation parameters
+python test_spanish_conditioned_model.py --temperature 0.9 --max_new_tokens 250
+```
+
+## Generation Parameters
+
+Recommended settings based on testing:
+- **max_new_tokens**: 256 (for typical social media posts)
+- **temperature**: 0.8 (balance between creativity and coherence)
+- **top_p**: 0.92 (nucleus sampling)
+- **top_k**: 50 (top-k sampling)
+- **repetition_penalty**: 1.2 (avoid repetitive text)
+- **do_sample**: True (enable sampling for diversity)
+
+## Model Capabilities
+
+### What the Model Does Well
+
+✅ Generate natural-sounding Spanish social media posts about climate change
+
+✅ Control civility level via simple parameter switch
+
+✅ Continue conversations with contextual awareness
+
+✅ Maintain topical coherence in climate discussions
+
+✅ Produce varied responses across different conversation contexts
+
+✅ Reflect authentic social media communication patterns
+
+### Limitations
+
+⚠️ Trained only on Facebook data (may not generalize perfectly to Twitter, Reddit, etc.)
+
+⚠️ Climate change domain-specific (not general-purpose Spanish generation)
+
+⚠️ Incivil content reflects training data patterns (may not cover all forms of toxicity)
+
+⚠️ No explicit fact-checking or accuracy verification
+
+⚠️ May occasionally generate biased or controversial viewpoints from training data
+
+## Intended Use Cases
+
+### Research Applications (Recommended)
+- **Discourse Analysis**: Study patterns of civil vs incivil climate discourse
+- **Content Moderation**: Generate training data for toxicity detection systems
+- **Counter-Speech Research**: Create civil responses to incivil content
+- **Polarization Studies**: Analyze dynamics of climate change discussions
+- **Data Augmentation**: Expand datasets with realistic synthetic examples
+- **Educational Tools**: Demonstrate effects of incivility in online discussions
+
+### Out-of-Scope Uses
+
+❌ Deploying to generate actual social media content at scale
+
+❌ Creating content intended to harass, mislead, or manipulate
+
+❌ Generating incivil content for malicious purposes
+
+❌ Using as a factual information source about climate science
+
+## Ethical Considerations
+
+### Incivil Content Generation
+This model can generate incivil content including rudeness, hate speech, and threats. This capability is designed **exclusively for research purposes**:
+- Understanding and mitigating online toxicity
+- Training content moderation systems
+- Studying polarization and discourse patterns
+- Developing counter-speech interventions
+
+**Users must ensure responsible use** and not deploy this model to create harmful content in real online environments.
+
+### Bias and Fairness
+- The model reflects biases present in Spanish Facebook climate discussions
+- Training data may contain controversial or fringe perspectives
+- Generated content should not be treated as factual or representative of consensus views
+- Researchers should be aware of potential demographic, political, and ideological biases
+
+### Privacy
+- Trained on public Facebook posts
+- No personally identifiable information (PII) should be present in outputs
+- Users should verify generated content doesn't inadvertently reproduce private information
+
+## Evaluation
+
+### Training Metrics
+- **Final Training Loss**: [See train_results.json]
+- **Final Validation Loss**: [See eval_results.json]
+- **Best Checkpoint**: Selected based on lowest validation loss
+
+### Qualitative Assessment
+The model successfully:
+- Generates fluent Spanish text
+- Switches between civil and incivil tones based on the civility parameter
+- Maintains relevance to climate change topics
+- Produces contextually appropriate responses
+
+### Recommended Validation
+Users should:
+- Manually inspect generated outputs for quality and appropriateness
+- Validate factual claims if using for educational purposes
+- Check for unintended biases or problematic content
+- Test across diverse conversation contexts
+
+## Technical Specifications
+
+### Model Architecture
+- **Base**: gpt-oss-20b (20B parameter decoder-only transformer)
+- **Format**: Harmony conversation format (OpenAI-style)
+- **Channels**: analysis, final (model uses structured reasoning)
+- **Vocabulary**: [From tokenizer - includes Spanish tokens]
+
+### Input Format
+Uses structured prompt with:
+1. System message (defines model role and capabilities)
+2. User instruction (includes `[CIVILITY: civil|incivil]` parameter)
+3. Conversation context (previous posts in thread)
+4. Assistant prefix (triggers generation)
+
+### Output Format
+Model generates:
+1. **Analysis channel** (optional): Internal reasoning about the task
+2. **Final channel**: The actual social media post response
+
+**Recommendation**: Skip directly to final channel at inference using `<|start|>final<|message|>` prefix.
+
+### Special Tokens
+- `<|start|>`, `<|end|>`: Conversation structure markers
+- `<|message|>`: Message content delimiter
+- Standard tokenizer special tokens (BOS, EOS, PAD)
+
+## Files and Artifacts
+
+### Model Files
+- `model-*.safetensors`: Model weights (9 shards)
+- `model.safetensors.index.json`: Weight shard index
+- `config.json`: Model configuration
+- `generation_config.json`: Default generation settings
+
+### Tokenizer Files
+- `tokenizer.json`: Tokenizer vocabulary and rules
+- `tokenizer_config.json`: Tokenizer configuration
+- `special_tokens_map.json`: Special token mappings
+- `chat_template.jinja`: Chat template (if applicable)
+
+### Training Files
+- `training_args.bin`: Complete training arguments
+- `train_results.json`: Final training metrics
+- `eval_results.json`: Final evaluation metrics
+- `all_results.json`: Combined results
+- `checkpoint-*/`: Training checkpoints (600, 1000, 1104)
+- `logs/`: TensorBoard training logs
+
+## Citation
+
+If you use this model in your research, please cite:
+
+```bibtex
+@misc{gpt-oss-spanish-climate-conditioned,
+  title={GPT-OSS-20B Spanish Climate Change Model with Civility Conditioning},
+  author={Max Pellert},
+  institution={Barcelona Supercomputing Center (BSC)},
+  year={2025},
+  note={Fine-tuned on Spanish Facebook climate discussions}
+}
+```
+
+## Acknowledgments
+
+- **Base Model**: gpt-oss-20b by OpenSourceAI
+- **Framework**: Hugging Face Transformers
+- **Data Source**: Spanish Facebook public climate change discussions
+- **Annotation Framework**: Multi-dimensional incivility labeling scheme
+
+## Version History
+
+- **v1.0** (2025-11): Initial release
+  - 5,875 training examples
+  - 3 epochs
+  - Civility-conditioned generation
+  - Checkpoint 1104 (best validation loss)
+
+## Additional Resources
+
+- **Test Script**: `test_spanish_conditioned_model.py` - Run inference tests with various scenarios
+- **Training Script**: `finetune_spanish_conditioned.py` - Complete training configuration
+- **Dataset Creation**: `create_spanish_conditioned_more_sequences.py` - Dataset preparation pipeline (extracts 20 sequences per thread vs 3 in original version)