README.md

---
library_name: transformers
license: apache-2.0
datasets:
- Chillarmo/common_voice_20_armenian
language:
- hy
metrics:
- cer
- wer
- exact_match
base_model:
- openai/whisper-large-v3-turbo
pipeline_tag: automatic-speech-recognition
model-index:
- name: whisper-large-v3-turbo-armenian
  results:
  - task:
      type: automatic-speech-recognition
      name: Automatic Speech Recognition
    dataset:
      type: Chillarmo/common_voice_20_armenian
      name: Common Voice 20 Armenian
    metrics:
    - type: wer
      value: 15.31
      name: Word Error Rate
    - type: cer
      value: 2.86
      name: Character Error Rate
    - type: exact_match
      value: 42.73
      name: Exact Match
tags:
- speech-recognition
- armenian
- whisper
- fine-tuned
- large-v3-turbo
---

# Whisper Large v3 Turbo Armenian: High-Performance Armenian Speech Recognition

This model is a fine-tuned version of [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo) on the [Chillarmo/common_voice_20_armenian](https://huggingface.co/datasets/Chillarmo/common_voice_20_armenian) dataset. This model represents a significant advancement in Armenian automatic speech recognition, achieving state-of-the-art performance with substantially improved accuracy compared to smaller variants.

## Model Details

### Model Description

This is a fine-tuned Whisper Large v3 Turbo model specifically optimized for Armenian speech recognition. The model leverages the latest Whisper architecture improvements while maintaining the efficiency of the Turbo variant, providing an optimal balance between performance and computational requirements for Armenian language processing.

- **Developed by:** Movses Movsesyan (Independent Research)
- **Model type:** Automatic Speech Recognition
- **Language(s):** Armenian (hy)
- **License:** Apache 2.0
- **Finetuned from model:** [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo)

### Model Sources

- **Repository:** [Hugging Face Model Hub](https://huggingface.co/models)
- **Base Model:** [OpenAI Whisper](https://github.com/openai/whisper)
- **Paper:** [Robust Speech Recognition via Large-Scale Weak Supervision](https://arxiv.org/abs/2212.04356)

## Performance Highlights

🚀 **Exceptional Accuracy**: Achieves a **15.31% WER** and **2.86% CER** - representing significant improvements over smaller models:

| Model | WER | CER | Exact Match |
|-------|-----|-----|-------------|
| **Whisper Large v3 Turbo Armenian** | **15.31%** | **2.86%** | **42.73%** |
| Whisper Small Armenian v2 | 24.01% | 4.77% | 28.14% |

**Key Improvements:**
- **36% reduction** in Word Error Rate compared to the small model
- **40% reduction** in Character Error Rate
- **52% improvement** in Exact Match accuracy
- Superior performance while maintaining efficient inference speed

## Uses

### Direct Use

This model excels at transcribing Armenian speech to text with high accuracy, making it suitable for:

- **Production-grade** Armenian speech transcription systems
- Real-time Armenian voice interfaces with minimal errors
- Professional Armenian media content processing
- High-accuracy Armenian voice assistants and applications
- Academic and research applications in Armenian computational linguistics

### Downstream Use

The model can be integrated into enterprise and research applications such as:

- Professional Armenian voice assistants and chatbots
- High-quality subtitle generation for Armenian media
- Accessibility tools requiring high transcription accuracy
- Educational platforms for Armenian language learning
- Call center analytics and voice processing systems

### Out-of-Scope Use

This model should not be used for:

- Speech recognition in languages other than Armenian
- Speaker identification or verification tasks
- Audio classification beyond speech transcription
- Critical applications requiring 100% accuracy (medical/legal without human review)

## Bias, Risks, and Limitations

While this model achieves excellent performance, users should be aware of:

- **Domain specificity:** Performance may vary across different speaking styles and domains
- **Audio quality dependency:** Optimal performance requires reasonably clear audio input
- **Dialectal variations:** Performance may vary across different Armenian dialects
- **Computational requirements:** Larger model size requires more computational resources than smaller variants

### Recommendations

For optimal results:

- Test thoroughly on your specific use case and audio conditions
- Implement appropriate error handling and confidence thresholds
- Consider computational requirements when deploying at scale
- Monitor performance across different speaker demographics and accents

## How to Get Started with the Model

Use the code below to get started with the model:

```python
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
import torch

# Load the processor and model
processor = AutoProcessor.from_pretrained("Chillarmo/whisper-large-v3-turbo-armenian")
model = AutoModelForSpeechSeq2Seq.from_pretrained("Chillarmo/whisper-large-v3-turbo-armenian")

# Enable half precision for faster inference (optional)
model = model.half()
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

def transcribe_armenian(audio_path):
    """
    Transcribe Armenian audio file to text
    
    Args:
        audio_path (str): Path to audio file
        
    Returns:
        str: Transcribed text
    """
    import librosa
    
    # Load and process audio file
    audio, sr = librosa.load(audio_path, sr=16000)
    
    # Process the audio
    inputs = processor(audio, sampling_rate=16000, return_tensors="pt")
    inputs = inputs.to(device)
    
    # Generate transcription
    with torch.no_grad():
        predicted_ids = model.generate(
            inputs["input_features"],
            max_new_tokens=448,
            do_sample=False,
            use_cache=True
        )
    
    # Decode the transcription
    transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)
    return transcription[0]

# Example usage
# transcription = transcribe_armenian("path/to/armenian_audio.wav")
# print(f"Transcription: {transcription}")
```

### Batch Processing Example

```python
def transcribe_batch(audio_files):
    """
    Transcribe multiple audio files efficiently
    
    Args:
        audio_files (list): List of audio file paths
        
    Returns:
        list: List of transcriptions
    """
    transcriptions = []
    
    for audio_file in audio_files:
        transcription = transcribe_armenian(audio_file)
        transcriptions.append(transcription)
        
    return transcriptions

# Example batch processing
# audio_files = ["audio1.wav", "audio2.wav", "audio3.wav"]
# results = transcribe_batch(audio_files)
```

## Training Details

### Training Data

The model was fine-tuned on the [Chillarmo/common_voice_20_armenian](https://huggingface.co/datasets/Chillarmo/common_voice_20_armenian) dataset, utilizing the same high-quality Armenian speech data that has proven effective for Armenian speech recognition tasks.

### Training Procedure

#### Training Hyperparameters

The following hyperparameters were used during training:

- **Training regime:** Mixed precision training with optimized settings for Large v3 Turbo
- **Epochs:** 2.65 (optimal convergence achieved)
- **Training runtime:** 48,521 seconds (approximately 13.5 hours)
- **Training samples per second:** 1.649
- **Training steps per second:** 0.103
- **Final training loss:** 0.106
- **Total training steps:** 5,000

#### Performance Metrics During Training

- **Final evaluation loss:** 0.069
- **Training efficiency:** Optimized for the Large v3 Turbo architecture
- **Convergence:** Achieved excellent performance within 2.65 epochs

## Evaluation

### Testing Data, Factors & Metrics

#### Testing Data

The model was evaluated on a held-out test set from the Chillarmo/common_voice_20_armenian dataset using the same evaluation methodology as the baseline models for fair comparison.

#### Metrics

The model was evaluated using standard speech recognition metrics:

- **Word Error Rate (WER):** Percentage of words incorrectly transcribed
- **Character Error Rate (CER):** Percentage of characters incorrectly transcribed  
- **Exact Match:** Percentage of utterances transcribed perfectly

### Results

The fine-tuned model achieved exceptional performance on the evaluation set:

| Metric | Value | Improvement vs Small Model |
|--------|-------|---------------------------|
| **Word Error Rate (WER)** | **15.31%** | **-36.2%** (24.01% → 15.31%) |
| **Character Error Rate (CER)** | **2.86%** | **-40.0%** (4.77% → 2.86%) |
| **Exact Match** | **42.73%** | **+51.8%** (28.14% → 42.73%) |
| **Average Prediction Length** | 7.76 tokens | Consistent with ground truth |
| **Average Label Length** | 7.77 tokens | - |
| **Length Ratio** | 0.999 | Excellent length calibration |

#### Performance Analysis

The results demonstrate exceptional performance characteristics:

- **Superior accuracy:** Significant improvements across all metrics compared to smaller models
- **Length calibration:** Near-perfect length ratio (0.999) indicates excellent model calibration
- **Consistency:** High exact match rate (42.73%) shows the model frequently produces perfect transcriptions
- **Robustness:** Low character error rate (2.86%) indicates strong character-level understanding

## Technical Specifications

### Model Architecture and Objective

This model is based on the Whisper Large v3 Turbo architecture, featuring:

- **Encoder:** Advanced Transformer encoder processing mel-spectrogram features
- **Decoder:** Optimized Transformer decoder for efficient text generation
- **Architecture:** Enhanced Transformer sequence-to-sequence model
- **Model size:** Large v3 Turbo (optimized for efficiency)
- **Input:** 128-dimensional log mel-spectrograms
- **Output:** High-accuracy Armenian text transcriptions
- **Special features:** Turbo optimizations for faster inference while maintaining quality

### Compute Infrastructure

#### Hardware

Training was performed on the following hardware configuration:

- **Training duration:** ~13.5 hours for optimal convergence
- **Computational efficiency:** Optimized training pipeline for Large v3 Turbo architecture
- **Memory optimization:** Efficient memory usage during training and inference

#### Software

- **Framework:** Hugging Face Transformers (latest version)
- **Training library:** PyTorch with advanced optimization
- **Audio processing:** librosa, soundfile
- **Evaluation:** datasets, evaluate, jiwer
- **Optimization:** Mixed precision training for efficiency

## Deployment Considerations

### Hardware Requirements

**Minimum Requirements:**
- GPU: 8GB VRAM (for optimal performance)
- CPU: Modern multi-core processor
- RAM: 16GB system memory

**Recommended for Production:**
- GPU: 16GB+ VRAM for batch processing
- CPU: High-performance multi-core processor
- RAM: 32GB+ for large-scale deployment

### Performance Optimization Tips

1. **Use half precision** (`model.half()`) for faster inference
2. **Batch processing** for multiple audio files
3. **GPU acceleration** strongly recommended
4. **Caching** enabled for repeated inference tasks

## Citation

**BibTeX:**

```bibtex
@misc{movsesyan2025whisper-large-v3-turbo-armenian,
  author = {Movsesyan, Movses},
  title = {Whisper Large v3 Turbo Armenian},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/Chillarmo/whisper-large-v3-turbo-armenian}
}

@article{radford2022robust,
  title={Robust speech recognition via large-scale weak supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  journal={International Conference on Machine Learning},
  pages={28492--28518},
  year={2023},
  organization={PMLR}
}
```

**APA:**

Movsesyan, M. (2025). Whisper Large v3 Turbo Armenian. Hugging Face. https://huggingface.co/Chillarmo/whisper-large-v3-turbo-armenian

## Model Card Authors

This model card was created by Movses Movsesyan based on the fine-tuning results and comprehensive performance evaluation of the Whisper Large v3 Turbo Armenian model.

---

## Acknowledgments

Special thanks to the OpenAI team for the Whisper architecture and the Common Voice project contributors for providing high-quality Armenian speech data that made this research possible.