README.md

---
license: apache-2.0
base_model: unsloth/gemma-3-4b-it
tags:
- gemma3
- unsloth
- conversational
- education
- instruction-tuning
- question-answering
- bengali
- indian-universities
- trl
- sft
language:
- en
datasets:
- millat/indian_university_guidance_for_bangladeshi_students
metrics:
- perplexity
- loss
library_name: transformers
pipeline_tag: text-generation
---

# Gemma-3-4B Indian University Guide for Bangladeshi Students

<div align="center">
  <img src="https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo-with-title.png" width="200"/>
  
  **A specialized educational counselor AI fine-tuned on 7,044 high-quality Q&A pairs**
  
  [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
  [![Model](https://img.shields.io/badge/Model-Gemma--3--4B-green.svg)](https://huggingface.co/google/gemma-3-4b-it)
  [![Dataset](https://img.shields.io/badge/Dataset-7K%20QA%20Pairs-orange.svg)](https://huggingface.co/datasets/millat/indian_university_guidance_for_bangladeshi_students)
  [![Training](https://img.shields.io/badge/Training-Unsloth%202x%20Faster-red.svg)](https://github.com/unslothai/unsloth)
</div>

---

## 📋 Model Description

**Gemma-3-4B Indian University Guide** is a fine-tuned Large Language Model specifically designed to assist Bangladeshi students in navigating the admission process for Indian universities. The model provides accurate, culturally-sensitive guidance on topics including:

- 🎓 **Admissions Requirements** - Entry criteria, eligibility, and application processes
- 📄 **Documentation** - Required documents, equivalence certificates, and attestation
- 💰 **Scholarships** - Merit-based scholarships, GPA requirements, and eligibility
- 🏫 **University Information** - Programs, fees, accommodation, and facilities
- 🛂 **Visa Guidance** - Student visa process, requirements, and timelines
- 📊 **Grade Conversion** - Bangladesh to India GPA/percentage equivalence
- 🔄 **Lateral Entry** - Polytechnic diploma to B.Tech admission pathways
- 🎯 **Program Equivalence** - Degree recognition between Bangladesh and India

### Model Details

- **Developed by:** [MD Millat Hosen](https://huggingface.co/millat)
- **Model type:** Causal Language Model (Instruction-tuned)
- **Base Model:** [unsloth/gemma-3-4b-it](https://huggingface.co/unsloth/gemma-3-4b-it)
- **Language:** English
- **License:** Apache 2.0
- **Parameters:** 4 Billion
- **Fine-tuning Method:** QLoRA (Quantized Low-Rank Adaptation)
- **Training Framework:** Unsloth + HuggingFace TRL
- **Precision:** 16-bit (BF16)
- **Context Length:** 1024 tokens

---

## 🎯 Intended Use

### Primary Use Cases

1. **Educational Counseling Chatbot** - Deploy as an AI assistant for Bangladeshi students seeking admission to Indian universities
2. **University Admission Support** - Provide instant, accurate answers about admission requirements, processes, and eligibility
3. **Scholarship Guidance** - Help students understand scholarship criteria and calculate their eligibility
4. **Document Preparation** - Guide students through required documentation and equivalence procedures
5. **Research Applications** - Academic research on instruction-tuned LLMs for specialized domains

### Target Users

- 🎓 Bangladeshi students applying to Indian universities
- 🏢 Educational consultancy firms
- 🏫 University admission offices
- 📚 Academic researchers in NLP and education technology

---

## 📊 Training Details

### Dataset

**Dataset:** [millat/indian_university_guidance_for_bangladeshi_students](https://huggingface.co/datasets/millat/indian_university_guidance_for_bangladeshi_students)

- **Size:** 7,044 instruction-formatted Q&A pairs
- **Format:** Question-Answer with context and metadata
- **Quality:** Multi-stage pipeline with deduplication and validation
- **Coverage:** Comprehensive guidance across 8 major topics
- **Cultural Sensitivity:** Designed specifically for Bangladesh-India educational context

**Data Split:**
- Training: 90% (6,340 examples)
- Validation: 10% (704 examples)

### Training Configuration

```python
Training Parameters:
  - Epochs: 3
  - Batch Size: 2 per device
  - Gradient Accumulation Steps: 8
  - Effective Batch Size: 16
  - Learning Rate: 2e-5 (cosine schedule)
  - Warmup Steps: 100
  - Max Sequence Length: 1024 tokens
  - Optimizer: AdamW 8-bit
  - Weight Decay: 0.01
  
LoRA Configuration:
  - Rank (r): 16
  - Alpha: 16
  - Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  - Dropout: 0
  - Bias: None
  
Hardware:
  - GPU: NVIDIA T4 (Google Colab)
  - Training Time: ~45 minutes
  - Speed: 2x faster with Unsloth optimizations
```

### Training Results

| Metric | Value | Assessment |
|--------|-------|------------|
| **Final Training Loss** | 0.593 | Excellent |
| **Validation Loss** | 0.614 | Excellent |
| **Perplexity** | 1.85 | Excellent |
| **Improvement vs Base** | 38% | Strong |
| **Trainable Parameters** | 83.9M (2.09%) | Efficient |

**Training Notebook:** [Google Colab](https://colab.research.google.com/drive/1rmH5p_PtlTlLaakc_Fmlb0jXxGvI3tAJ?usp=sharing)

---

## 🚀 How to Use

### Installation

```bash
pip install unsloth transformers accelerate peft bitsandbytes
```

### Basic Inference

```python
from unsloth import FastLanguageModel
import torch

# Load model and tokenizer
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="millat/gemma4b-indian-university-guide-16bit",
    max_seq_length=1024,
    dtype=None,  # Auto-detect
    load_in_4bit=True,  # Use 4-bit quantization for efficiency
)

# Prepare for inference
FastLanguageModel.for_inference(model)

# Format your question
question = "What documents do I need to apply to Indian universities from Bangladesh?"

# Create prompt in Gemma3 format
prompt = f"<start_of_turn>user\n{question}<end_of_turn>\n<start_of_turn>model\n"

# Tokenize
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

# Generate response
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
    top_k=50,
    repetition_penalty=1.2,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

# Decode and print
response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
print(response)
```

### Advanced Usage with Streaming

```python
from transformers import TextStreamer

# Create streamer for real-time output
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

# Generate with streaming
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
    streamer=streamer,  # Enable streaming
)
```

### Batch Inference

```python
questions = [
    "Can I get a scholarship at Sharda University with a GPA of 3.5?",
    "What is the admission process for Bangladeshi students?",
    "Am I eligible for lateral entry with a Polytechnic diploma?"
]

for question in questions:
    prompt = f"<start_of_turn>user\n{question}<end_of_turn>\n<start_of_turn>model\n"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    
    outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7)
    response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
    
    print(f"Q: {question}")
    print(f"A: {response}\n")
```

### API Integration Example (Flask)

```python
from flask import Flask, request, jsonify
from unsloth import FastLanguageModel

app = Flask(__name__)

# Load model once at startup
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="millat/gemma4b-indian-university-guide-16bit",
    max_seq_length=1024,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

@app.route('/ask', methods=['POST'])
def ask():
    data = request.json
    question = data.get('question', '')
    
    prompt = f"<start_of_turn>user\n{question}<end_of_turn>\n<start_of_turn>model\n"
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    
    outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
    response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)
    
    return jsonify({'answer': response})

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=5000)
```

---

## 📈 Performance Evaluation

### Comprehensive Testing Results

The model underwent extensive evaluation across 5 dimensions:

| Test Category | Metric | Score | Grade |
|--------------|--------|-------|-------|
| **Factual Accuracy** | Correctness | 87.5% | A+ ⭐⭐⭐ |
| **Topic Coverage** | Domain Coverage | 100% | A+ ⭐⭐⭐ |
| **Consistency** | Response Stability | 80% | A ⭐⭐ |
| **Response Quality** | Coherence & Length | High (σ=8.5) | A+ ⭐⭐⭐ |
| **Overall Performance** | Weighted Average | 93.5% | A+ ⭐⭐⭐ |

**Status:** ✅ **PRODUCTION READY**

### Example Outputs

#### Example 1: Scholarship Query

**Input:**
```
Can I get a scholarship at Sharda University with a GPA of 3.5?
```

**Output:**
```
Yes, you are eligible for a 50% scholarship on tuition fees at Sharda University! 
With a GPA of 3.5 out of 5.0 (equivalent to 70% or CGPA 7.0/10 in India), you 
qualify for the 50% scholarship tier. This scholarship is available for eligible 
engineering programs like B.Tech (which is equivalent to B.Sc. Engineering in 
Bangladesh). To maintain the scholarship from the second year onwards, you must:
1) Pass all exams without any backlogs
2) Maintain at least 75% attendance
The scholarship covers 50% of your tuition fees throughout your program duration.
```

#### Example 2: Lateral Entry

**Input:**
```
Am I eligible for lateral entry with a Polytechnic diploma?
```

**Output:**
```
Yes, you are eligible for lateral entry into the second year of B.Tech/B.E. 
programs in India with your Polytechnic Diploma from Bangladesh. Your 4-year 
diploma after SSC is recognized for direct admission to the second year. Make 
sure the Indian university's B.Tech program is AICTE-approved to ensure your 
degree is recognized in Bangladesh. You'll need to provide your diploma 
certificate, transcripts, and equivalence certificate from the Bangladesh 
Technical Education Board (BTEB) for the admission process.
```

#### Example 3: Document Requirements

**Input:**
```
What documents do I need to apply to Indian universities from Bangladesh?
```

**Output:**
```
For applying to Indian universities from Bangladesh, you'll need:

1. Academic Documents:
   - HSC Certificate and Mark Sheet
   - SSC Certificate and Mark Sheet
   - Academic transcripts for all years

2. Identification:
   - Valid passport
   - Recent passport-size photographs

3. Equivalence Certificate:
   - From Bangladesh Ministry of Education or UGC
   - Shows your qualifications are equivalent to Indian standards

4. Additional Documents:
   - Transfer Certificate (TC) from your last institution
   - Character Certificate
   - Medical fitness certificate
   - Gap certificate (if applicable)

All documents should be attested by the Bangladesh Ministry of Foreign Affairs 
and the Indian High Commission in Dhaka for visa purposes.
```

---

## ⚙️ Model Architecture

### Base Model: Gemma-3-4B-IT

- **Architecture:** Decoder-only Transformer
- **Attention:** Multi-head attention with rotary positional embeddings
- **Activation:** GeGLU
- **Normalization:** RMSNorm
- **Vocabulary Size:** 256,000 tokens
- **Hidden Size:** 2,560
- **Intermediate Size:** 15,360
- **Number of Layers:** 26
- **Attention Heads:** 16
- **Key-Value Heads:** 4 (Grouped-Query Attention)

### LoRA Adaptations

Fine-tuning was performed using QLoRA with the following adapter configuration:

```python
LoRA Config:
  - Target Modules: [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj]
  - Rank (r): 16
  - Alpha: 16
  - Dropout: 0.0
  - Task Type: Causal Language Modeling
  - Trainable Parameters: 83,886,080 (2.09% of total)
  - Total Parameters: 4,013,133,568
```

---

## 💡 Prompt Format

The model is trained using the **Gemma3 Chat Template**:

```
<start_of_turn>user
{Your question here}<end_of_turn>
<start_of_turn>model
{Model's response}<end_of_turn>
```

**Important:** Always use this format for optimal performance. The tokenizer's `apply_chat_template()` method handles this automatically.

---

## 🔧 Technical Specifications

### Memory Requirements

| Precision | Memory Usage | Inference Speed |
|-----------|--------------|-----------------|
| 16-bit (BF16) | ~8 GB VRAM | Baseline |
| 8-bit | ~4 GB VRAM | 1.2x faster |
| 4-bit (NF4) | ~2.5 GB VRAM | 2x faster |

**Recommended:** Use 4-bit quantization for deployment (load_in_4bit=True)

### Generation Parameters

For optimal results, use these parameters:

```python
generation_config = {
    "max_new_tokens": 200-256,     # Adjust based on expected answer length
    "temperature": 0.7,             # 0.3 for factual, 0.7 for conversational
    "top_p": 0.9,                   # Nucleus sampling
    "top_k": 50,                    # Top-k sampling
    "repetition_penalty": 1.2,      # Prevent repetition
    "no_repeat_ngram_size": 3,      # Block repeated 3-grams
    "do_sample": True,              # Enable sampling
    "early_stopping": True,         # Stop at EOS token
}
```

---

## ⚠️ Limitations

### Known Limitations

1. **Temporal Knowledge Cutoff** - Information is based on data collected at a specific point in time (October 2025) and may become outdated as university policies change.

2. **Scope Limitation** - The model is specialized for Bangladeshi students applying to Indian universities. It may not generalize well to:
   - Other countries' education systems
   - General-purpose conversational tasks
   - Non-educational domains

3. **Factual Accuracy** - While the model achieves 87.5% factual accuracy, always verify critical information (fees, deadlines, requirements) with official university sources.

4. **University Coverage** - The dataset focuses on major Indian universities accepting Bangladeshi students. Smaller or newer institutions may have limited coverage.

5. **Language** - The model operates in English only. It does not support Bengali/Bangla language queries.

6. **Hallucination Risk** - Like all LLMs, the model may occasionally generate plausible-sounding but incorrect information. Use with appropriate supervision.

### Ethical Considerations

- **Advisory Role Only** - This model should supplement, not replace, professional educational counseling.
- **Verification Required** - Students should verify all information with official university websites before making decisions.
- **Cultural Sensitivity** - The model is designed with cultural awareness but may not capture all nuances of individual circumstances.
- **Bias Awareness** - The model reflects the biases present in the training data and base model.

---

## 📚 Citation

If you use this model in your research or applications, please cite:

```bibtex
@misc{millat2025gemma4b_indian_university_guide,
  author = {MD Millat Hosen and Md Moudud Ahmed Misil and Dr. Rohit Kumar Sachan},
  title = {Gemma-3-4B Indian University Guide for Bangladeshi Students},
  year = {2025},
  publisher = {HuggingFace},
  journal = {HuggingFace Model Hub},
  howpublished = {\url{https://huggingface.co/millat/gemma4b-indian-university-guide-16bit}},
}
```

**Dataset Citation:**
```bibtex
@misc{md_millat_hosen_2025,
  author = {MD Millat Hosen and Md Moudud Ahmed Misil and Dr. Rohit Kumar Sachan},
  title = {indian_university_guidance_for_bangladeshi_students},
  year = {2025},
  url = {https://huggingface.co/datasets/millat/indian_university_guidance_for_bangladeshi_students},
  doi = {10.57967/hf/6295},
  publisher = {Hugging Face}
}
```

---

## 🤝 Contributing

We welcome contributions to improve this model! Areas for contribution:

- 📊 **Dataset Expansion** - Add more universities, update policies, expand coverage
- 🧪 **Evaluation** - Conduct additional testing and provide feedback
- 🐛 **Bug Reports** - Report issues or incorrect responses
- 📝 **Documentation** - Improve usage guides and examples
- 🌐 **Deployment** - Share deployment experiences and best practices

---

## 📞 Contact & Support

- **Model Author:** [MD Millat Hosen](https://huggingface.co/millat)
- **Issues:** Report on HuggingFace Model Hub
- **Updates:** Follow for model updates and improvements

---

## 🙏 Acknowledgments

- **Google DeepMind** - For the excellent Gemma-3-4B base model
- **Unsloth AI** - For 2x faster training optimizations
- **HuggingFace** - For the Transformers library and model hosting
- **TRL Team** - For Supervised Fine-Tuning utilities
- **Research Supervisor** - Dr. Rohit Kumar Sachan
- **Team Member** - Md Moudud Ahmed Misil

---

## 📄 License

This model is released under the **Apache 2.0 License**, inherited from the base Gemma-3-4B model.

- ✅ Commercial use allowed
- ✅ Modification allowed
- ✅ Distribution allowed
- ✅ Private use allowed
- ⚠️ Must include license and copyright notice
- ⚠️ Must state changes made

---

## 🔗 Related Resources

- 📊 **Dataset:** [indian_university_guidance_for_bangladeshi_students](https://huggingface.co/datasets/millat/indian_university_guidance_for_bangladeshi_students)
- 🤖 **Base Model:** [unsloth/gemma-3-4b-it](https://huggingface.co/unsloth/gemma-3-4b-it)
- 📓 **Training Notebook:** [Google Colab](https://colab.research.google.com/drive/1rmH5p_PtlTlLaakc_Fmlb0jXxGvI3tAJ?usp=sharing)
- 🔧 **Unsloth Library:** [GitHub](https://github.com/unslothai/unsloth)
- 📖 **Documentation:** [Unsloth Docs](https://docs.unsloth.ai/)

---

<div align="center">
  
**🎓 Empowering Bangladeshi Students to Achieve Their Dreams in India 🇧🇩 🤝 🇮🇳**

*Built with ❤️ using Unsloth + HuggingFace*

[![HuggingFace](https://img.shields.io/badge/🤗-HuggingFace-yellow.svg)](https://huggingface.co/millat/gemma4b-indian-university-guide-16bit)
[![Dataset](https://img.shields.io/badge/📊-Dataset-blue.svg)](https://huggingface.co/datasets/millat/indian_university_guidance_for_bangladeshi_students)
[![Colab](https://img.shields.io/badge/📓-Colab-orange.svg)](https://colab.research.google.com/drive/1rmH5p_PtlTlLaakc_Fmlb0jXxGvI3tAJ?usp=sharing)

</div>