Model Card for Rohit462/fine_tuned_distilgpt2_dialogsum

A DistilGPT-2 model fine-tuned using LoRA on the DialogSum dataset for English dialogue summarization. Generates concise summaries for dialogues in various topics like meetings, plans, and conversations.

Model Details

Model Description

Developed by: Rohit Rawat
Model type: Causal Language Model (GPT-2) with LoRA
Language(s): English
License: Apache-2.0
Finetuned from: distilgpt2

Model Sources

Repository: https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum

Uses

Direct Use

Useful for generating summaries from short conversations. Example: Summary: Two friends discussed their weekend plans. Topic: Weekend planning

Downstream Use

Can be integrated into meeting tools, chatbot logs, and dialogue-based analytics.

Out-of-Scope Use

Non-English text
Factual or long-form summarization
High-risk applications

Bias, Risks, and Limitations

May reflect biases from the DialogSum dataset
Accuracy may degrade on complex or domain-specific dialogue

Recommendations

Human validation is recommended
Avoid use in critical or factual applications

How to Get Started with the Model

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer, pipeline

model = AutoPeftModelForCausalLM.from_pretrained("Rohit462/fine_tuned_distilgpt2_dialogsum", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("Rohit462/fine_tuned_distilgpt2_dialogsum")
tokenizer.pad_token = tokenizer.eos_token

generator = pipeline("text-generation", model=model, tokenizer=tokenizer, device=model.device)
prompt = "Summary: Two friends planned a trip. Topic: Travel discussion"
output = generator(prompt, max_new_tokens=50, do_sample=True, top_p=0.9, temperature=0.7)
print(output[0]["generated_text"])

Training Details
Training Data
Subset of the DialogSum dataset, ~1000 samples used for fine-tuning.

Training Procedure
LoRA applied on c_attn layers

Epochs: 1

Batch Size: 4

LR: 2e-5 → 1e-5

Max length: 160

FP16 precision

Platform: Google Colab T4

Evaluation
Metric: Perplexity (TBD)

Evaluation: Manual review of summary coherence and topic alignment

Environmental Impact
Hardware Type: Google Colab T4 GPU

Training Time: ~30 minutes

Carbon Emitted: < 100g CO2eq (estimated)

Technical Specifications
Architecture: DistilGPT-2 with LoRA (c_attn only)

Libraries: Hugging Face Transformers, PEFT, TRL, PyTorch

Citation
BibTeX:
Training Details
Training Data
Subset of the DialogSum dataset, ~1000 samples used for fine-tuning.

Training Procedure
LoRA applied on c_attn layers

Epochs: 1

Batch Size: 4

LR: 2e-5 → 1e-5

Max length: 160

FP16 precision

Platform: Google Colab T4

Evaluation
Metric: Perplexity (TBD)

Evaluation: Manual review of summary coherence and topic alignment

Environmental Impact
Hardware Type: Google Colab T4 GPU

Training Time: ~30 minutes

Carbon Emitted: < 100g CO2eq (estimated)

Technical Specifications
Architecture: DistilGPT-2 with LoRA (c_attn only)

Libraries: Hugging Face Transformers, PEFT, TRL, PyTorch

Citation
BibTeX:
@misc{fine_tuned_distilgpt2_dialogsum,
  author = {Rohit Rawat},
  title = {Rohit462/fine_tuned_distilgpt2_dialogsum},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum}}
}

APA:
Rohit Rawat. (2025). Rohit462/fine_tuned_distilgpt2_dialogsum. Hugging Face. https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum

Model Card Contact
For questions or issues, open a discussion at:
https://huggingface.co/Rohit462/fine_tuned_distilgpt2_dialogsum/discussions