Spaces:
Sleeping
Sleeping
File size: 3,576 Bytes
fd80dd0 0738679 fd80dd0 e926fee 398bf48 e926fee 398bf48 e926fee 398bf48 fd80dd0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 |
---
license: mit
title: LoRA Fine-Tuning and CoreML Conversion with Streamlit
sdk: streamlit
emoji: 🦀
colorFrom: green
colorTo: blue
pinned: true
sdk_version: 1.46.0
---
# LoRA Fine-Tuning and CoreML Conversion with Streamlit
This project demonstrates how to fine-tune a large language model (LLM) using Low-Rank Adaptation (LoRA) and convert it to the CoreML format for on-device deployment. The entire process is wrapped in a user-friendly Streamlit web application.
## Features
- **Fine-Tune LLMs with LoRA:** Easily fine-tune the `distilbert/distilgpt2` model on the `roneneldan/TinyStories` dataset using Parameter-Efficient Fine-Tuning (PEFT) with LoRA.
- **Text Generation:** Generate creative stories from a text prompt using the fine-tuned model.
- **Adjustable Generation Parameters:** Control text generation with parameters like temperature, max length, and repetition penalty.
- **CoreML Conversion:** Convert the fine-tuned model to a `.mlpackage` file, ready for integration into Apple ecosystem applications.
- **Interactive Web UI:** A simple and interactive user interface built with Streamlit.
## How it Works
The application follows a simple workflow:
1. **Load Base Model:** It starts by loading the pre-trained `distilbert/distilgpt2` model and its tokenizer from the Hugging Face Hub.
2. **Fine-Tuning:** The user can initiate the fine-tuning process. The application uses the `peft` library to apply LoRA to the base model and trains it on the `TinyStories` dataset. The resulting LoRA adapter is saved locally.
3. **Text Generation:** Once the model is fine-tuned (or a pre-existing adapter is loaded), you can provide a prompt to generate stories.
4. **CoreML Conversion:** The application can merge the LoRA adapter with the base model and then convert the merged model into the CoreML format, which can be downloaded as a `.zip` file.
## Technologies Used
- **Model:** `distilbert/distilgpt2` from Hugging Face
- **Dataset:** `roneneldan/TinyStories` from Hugging Face
- **Fine-Tuning:** `peft` (Parameter-Efficient Fine-Tuning) library with LoRA
- **Framework:** PyTorch
- **Web App:** Streamlit
- **Model Conversion:** CoreMLTools
- **Core Libraries:** `transformers`, `datasets`, `accelerate`, `sentencepiece`
## Setup and Usage
1. **Clone the repository:**
```bash
git clone <repository-url>
cd <repository-directory>
```
2. **Create a virtual environment and install dependencies:**
```bash
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
3. **Run the Streamlit application:**
```bash
streamlit run app.py
```
4. **Open the application in your browser:**
Navigate to the URL provided by Streamlit (usually `http://localhost:8501`).
## File Structure
- `app.py`: The main Python script containing the Streamlit application logic.
- `requirements.txt`: A list of the Python packages required to run the project.
- `README.md`: This file, providing information about the project.
- `distilgpt2-lora-tinystories/`: (Generated Directory) This directory will be created to store the LoRA adapter after fine-tuning.
- `results/`: (Generated Directory) This directory is used by the `transformers.Trainer` to save training outputs.
- `distilgpt2-lora-tinystories.mlpackage/`: (Generated Directory) This directory will be created after the CoreML conversion.
- `distilgpt2-lora-tinystories.zip`: (Generated File) The zipped CoreML model ready for download.
---
Happy fine-tuning and story generating! |