|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- qlora |
|
|
- verilog |
|
|
- code-generation |
|
|
- circuit-synthesis |
|
|
- electronic-design-automation |
|
|
language: en |
|
|
datasets: |
|
|
- bnadimi/PyraNet-Verilog |
|
|
model-index: |
|
|
- name: veriforge-deepseek-coder-1.3b-instruct |
|
|
results: [] |
|
|
--- |
|
|
|
|
|
# veriforge-deepseek-coder-1.3b-instruct |
|
|
|
|
|
This model is a QLoRA fine-tuned version of [`deepseek-ai/deepseek-coder-1.3b-instruct`](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct), designed for the domain of **Verilog RTL synthesis**. It accepts natural-language descriptions of digital circuits and generates Verilog code modules. |
|
|
|
|
|
## ✨ Model Details |
|
|
- **Base Model**: DeepSeek-Coder-1.3B-Instruct (4-bit quantized) |
|
|
- **Fine-Tuning**: QLoRA on Hugging Face `Trainer` API |
|
|
- **Domain**: Hardware Description Language (HDL), Electronic Design Automation (EDA) |
|
|
- **Tokenizer**: AutoTokenizer with trust_remote_code=True |
|
|
|
|
|
## 📚 Dataset |
|
|
- **Source**: [PyraNet-Verilog](https://huggingface.co/datasets/bnadimi/PyraNet-Verilog) |
|
|
- **Content**: Natural-language descriptions paired with their corresponding Verilog implementations |
|
|
- **Preprocessing**: Reformatted into instruction-style prompts with markdown headers |
|
|
|
|
|
## 🧪 Training Configuration |
|
|
- Framework: `transformers`, `peft`, `accelerate`, `bitsandbytes` |
|
|
- Epochs: 10 |
|
|
- Batch Size: 4 (with gradient accumulation of 4) |
|
|
- Optimizer: AdamW |
|
|
- Learning Rate: 2e-4 |
|
|
- Device: Google Colab GPU (supports A100/T4) |
|
|
- Precision: 4-bit (QLoRA) + FP16 mixed-precision |
|
|
|
|
|
## 🚀 Usage |
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
model = AutoModelForCausalLM.from_pretrained("louijiec/veriforge-deepseek-coder-1.3b-instruct") |
|
|
tokenizer = AutoTokenizer.from_pretrained("louijiec/veriforge-deepseek-coder-1.3b-instruct") |
|
|
|
|
|
prompt = """### Task: Synthesize Verilog\nDesign a 2-to-1 multiplexer using behavioral modeling.\n### Verilog Code:""" |
|
|
inputs = tokenizer(prompt, return_tensors="pt").to(model.device) |
|
|
outputs = model.generate(**inputs, max_new_tokens=256) |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
## ✅ Evaluation |
|
|
This model has been sanity-checked using prompt-based outputs that are expected to include: |
|
|
- Valid Verilog keywords (`module`, `input`, `output`, `assign`, `endmodule`) |
|
|
- Structured code starting with `module` |
|
|
- Coherent outputs for standard digital design prompts (e.g., multiplexers, adders, encoders) |
|
|
|
|
|
For functional verification, use [Icarus Verilog](http://iverilog.icarus.com/) or [Verilator](https://www.veripool.org/verilator/) to simulate output. |
|
|
|
|
|
## 📎 Citations |
|
|
- Dettmers et al. (2023). [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314) |
|
|
- DeepSeek. [deepseek-ai/deepseek-coder-1.3b-instruct](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct) |
|
|
- Bnadimi. [PyraNet-Verilog dataset](https://huggingface.co/datasets/bnadimi/PyraNet-Verilog) |
|
|
- Hugging Face 🤗 Transformers. https://github.com/huggingface/transformers |
|
|
|
