Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
- zh
|
| 6 |
+
- id
|
| 7 |
+
---
|
| 8 |
+
# Just-Go-Parallel (Parallel Distributed)
|
| 9 |
+
The model repository for the "Parallel Distributed" setting of the following paper:
|
| 10 |
+
> **Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models**
|
| 11 |
+
>
|
| 12 |
+
> [Muhammad Reza Qorib](https://mrqorib.github.io/), [Junyi Li](https://lijunyi.tech/), and [Hwee Tou Ng](https://www.comp.nus.edu.sg/~nght/)
|
| 13 |
+
>
|
| 14 |
+
> The 63rd Annual Meeting of the Association for Computational Linguistics (to appear)
|
| 15 |
+
|
| 16 |
+
- **Paper:** [arXiv](https://arxiv.org/abs/2506.13044)
|
| 17 |
+
- **Codebase:** [https://github.com/nusnlp/Just-Go-Parallel/](https://github.com/nusnlp/just-Go-Parallel/)
|
| 18 |
+
|
| 19 |
+
We use the architecture and tokenizer of [TinyLlama v1.1](https://huggingface.co/TinyLlama/TinyLlama_v1.1).
|
| 20 |
+
Please use transformers>=4.35.
|
| 21 |
+
|
| 22 |
+
## Models
|
| 23 |
+
The main branch of the repository contains the best-performing model that was evaluated in the paper. Other checkpoints produced during training are also hosted in this repository under different branch names (also called "revisions" in HuggingFace), with each branch name referring to the number of training steps.
|
| 24 |
+
* No Parallel: [nusnlp/JGP-No-Parallel](https://huggingface.co/nusnlp/JGP-No-Parallel)
|
| 25 |
+
* Multilingual: [nusnlp/JGP-Multilingual](https://huggingface.co/nusnlp/JGP-Multilingual)
|
| 26 |
+
* Parallel Non-Adjacent: [nusnlp/JGP-Parallel-Non-Adjacent](https://huggingface.co/nusnlp/JGP-Parallel-Non-Adjacent)
|
| 27 |
+
* Parallel First: [nusnlp/JGP-Parallel-First](https://huggingface.co/nusnlp/JGP-Parallel-First)
|
| 28 |
+
* Parallel Distributed: [nusnlp/JGP-Parallel-Distributed](https://huggingface.co/nusnlp/JGP-Parallel-Distributed)
|
| 29 |
+
* Parallel Last (all): [nusnlp/JGP-Parallel-Last-all](https://huggingface.co/nusnlp/JGP-Parallel-Last-all)
|
| 30 |
+
* Parallel Last (uni):
|
| 31 |
+
* EN→ID: [nusnlp/JGP-Parallel-Last-EN-ID](https://huggingface.co/nusnlp/JGP-Parallel-Last-EN-ID)
|
| 32 |
+
* ID→EN: [nusnlp/JGP-Parallel-Last-ID-EN](https://huggingface.co/nusnlp/JGP-Parallel-Last-ID-EN)
|
| 33 |
+
* EN→ZH: [nusnlp/JGP-Parallel-Last-EN-ZH](https://huggingface.co/nusnlp/JGP-Parallel-Last-EN-ZH)
|
| 34 |
+
* ZH→EN: [nusnlp/JGP-Parallel-Last-ZH-EN](https://huggingface.co/nusnlp/JGP-Parallel-Last-ZH-EN)
|