![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
]
},
{
"cell_type": "markdown",
"source": [
"##**Final Project**\n",
"This is my Final Project of Learner's Space on Natural Language Processing, where I will be using some datasets and model for machine translation and then fine-tune the same model. I engaged in discussion with some of my friends who had taken this Learner's Space project as well, so that I can help them whenever they have a doubt and they help me when I have a doubt or run into an error."
],
"metadata": {
"id": "nxLRozg71Twh"
}
},
{
"cell_type": "markdown",
"source": [
"First we give T4 GPU access and check whether the GPU is working or not using the code below"
],
"metadata": {
"id": "engemglKJlAc"
}
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "99bjwu4z1I8E",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "223a5c2f-1047-4b23-c6d7-84321751e1aa"
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Mon Aug 14 15:53:22 2023 \n",
"+-----------------------------------------------------------------------------+\n",
"| NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 |\n",
"|-------------------------------+----------------------+----------------------+\n",
"| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |\n",
"| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |\n",
"| | | MIG M. |\n",
"|===============================+======================+======================|\n",
"| 0 Tesla T4 Off | 00000000:00:04.0 Off | 0 |\n",
"| N/A 60C P8 10W / 70W | 0MiB / 15360MiB | 0% Default |\n",
"| | | N/A |\n",
"+-------------------------------+----------------------+----------------------+\n",
" \n",
"+-----------------------------------------------------------------------------+\n",
"| Processes: |\n",
"| GPU GI CI PID Type Process name GPU Memory |\n",
"| ID ID Usage |\n",
"|=============================================================================|\n",
"| No running processes found |\n",
"+-----------------------------------------------------------------------------+\n"
]
}
],
"source": [
"!nvidia-smi"
]
},
{
"cell_type": "markdown",
"source": [
"After giving GPU access, we download necessary libraries like :\n",
"\n",
"\n",
"1. `transformers` for importing the pipelines and for tokenizer\n",
"2. `accelerate` for faster loading.\n",
"3. `sentencepiece` while applying the pipeline.\n",
"4. `gradio` for interactive app experience\n",
"5. `datasets` to import datasets\n",
"6. `evaluate` for loading the metric and evaluating\n",
"7. `sacrebleu` for the metric\n"
],
"metadata": {
"id": "BAEBawzyJw2S"
}
},
{
"cell_type": "code",
"source": [
"! pip install -q transformers accelerate sentencepiece gradio datasets evaluate sacrebleu"
],
"metadata": {
"id": "10En4sOM-7Zh",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "6f0312ff-9982-44a8-904c-fb46be8e788c"
},
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.4/7.4 MB\u001b[0m \u001b[31m17.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m28.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m34.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m20.0/20.0 MB\u001b[0m \u001b[31m40.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m519.3/519.3 kB\u001b[0m \u001b[31m51.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m81.4/81.4 kB\u001b[0m \u001b[31m11.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m118.9/118.9 kB\u001b[0m \u001b[31m13.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m27.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m82.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m66.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m65.8/65.8 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m297.4/297.4 kB\u001b[0m \u001b[31m31.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.4/75.4 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m3.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m140.3/140.3 kB\u001b[0m \u001b[31m16.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m5.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.5/59.5 kB\u001b[0m \u001b[31m5.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m15.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m14.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m21.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m16.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m5.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m5.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m4.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m4.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m10.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m10.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m8.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.5/74.5 kB\u001b[0m \u001b[31m8.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## **Dataset and Preprocessing**\n",
"Now we import the \"enimai/MuST-C-it\" dataset which has both english and italian sentences in it."
],
"metadata": {
"id": "4tYxEjXALVCa"
}
},
{
"cell_type": "code",
"source": [
"from datasets import load_dataset\n",
"raw_dataset = load_dataset(\"enimai/MuST-C-it\")\n",
"raw_dataset"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 548,
"referenced_widgets": [
"2cbcf42ba9d14aebab15f8c600e46192",
"03ce1312d25d4a1081d34aa6ff959a13",
"709bf53e72f74e01ad60449721f29c30",
"7056ef827d6b4cc7938019b5bb3fec79",
"62ab45ed1992423882c170d1cc33d5f6",
"a41fa88d16e04a0697840d0405c83f12",
"26a1ad113c4042ac8b7b61430dad40f6",
"b31844929861446098d17b251700e695",
"0d07032e4fcd4146a5af747c2f233c7b",
"0e7a80dc819d4425b6349582ddcc2077",
"fe12ed140e8c4cd097e7e8f9fccd0925",
"78290aaedc724d77b581c670d7ae7dd0",
"aec7b24386d24bc6a27ad8186dc410fb",
"952734a630fc4e44af4e26cb5cc76031",
"142275b02f6d4e9896590ccb5bff4319",
"15754eac03ef4609aea1de79f4a528de",
"a06446e891c141629f2e79e2044373e2",
"380267ce71a7431a97b109932816f584",
"f59ba46c7af24e9c8fd635bc038cb42f",
"d4ce3d08512344b8947139665ad1f29b",
"e22f9287e63442e7bde34ada0628501e",
"142a1b2bc1bf425aa2624e3203dc71fa",
"be3130c26bf34a5eb7eb717941fcb121",
"387b13447c3a42d0bcc7bed32e85d4f0",
"ea87e25564ab4dbe94bb3666e1f15177",
"9aea088637d346f49b10e65852c3cc93",
"abf9285d8e114e4b9374126fd7adbe1d",
"5327f0e5ec0d40eaac08bf410bfe80b5",
"2719c7793f69491ebb996126bfbdc2da",
"63c0f17bf16b462cb003d037deac8ef7",
"64f54da115754f94be74647f10742642",
"0e5a9a9ff65e455190b70ca203f9d34a",
"e24f5953b3764cde93feee8685e0a290",
"7bf365f0e2ee427e8e7c5873545ce7cf",
"fecf4648693544609f1650cc8458124f",
"0b07ecdd3bf648ffa2b743294e04b56f",
"dd7ac6aea57147a48c1139602612f82c",
"bd7a3ee997064df8ab6a181117ba509e",
"c4a33ade20904c27a8e91c46e632e405",
"aeae8bcb97144c438dbe3496d0933387",
"4ee764bd3a1a46b6b9f5ffdd0ab6169e",
"e33e80db337b4788956d3a63458e0f62",
"492b652072b24ed3a3e47b00d029ff90",
"7cf4e238f68346e786b34b9eb6b8f32a",
"eaca7c5f54fe47ff994bf4b6bd0d3a0e",
"2be26e00ae054dd1b0a959ad6d0930ee",
"b6d4a601e4dd4da9906ebd3f623cf533",
"1f04aa5fc5044919a335be3a707da739",
"9161660f2f18496991db064fcdf2ce2d",
"01eabc15f13b4c33891225944a438417",
"5c9750b1fd3649888da32339518c06df",
"7b5497c279bb4467913504f47c09f3ca",
"35ff054f9206432bb3ead5402f310daf",
"e012bd85e8ab4f49b328012229f2ff1c",
"dee6e9302fc5420bb8a877fa3d3ec7d2",
"6c031a8cee414cc384e68704d0093a15",
"e37a1f3281d84764b2f08889bd8f20a2",
"00821e9bf0aa473997df04788677c308",
"a0a4687c94b4470d8ee291402531deae",
"bc9f91bc43eb4e2cab608fe1897734bb",
"8182f8bd4ceb4445b8cc3a9ad00247b3",
"66de098a18504f3a97ea74d63c40841d",
"fd61239d0f9d4ba5b9b6a07305fc3276",
"5b168e7886f34ae0864a02e92d7ac5bb",
"ff1ddd4286fc48f08a59f48d6d0832a2",
"e42db435c87646fcbd16bac8628c3ecf",
"7e4c03d6bafc415ea450beb3c0fac979",
"b1d856172c2044c384f692c882df75c8",
"4aa4213cf8ea4222aeffcadbeedeaf3a",
"376a33601c5b47dea01a02715fec5f10",
"5dbb20801ad141029d5cbd494650aa44",
"a67eac6743af4d3d9092bb8fad13e1ce",
"459d4792956245c58b9c77444a65d535",
"9aa22a97fa0a4dc0b1962e698e46fded",
"b46715cc264f4c609f147be885e95686",
"35146db51f314f899a66bb31833ed0d7",
"6610a025ea5e456c9f8af3c5ae4bd640",
"387b65d0538c4c3aba71f934de1f1754",
"ca976d6eeff24237aae356a406ac74e3",
"93337e2a87b745eeb052afd176a989ed",
"be9423439b404babb1bf721a0a35ea06",
"f5d005f663e6479e9d702a5b0a35f391",
"75cfaac01fd04c1cb5c76f795f7bec50",
"d6b4a8bcd543476889192d5d3e20021d",
"357b2f4be7c143e4a33e247009935d08",
"f9b29e33fc6a447ba46ed5a3abc38b12",
"c40094662d194db6aaf264017801c858",
"317d47e0f6b94efe88f91307b0af2e38",
"bb7e9fe0c9ab444cb983fa38092965eb",
"065baef4f92442a29c5f6b7a6fab3fc7",
"a5aea64cd5c942f4a356b53908d65744",
"359acbf5a980409d93ad2b425ec1dc92",
"a18206f8c5894a2ab3f558b7e776e2c2",
"a5a8a88aa3464d28a928da8c4daf8eff",
"b3cdb396fd544da886802434e188564d",
"363a50309c3e48d5b96669ca5156f6b1",
"d5dba590c6b243bb86c79cce4ba5842d",
"5cc6ecf7bc3f4ab1b4477a8f0f816e26",
"be46547d05b147919d7f7fe89e881cfa"
]
},
"id": "j8wz4fZ_--eI",
"outputId": "9c1e588f-0b62-4966-b091-4a75aa19ba46"
},
"execution_count": 3,
"outputs": [
{
"output_type": "display_data",
"data": {
"text/plain": [
"Downloading readme: 0%| | 0.00/25.0 [00:00 | Step | \n", "Training Loss | \n", "
|---|---|
| 500 | \n", "1.294000 | \n", "
| 1000 | \n", "1.267000 | \n", "
| 1500 | \n", "1.241400 | \n", "
| 2000 | \n", "1.260000 | \n", "
| 2500 | \n", "1.252200 | \n", "
| 3000 | \n", "1.250200 | \n", "
| 3500 | \n", "1.241500 | \n", "
| 4000 | \n", "1.249200 | \n", "
| 4500 | \n", "1.236400 | \n", "
| 5000 | \n", "1.227700 | \n", "
| 5500 | \n", "1.227300 | \n", "
| 6000 | \n", "1.235500 | \n", "
| 6500 | \n", "1.180700 | \n", "
| 7000 | \n", "1.120300 | \n", "
| 7500 | \n", "1.118800 | \n", "
| 8000 | \n", "1.119700 | \n", "
| 8500 | \n", "1.125000 | \n", "
| 9000 | \n", "1.122100 | \n", "
| 9500 | \n", "1.127500 | \n", "
| 10000 | \n", "1.115300 | \n", "
| 10500 | \n", "1.130200 | \n", "
| 11000 | \n", "1.131400 | \n", "
| 11500 | \n", "1.127600 | \n", "
| 12000 | \n", "1.123700 | \n", "
| 12500 | \n", "1.122900 | \n", "
| 13000 | \n", "1.085900 | \n", "
| 13500 | \n", "1.049900 | \n", "
| 14000 | \n", "1.060700 | \n", "
| 14500 | \n", "1.052200 | \n", "
| 15000 | \n", "1.060300 | \n", "
| 15500 | \n", "1.054100 | \n", "
| 16000 | \n", "1.051900 | \n", "
| 16500 | \n", "1.055800 | \n", "
| 17000 | \n", "1.057300 | \n", "
| 17500 | \n", "1.066100 | \n", "
| 18000 | \n", "1.058500 | \n", "
| 18500 | \n", "1.060300 | \n", "
| 19000 | \n", "1.048900 | \n", "
" ] }, "metadata": {} }, { "output_type": "stream", "name": "stderr", "text": [ "Adding files tracked by Git LFS: ['source.spm', 'target.spm']. This may take a bit of time if the files are large.\n", "WARNING:huggingface_hub.repository:Adding files tracked by Git LFS: ['source.spm', 'target.spm']. This may take a bit of time if the files are large.\n", "Several commits (2) will be pushed upstream.\n", "WARNING:huggingface_hub.repository:Several commits (2) will be pushed upstream.\n", "Several commits (3) will be pushed upstream.\n", "WARNING:huggingface_hub.repository:Several commits (3) will be pushed upstream.\n" ] }, { "output_type": "execute_result", "data": { "text/plain": [ "TrainOutput(global_step=19020, training_loss=1.1422517776489258, metrics={'train_runtime': 4588.4073, 'train_samples_per_second': 132.641, 'train_steps_per_second': 4.145, 'total_flos': 1.3068977520771072e+16, 'train_loss': 1.1422517776489258, 'epoch': 3.0})" ] }, "metadata": {}, "execution_count": 19 } ] }, { "cell_type": "markdown", "source": [ "##**Saving in Drive**\n", "Here after training the model, we save it into google drive which can be accessed later if we need to build gradio interface" ], "metadata": { "id": "UhZ0aT0OuFQN" } }, { "cell_type": "code", "source": [ "from google.colab import drive\n", "drive.mount('/content/drive')" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "MVIbmwFGkEWC", "outputId": "4b12d218-7633-4848-9603-f1c93acdc9e7" }, "execution_count": 22, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n" ] } ] }, { "cell_type": "code", "source": [ "model.save_pretrained('/content/drive/MyDrive/LS_NLP_Parth')" ], "metadata": { "id": "R0a9GwdPkRTa" }, "execution_count": 23, "outputs": [] } ] }