opus_books_es_pt

This model is a fine-tuned version of t5-base on the opus_books dataset. It achieves the following results on the evaluation set:

• Loss: 2.0763
• Bleu: 1.2169
• Gen Len: 18.5038

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	133	2.5227	0.5795	18.5789
No log	2.0	266	2.3918	0.6703	18.5451
No log	3.0	399	2.3166	0.8471	18.5301
2.6664	4.0	532	2.2665	0.8914	18.4737
2.6664	5.0	665	2.2319	0.928	18.4549
2.6664	6.0	798	2.2025	1.0067	18.5113
2.6664	7.0	931	2.1784	1.0162	18.515
2.2503	8.0	1064	2.1580	1.1102	18.5113
2.2503	9.0	1197	2.1420	1.0638	18.515
2.2503	10.0	1330	2.1257	1.1149	18.5113
2.2503	11.0	1463	2.1142	1.1334	18.4474
2.1172	12.0	1596	2.1091	1.1308	18.4925
2.1172	13.0	1729	2.0980	1.1655	18.5075
2.1172	14.0	1862	2.0950	1.1464	18.4925
2.1172	15.0	1995	2.0890	1.1383	18.5038
2.0185	16.0	2128	2.0833	1.1671	18.5
2.0185	17.0	2261	2.0806	1.1555	18.5038
2.0185	18.0	2394	2.0777	1.15	18.5113
1.9882	19.0	2527	2.0770	1.2252	18.5113
1.9882	20.0	2660	2.0763	1.2169	18.5038

Framework versions

• Transformers 4.36.1
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.15.0