train_boolq_123_1762628463

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1509
• Num Input Tokens Seen: 42678144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2441	1.0	2121	0.2164	2131904
0.1636	2.0	4242	0.1829	4264768
0.1143	3.0	6363	0.1703	6404896
0.1735	4.0	8484	0.1635	8537088
0.1825	5.0	10605	0.1568	10677088
0.1657	6.0	12726	0.1543	12814080
0.0268	7.0	14847	0.1509	14950432
0.0805	8.0	16968	0.1509	17082336
0.3383	9.0	19089	0.1519	19211360
0.0432	10.0	21210	0.1517	21342336
0.14	11.0	23331	0.1514	23472352
0.0925	12.0	25452	0.1521	25602144
0.0795	13.0	27573	0.1525	27739072
0.0417	14.0	29694	0.1518	29880544
0.0246	15.0	31815	0.1537	32013760
0.0928	16.0	33936	0.1536	34138272
0.0424	17.0	36057	0.1527	36269152
0.1512	18.0	38178	0.1529	38408800
0.0632	19.0	40299	0.1531	40541312
0.0778	20.0	42420	0.1539	42678144

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1