train_boolq_123_1762618900

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the boolq dataset. It achieves the following results on the evaluation set:

• Loss: 0.1626
• Num Input Tokens Seen: 42678144

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.2178	1.0	2121	0.2072	2131904
0.1702	2.0	4242	0.1700	4264768
0.1224	3.0	6363	0.1626	6404896
0.131	4.0	8484	0.1692	8537088
0.1128	5.0	10605	0.1783	10677088
0.2673	6.0	12726	0.1823	12814080
0.0028	7.0	14847	0.2114	14950432
0.0025	8.0	16968	0.2463	17082336
0.321	9.0	19089	0.2489	19211360
0.0014	10.0	21210	0.2931	21342336
0.1209	11.0	23331	0.3105	23472352
0.001	12.0	25452	0.3424	25602144
0.0009	13.0	27573	0.3634	27739072
0.0005	14.0	29694	0.4114	29880544
0.0005	15.0	31815	0.4213	32013760
0.0004	16.0	33936	0.4479	34138272
0.0008	17.0	36057	0.4568	36269152
0.0005	18.0	38178	0.4607	38408800
0.0001	19.0	40299	0.4635	40541312
0.031	20.0	42420	0.4603	42678144

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.8.0+cu128
• Datasets 3.6.0
• Tokenizers 0.21.1