finetuned_mistral_on_ads

This model is a fine-tuned version of mistralai/Mistral-7B-Instruct-v0.3 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5249

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
3.7417	0.0444	2	3.6685
3.6314	0.0889	4	3.2304
3.0686	0.1333	6	2.8771
2.5057	0.1778	8	2.7170
2.5453	0.2222	10	2.5886
2.5759	0.2667	12	2.4625
2.4252	0.3111	14	2.3477
2.4227	0.3556	16	2.2455
1.987	0.4	18	2.1370
2.0229	0.4444	20	2.0484
2.0755	0.4889	22	1.9746
1.9004	0.5333	24	1.9032
1.9381	0.5778	26	1.8405
1.7879	0.6222	28	1.7911
1.7544	0.6667	30	1.7584
1.7485	0.7111	32	1.7290
1.6927	0.7556	34	1.7030
1.8931	0.8	36	1.6825
1.5624	0.8444	38	1.6656
1.7061	0.8889	40	1.6528
1.7288	0.9333	42	1.6426
1.7839	0.9778	44	1.6347
1.5954	1.0222	46	1.6270
1.4288	1.0667	48	1.6177
1.5201	1.1111	50	1.6094
1.5281	1.1556	52	1.6037
1.4132	1.2	54	1.5998
1.4271	1.2444	56	1.5976
1.4778	1.2889	58	1.5952
1.5138	1.3333	60	1.5921
1.4539	1.3778	62	1.5875
1.4293	1.4222	64	1.5823
1.3673	1.4667	66	1.5773
1.5272	1.5111	68	1.5734
1.506	1.5556	70	1.5701
1.2929	1.6	72	1.5669
1.387	1.6444	74	1.5637
1.3375	1.6889	76	1.5609
1.4666	1.7333	78	1.5586
1.2295	1.7778	80	1.5553
1.5195	1.8222	82	1.5521
1.5116	1.8667	84	1.5488
1.2947	1.9111	86	1.5449
1.4651	1.9556	88	1.5399
1.5171	2.0	90	1.5351
1.1823	2.0444	92	1.5312
1.3729	2.0889	94	1.5286
1.2607	2.1333	96	1.5256
1.2048	2.1778	98	1.5237
1.2862	2.2222	100	1.5229
1.2584	2.2667	102	1.5224
1.2285	2.3111	104	1.5223
1.2794	2.3556	106	1.5222
1.2196	2.4	108	1.5227
1.2526	2.4444	110	1.5232
1.2876	2.4889	112	1.5237
1.1812	2.5333	114	1.5247
1.3622	2.5778	116	1.5255
1.229	2.6222	118	1.5261
1.2796	2.6667	120	1.5262
1.2059	2.7111	122	1.5258
1.3327	2.7556	124	1.5257
1.254	2.8	126	1.5257
1.2183	2.8444	128	1.5256
1.1979	2.8889	130	1.5254
1.2558	2.9333	132	1.5251
1.1405	2.9778	134	1.5249

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1