camembert_ccnet_classification_tools_qlora-8bit_fr

This model is a fine-tuned version of camembert/camembert-base-ccnet on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.5066
• Accuracy: 0.85
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 24
• eval_batch_size: 192
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rate
2.0629	1.0	7	2.0715	0.125	0.0001
2.0638	2.0	14	2.0676	0.15	0.0001
2.0614	3.0	21	2.0661	0.1	0.0001
2.0676	4.0	28	2.0622	0.125	0.0001
2.0502	5.0	35	2.0530	0.15	0.0001
2.0409	6.0	42	2.0480	0.175	9e-05
2.0457	7.0	49	2.0431	0.1	0.0001
2.0401	8.0	56	2.0398	0.125	0.0001
2.0239	9.0	63	2.0283	0.275	0.0001
2.0257	10.0	70	2.0131	0.3	0.0001
1.999	11.0	77	1.9926	0.425	0.0001
2.0053	12.0	84	1.9866	0.4	8e-05
1.9893	13.0	91	1.9710	0.475	0.0001
1.9762	14.0	98	1.9572	0.475	0.0001
1.9726	15.0	105	1.9398	0.425	0.0001
1.9456	16.0	112	1.9152	0.5	0.0001
1.9425	17.0	119	1.8971	0.575	0.0001
1.9078	18.0	126	1.8788	0.625	7e-05
1.901	19.0	133	1.8608	0.65	0.0001
1.8865	20.0	140	1.8407	0.625	0.0001
1.8696	21.0	147	1.8234	0.675	0.0001
1.8618	22.0	154	1.7964	0.725	0.0001
1.8238	23.0	161	1.7778	0.725	0.0001
1.8164	24.0	168	1.7739	0.675	6e-05
1.823	25.0	175	1.7442	0.675	0.0001
1.7892	26.0	182	1.7265	0.7	0.0001
1.7865	27.0	189	1.7110	0.75	0.0001
1.7643	28.0	196	1.7114	0.725	0.0001
1.7613	29.0	203	1.6859	0.775	0.0001
1.7681	30.0	210	1.6686	0.775	5e-05
1.7251	31.0	217	1.6547	0.8	0.0000
1.7215	32.0	224	1.6431	0.8	0.0000
1.7304	33.0	231	1.6285	0.75	0.0000
1.7182	34.0	238	1.6164	0.8	0.0000
1.7099	35.0	245	1.6066	0.825	0.0000
1.6902	36.0	252	1.6060	0.8	4e-05
1.6839	37.0	259	1.5949	0.875	0.0000
1.6627	38.0	266	1.5822	0.875	0.0000
1.6558	39.0	273	1.5723	0.875	0.0000
1.6667	40.0	280	1.5639	0.85	0.0000
1.6663	41.0	287	1.5576	0.85	0.0000
1.642	42.0	294	1.5506	0.825	3e-05
1.6479	43.0	301	1.5471	0.825	0.0000
1.6425	44.0	308	1.5414	0.825	0.0000
1.6214	45.0	315	1.5351	0.85	0.0000
1.6331	46.0	322	1.5310	0.875	0.0000
1.6406	47.0	329	1.5281	0.875	0.0000
1.6332	48.0	336	1.5265	0.875	2e-05
1.6474	49.0	343	1.5218	0.875	0.0000
1.6122	50.0	350	1.5184	0.875	0.0000
1.6166	51.0	357	1.5162	0.875	0.0000
1.6245	52.0	364	1.5343	0.875	0.0000
1.6027	53.0	371	1.5272	0.85	0.0000
1.6305	54.0	378	1.5261	0.85	1e-05
1.6172	55.0	385	1.5173	0.85	0.0000
1.6109	56.0	392	1.5168	0.875	0.0000
1.6037	57.0	399	1.5154	0.875	5e-06
1.6031	58.0	406	1.4970	0.875	0.0000
1.617	59.0	413	1.4961	0.875	0.0000
1.5955	60.0	420	1.5066	0.85	0.0

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.1