sentiment-lora-r8-0

This model is a fine-tuned version of indolem/indobert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3235
• Accuracy: 0.8596
• Precision: 0.8307
• Recall: 0.8307
• F1: 0.8307

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 30
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
0.5593	1.0	122	0.4990	0.7243	0.6593	0.6374	0.6446
0.4939	2.0	244	0.4696	0.7619	0.7265	0.7590	0.7346
0.4469	3.0	366	0.4036	0.8070	0.7670	0.7760	0.7711
0.3781	4.0	488	0.3748	0.8195	0.7827	0.7798	0.7812
0.3532	5.0	610	0.4110	0.8045	0.7687	0.8017	0.7792
0.3273	6.0	732	0.3612	0.8321	0.7960	0.8137	0.8036
0.3098	7.0	854	0.3552	0.8371	0.8017	0.8197	0.8094
0.2953	8.0	976	0.3438	0.8546	0.8239	0.8272	0.8255
0.28	9.0	1098	0.3590	0.8446	0.8102	0.8301	0.8186
0.2701	10.0	1220	0.3354	0.8571	0.8299	0.8214	0.8255
0.2694	11.0	1342	0.3366	0.8571	0.8281	0.8264	0.8272
0.2657	12.0	1464	0.3378	0.8596	0.8287	0.8382	0.8332
0.2603	13.0	1586	0.3295	0.8647	0.8377	0.8342	0.8359
0.2564	14.0	1708	0.3318	0.8596	0.8299	0.8332	0.8315
0.2583	15.0	1830	0.3291	0.8596	0.8299	0.8332	0.8315
0.2438	16.0	1952	0.3323	0.8571	0.8260	0.8339	0.8298
0.2465	17.0	2074	0.3225	0.8546	0.8273	0.8171	0.8219
0.2494	18.0	2196	0.3292	0.8571	0.8266	0.8314	0.8289
0.2382	19.0	2318	0.3246	0.8571	0.8273	0.8289	0.8281
0.236	20.0	2440	0.3235	0.8596	0.8307	0.8307	0.8307

Framework versions

• Transformers 4.39.3
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.15.2