Labira/LabiraPJOK_1_1000x

This model is a fine-tuned version of indolem/indobert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 0.2616
• Validation Loss: 4.8341
• Epoch: 24

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': False, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 3000, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
5.9029	5.5338	0
5.5052	5.1945	1
5.0405	4.9078	2
4.5254	4.5685	3
4.1376	4.3884	4
3.8263	4.1669	5
3.5461	3.8994	6
3.1371	3.7182	7
2.7303	3.6502	8
2.4936	3.7608	9
2.2117	3.9550	10
1.9386	4.0934	11
1.7866	4.1102	12
1.4512	4.2896	13
1.1873	4.5401	14
0.9892	4.8950	15
0.8121	4.9718	16
0.7331	4.6763	17
0.6712	4.5185	18
0.5773	4.8674	19
0.4841	4.9185	20
0.3451	4.8513	21
0.2938	4.8199	22
0.3125	4.9438	23
0.2616	4.8341	24

Framework versions

• Transformers 4.45.2
• TensorFlow 2.17.0
• Datasets 2.20.0
• Tokenizers 0.20.1