Neuria_BERT_Contexto_0123

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-cased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0738
• Accuracy: 0.8188
• F1 Micro: 0.8936
• F1 Macro: 0.7547

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1 Micro	F1 Macro
0.409	0.9855	34	0.3094	0.0	0.0	0.0
0.2889	2.0	69	0.2621	0.1014	0.1489	0.0909
0.2458	2.9855	103	0.2169	0.3116	0.5289	0.2611
0.1853	4.0	138	0.1742	0.4855	0.6889	0.4577
0.1507	4.9855	172	0.1504	0.5580	0.7240	0.5140
0.1167	6.0	207	0.1311	0.6884	0.8301	0.6065
0.097	6.9855	241	0.1174	0.7609	0.8652	0.6406
0.0776	8.0	276	0.1064	0.7174	0.8526	0.6312
0.0686	8.9855	310	0.1076	0.7246	0.8435	0.6302
0.0585	10.0	345	0.0968	0.7754	0.8812	0.6572
0.0534	10.9855	379	0.0919	0.7681	0.8742	0.6487
0.0463	12.0	414	0.0906	0.7536	0.8589	0.6447
0.0433	12.9855	448	0.0894	0.7536	0.8562	0.6433
0.0385	14.0	483	0.0830	0.8043	0.8896	0.7455
0.0366	14.9855	517	0.0860	0.7754	0.8660	0.6475
0.0332	16.0	552	0.0852	0.7826	0.8758	0.6554
0.0318	16.9855	586	0.0817	0.7826	0.8785	0.6574
0.0289	18.0	621	0.0798	0.7826	0.8793	0.7464
0.0282	18.9855	655	0.0783	0.8116	0.8936	0.6604
0.026	20.0	690	0.0784	0.8043	0.8923	0.7535
0.0254	20.9855	724	0.0765	0.7899	0.8827	0.7472
0.0235	22.0	759	0.0764	0.8261	0.8997	0.7553
0.0232	22.9855	793	0.0771	0.7971	0.8889	0.7523
0.0217	24.0	828	0.0754	0.8116	0.8930	0.7563
0.0214	24.9855	862	0.0749	0.8116	0.8930	0.7509
0.0201	26.0	897	0.0759	0.8188	0.8936	0.7538
0.0199	26.9855	931	0.0761	0.8188	0.8963	0.7568
0.0187	28.0	966	0.0762	0.8333	0.8976	0.7567
0.0186	28.9855	1000	0.0764	0.8261	0.8970	0.7564
0.0177	30.0	1035	0.0727	0.8261	0.8997	0.7606
0.0177	30.9855	1069	0.0742	0.8333	0.9003	0.7577
0.0168	32.0	1104	0.0740	0.8333	0.9003	0.7579
0.0167	32.9855	1138	0.0743	0.8188	0.8963	0.7558
0.0159	34.0	1173	0.0738	0.8188	0.8963	0.7560
0.016	34.9855	1207	0.0736	0.8333	0.9030	0.7592
0.0153	36.0	1242	0.0742	0.8188	0.8909	0.7535
0.0155	36.9855	1276	0.0748	0.8406	0.9009	0.7614
0.0147	38.0	1311	0.0745	0.8333	0.9003	0.7577
0.0149	38.9855	1345	0.0738	0.8333	0.8976	0.7567
0.0142	40.0	1380	0.0738	0.8188	0.8936	0.7547

Framework versions

• Transformers 4.44.1
• Pytorch 2.4.1
• Datasets 2.19.1
• Tokenizers 0.19.1