kavg/LiLT-RE-ES-SIN

checkpoints

This model is a fine-tuned version of kavg/LiLT-RE-ES on the xfun dataset. It achieves the following results on the evaluation set:

• Precision: 0.2886
• Recall: 0.3586
• F1: 0.3198
• Loss: 0.2312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Precision	Recall	F1	Validation Loss
0.1103	41.67	500	0.4808	0.0631	0.1116	0.2442
0.0871	83.33	1000	0.2886	0.3586	0.3198	0.2312
0.0905	125.0	1500	0.2904	0.5177	0.3721	0.2402
0.0521	166.67	2000	0.3065	0.5581	0.3957	0.2793
0.0508	208.33	2500	0.3080	0.6136	0.4101	0.4084
0.0509	250.0	3000	0.3250	0.5934	0.4200	0.4008
0.0406	291.67	3500	0.3290	0.5808	0.4201	0.4593
0.0333	333.33	4000	0.3488	0.5884	0.4380	0.4806
0.0358	375.0	4500	0.3456	0.5682	0.4298	0.6472
0.0289	416.67	5000	0.3657	0.5808	0.4488	0.6532
0.0255	458.33	5500	0.3601	0.5783	0.4438	0.7617
0.0183	500.0	6000	0.3736	0.5859	0.4562	0.7025
0.0213	541.67	6500	0.3606	0.5783	0.4442	0.8442
0.0296	583.33	7000	0.3621	0.5505	0.4369	0.7416
0.0418	625.0	7500	0.3659	0.5682	0.4451	0.7372
0.0225	666.67	8000	0.3729	0.5556	0.4462	0.8660
0.0225	708.33	8500	0.3723	0.5707	0.4506	0.8646
0.0128	750.0	9000	0.375	0.5606	0.4494	0.7905
0.0182	791.67	9500	0.3758	0.5657	0.4516	0.8551
0.0061	833.33	10000	0.3788	0.5606	0.4521	0.8355

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1