kavg/LiLT-RE-DE-SIN

checkpoints

This model is a fine-tuned version of kavg/LiLT-RE-DE on the xfun dataset. It achieves the following results on the evaluation set:

• Precision: 0.2952
• Recall: 0.4167
• F1: 0.3455
• Loss: 0.3186

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• training_steps: 10000

Training results

Training Loss	Epoch	Step	Precision	Recall	F1	Validation Loss
0.1035	41.67	500	0.2905	0.1540	0.2013	0.2291
0.0691	83.33	1000	0.2952	0.4167	0.3455	0.3186
0.0442	125.0	1500	0.2970	0.5909	0.3953	0.2765
0.024	166.67	2000	0.3227	0.5884	0.4168	0.4144
0.0216	208.33	2500	0.3234	0.6035	0.4211	0.4036
0.0096	250.0	3000	0.3534	0.6364	0.4545	0.5716
0.0079	291.67	3500	0.3456	0.5934	0.4368	0.6643
0.0045	333.33	4000	0.3427	0.6187	0.4410	0.6955
0.0017	375.0	4500	0.3587	0.6187	0.4541	0.8144
0.0147	416.67	5000	0.3407	0.6212	0.4401	0.8101
0.0027	458.33	5500	0.3491	0.6162	0.4457	0.8809
0.0079	500.0	6000	0.3183	0.6061	0.4174	0.8863
0.0028	541.67	6500	0.3506	0.5985	0.4422	0.9944
0.0075	583.33	7000	0.3476	0.5960	0.4391	0.9920
0.0002	625.0	7500	0.3448	0.6061	0.4396	0.9752
0.0025	666.67	8000	0.3456	0.6162	0.4428	0.9866
0.0037	708.33	8500	0.3465	0.6187	0.4442	1.0153
0.0041	750.0	9000	0.3442	0.6136	0.4410	1.1227
0.0023	791.67	9500	0.3450	0.6237	0.4442	1.0995
0.0007	833.33	10000	0.3408	0.6162	0.4388	1.1097

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.1