git-base-naruto

This model is a fine-tuned version of microsoft/git-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.0600
• Wer Score: 2.1639

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
7.2846	3.7037	50	4.4712	22.9180
2.2861	7.4074	100	0.4239	10.6230
0.1211	11.1111	150	0.0471	0.4754
0.0161	14.8148	200	0.0453	0.4098
0.0114	18.5185	250	0.0474	0.4426
0.0093	22.2222	300	0.0501	1.4754
0.0084	25.9259	350	0.0503	0.7049
0.0068	29.6296	400	0.0534	0.4590
0.0058	33.3333	450	0.0562	0.4426
0.0048	37.0370	500	0.0572	0.4590
0.0035	40.7407	550	0.0597	0.7869
0.0025	44.4444	600	0.0602	2.0164
0.0017	48.1481	650	0.0600	2.1639

Framework versions

• Transformers 4.40.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1