bert_seq_training_model_multiple

This model is a fine-tuned version of bert-base-uncased on Brecon/Train_Test. It achieves the following results on the evaluation set:

• Accuracy: 0.3846
• Precision: 0.384
• Recall: 0.384
• f1_score: 0.372

Model description

This model was created as part of a university project with the goal of developing a transformer model for multi-sentence claim validation. The model was devloped on bert-base-uncased transform because of it's ability to capture sequences in a bidirectional manner. The model was first fine tuned on Brecon/Train_Test using f1 score as it's evaluation metric. Afterwards it was fine tuned on Brecon/Master_Train_Test using recall as an evaluation metric due to the imbalanced nature of the dataset.

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss	Recall
No log	1.0	23	1.0486	0.4835
No log	2.0	46	1.0304	0.3846

Framework versions

• Transformers 4.33.1
• Pytorch 2.0.1+cpu
• Datasets 2.14.5
• Tokenizers 0.11.0