w266_model3_BERT_CNN

This model is a fine-tuned version of bert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7935
• Accuracy: {'accuracy': 0.67}
• F1: {'f1': 0.6539863523155215}
• Precision: {'precision': 0.6655888523241464}
• Recall: {'recall': 0.67}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5.0

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
0.7881	1.0	1923	0.8177	{'accuracy': 0.638}	{'f1': 0.6219209356584174}	{'precision': 0.6325213408748697}	{'recall': 0.638}
0.649	2.0	3846	0.8257	{'accuracy': 0.669}	{'f1': 0.6701535233107099}	{'precision': 0.672307962349643}	{'recall': 0.669}
0.4771	3.0	5769	0.8922	{'accuracy': 0.676}	{'f1': 0.6778795418743319}	{'precision': 0.6805694646691987}	{'recall': 0.676}
0.3403	4.0	7692	1.4285	{'accuracy': 0.669}	{'f1': 0.666176554548987}	{'precision': 0.6653390405441227}	{'recall': 0.669}
0.2088	5.0	9615	1.7417	{'accuracy': 0.67}	{'f1': 0.6716636513157895}	{'precision': 0.6752339933799478}	{'recall': 0.67}

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.3
• Tokenizers 0.13.3