Hugging Face's logo

simonycl
/
roberta-base-sst-2-16-42-smoothed

Text Classification
Transformers
PyTorch
bert
Generated from Trainer
Inference Endpoints
Model card Files Community
roberta-base-sst-2-16-42-smoothed / README.md
simonycl's picture
simonycl
update model card README.md
59c6c35
preview code
|
raw
history blame contribute delete
6.06 kB
metadata
license: apache-2.0
base_model: bert-base-uncased
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: bert-base-uncased-sst-2-16-42-smoothed
    results: []

bert-base-uncased-sst-2-16-42-smoothed

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6339
  • Accuracy: 0.7812

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 50
  • num_epochs: 75
  • label_smoothing_factor: 0.45

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 1.0 4 0.7048 0.5
No log 2.0 8 0.7021 0.5
0.7078 3.0 12 0.6982 0.5
0.7078 4.0 16 0.6936 0.4922
0.6935 5.0 20 0.6898 0.5625
0.6935 6.0 24 0.6878 0.625
0.6935 7.0 28 0.6856 0.6484
0.682 8.0 32 0.6818 0.6797
0.682 9.0 36 0.6759 0.7109
0.6567 10.0 40 0.6692 0.6562
0.6567 11.0 44 0.6662 0.6562
0.6567 12.0 48 0.6669 0.6484
0.6177 13.0 52 0.6612 0.6719
0.6177 14.0 56 0.6589 0.6641
0.5787 15.0 60 0.6565 0.6797
0.5787 16.0 64 0.6519 0.7109
0.5787 17.0 68 0.6506 0.7266
0.5487 18.0 72 0.6496 0.75
0.5487 19.0 76 0.6489 0.7344
0.5398 20.0 80 0.6492 0.75
0.5398 21.0 84 0.6467 0.7578
0.5398 22.0 88 0.6439 0.75
0.5384 23.0 92 0.6419 0.75
0.5384 24.0 96 0.6442 0.75
0.5389 25.0 100 0.6442 0.7422
0.5389 26.0 104 0.6391 0.75
0.5389 27.0 108 0.6428 0.7734
0.5373 28.0 112 0.6449 0.7656
0.5373 29.0 116 0.6412 0.7812
0.5379 30.0 120 0.6359 0.7578
0.5379 31.0 124 0.6348 0.7734
0.5379 32.0 128 0.6343 0.7891
0.5366 33.0 132 0.6340 0.7969
0.5366 34.0 136 0.6340 0.7891
0.5366 35.0 140 0.6339 0.7734
0.5366 36.0 144 0.6337 0.7734
0.5366 37.0 148 0.6335 0.7734
0.5363 38.0 152 0.6334 0.7969
0.5363 39.0 156 0.6347 0.7734
0.5367 40.0 160 0.6355 0.7656
0.5367 41.0 164 0.6363 0.7578
0.5367 42.0 168 0.6374 0.7656
0.5359 43.0 172 0.6375 0.7656
0.5359 44.0 176 0.6357 0.7578
0.5358 45.0 180 0.6351 0.7656
0.5358 46.0 184 0.6339 0.7734
0.5358 47.0 188 0.6334 0.7812
0.5362 48.0 192 0.6330 0.7891
0.5362 49.0 196 0.6327 0.7891
0.5364 50.0 200 0.6329 0.7891
0.5364 51.0 204 0.6341 0.7734
0.5364 52.0 208 0.6337 0.7734
0.5365 53.0 212 0.6326 0.7891
0.5365 54.0 216 0.6325 0.7891
0.5361 55.0 220 0.6326 0.7969
0.5361 56.0 224 0.6328 0.7891
0.5361 57.0 228 0.6328 0.7891
0.5361 58.0 232 0.6330 0.7891
0.5361 59.0 236 0.6335 0.7812
0.5363 60.0 240 0.6340 0.7812
0.5363 61.0 244 0.6343 0.7812
0.5363 62.0 248 0.6346 0.7734
0.536 63.0 252 0.6348 0.7734
0.536 64.0 256 0.6349 0.7734
0.5362 65.0 260 0.6353 0.7656
0.5362 66.0 264 0.6358 0.7656
0.5362 67.0 268 0.6361 0.7578
0.536 68.0 272 0.6355 0.7656
0.536 69.0 276 0.6349 0.7734
0.5358 70.0 280 0.6344 0.7734
0.5358 71.0 284 0.6342 0.7812
0.5358 72.0 288 0.6340 0.7812
0.5357 73.0 292 0.6340 0.7812
0.5357 74.0 296 0.6339 0.7812
0.5358 75.0 300 0.6339 0.7812

Framework versions

  • Transformers 4.32.0.dev0
  • Pytorch 2.0.1+cu118
  • Datasets 2.4.0
  • Tokenizers 0.13.3