Hugging Face's logo

gokulsrinivasagan
/
bert_tiny_olda_book_5_v1

Transformers
TensorBoard
Safetensors
distilbert
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Metrics Community
bert_tiny_olda_book_5_v1 / README.md
gokulsrinivasagan's picture
gokulsrinivasagan
End of training
b94b157 verified
preview code
|
raw
history blame contribute delete
4.07 kB
metadata
library_name: transformers
tags:
  - generated_from_trainer
datasets:
  - gokulsrinivasagan/processed_book_corpus-ld-5
metrics:
  - accuracy
model-index:
  - name: bert_tiny_olda_book_5_v1
    results:
      - task:
          name: Masked Language Modeling
          type: fill-mask
        dataset:
          name: gokulsrinivasagan/processed_book_corpus-ld-5
          type: gokulsrinivasagan/processed_book_corpus-ld-5
        metrics:
          - name: Accuracy
            type: accuracy
            value: 0.00026116278421770196

bert_tiny_olda_book_5_v1

This model is a fine-tuned version of on the gokulsrinivasagan/processed_book_corpus-ld-5 dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4052
  • Accuracy: 0.0003

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 160
  • eval_batch_size: 160
  • seed: 10
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 10000
  • num_epochs: 25

Training results

Training Loss Epoch Step Validation Loss Accuracy
1.4427 0.7025 10000 1.4406 0.0001
1.4355 1.4051 20000 1.4316 0.0000
1.433 2.1076 30000 1.4302 0.0000
1.4306 2.8102 40000 1.4287 0.0000
1.431 3.5127 50000 1.4299 0.0000
1.4295 4.2153 60000 1.4261 0.0000
1.4266 4.9178 70000 1.4238 0.0000
1.4205 5.6203 80000 1.4168 0.0000
1.4191 6.3229 90000 1.4146 0.0001
1.419 7.0254 100000 1.4147 0.0001
1.4181 7.7280 110000 1.4134 0.0001
1.4163 8.4305 120000 1.4129 0.0001
1.4164 9.1331 130000 1.4111 0.0002
1.4147 9.8356 140000 1.4100 0.0002
1.414 10.5381 150000 1.4106 0.0002
1.4141 11.2407 160000 1.4098 0.0002
1.413 11.9432 170000 1.4093 0.0003
1.4124 12.6458 180000 1.4087 0.0003
1.4114 13.3483 190000 1.4078 0.0002
1.4115 14.0509 200000 1.4078 0.0002
1.4104 14.7534 210000 1.4080 0.0003
1.411 15.4560 220000 1.4075 0.0003
1.411 16.1585 230000 1.4069 0.0003
1.4104 16.8610 240000 1.4066 0.0003
1.4102 17.5636 250000 1.4068 0.0003
1.4094 18.2661 260000 1.4064 0.0003
1.4096 18.9687 270000 1.4059 0.0003
1.4098 19.6712 280000 1.4059 0.0003
1.409 20.3738 290000 1.4058 0.0002
1.4081 21.0763 300000 1.4057 0.0003
1.4082 21.7788 310000 1.4052 0.0003
1.4084 22.4814 320000 1.4054 0.0003
1.4088 23.1839 330000 1.4052 0.0002
1.4088 23.8865 340000 1.4050 0.0003
1.4084 24.5890 350000 1.4052 0.0003

Framework versions

  • Transformers 4.46.1
  • Pytorch 2.2.0+cu121
  • Datasets 3.1.0
  • Tokenizers 0.20.1