Hugging Face's logo

anikethjr
/
PromoGen_K562_2080Ti_restart

Text Generation
Transformers
PyTorch
TensorBoard
prophetnet
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
PromoGen_K562_2080Ti_restart / README.md
anikethjr's picture
anikethjr
update model card README.md
796d40e
preview code
|
raw
history blame contribute delete
4.01 kB
metadata
tags:
  - generated_from_trainer
model-index:
  - name: PromoGen_K562_2080Ti_restart
    results: []

PromoGen_K562_2080Ti_restart

This model is a fine-tuned version of on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4624

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0005
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • gradient_accumulation_steps: 8
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 1000
  • num_epochs: 25
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
0.7676 0.49 2500 0.7383
0.7121 0.97 5000 0.6867
0.6914 1.46 7500 0.6705
0.6837 1.95 10000 0.6622
0.6778 2.44 12500 0.6558
0.6748 2.92 15000 0.6517
0.6676 3.41 17500 0.6433
0.6593 3.9 20000 0.6358
0.6584 4.38 22500 0.6320
0.6557 4.87 25000 0.6301
0.6523 5.36 27500 0.6257
0.6478 5.84 30000 0.6236
0.6393 6.33 32500 0.6145
0.6039 6.82 35000 0.5658
0.5616 7.31 37500 0.5376
0.5518 7.79 40000 0.5310
0.5509 8.28 42500 0.5273
0.5487 8.77 45000 0.5261
0.5479 9.25 47500 0.5249
0.546 9.74 50000 0.5242
0.5447 10.23 52500 0.5229
0.5439 10.71 55000 0.5220
0.5433 11.2 57500 0.5209
0.5394 11.69 60000 0.5162
0.5153 12.18 62500 0.4944
0.5137 12.66 65000 0.4932
0.514 13.15 67500 0.4924
0.5131 13.64 70000 0.4919
0.5104 14.12 72500 0.4914
0.5122 14.61 75000 0.4906
0.5089 15.1 77500 0.4901
0.5076 15.59 80000 0.4891
0.4986 16.07 82500 0.4721
0.4875 16.56 85000 0.4672
0.4887 17.05 87500 0.4669
0.4839 17.53 90000 0.4661
0.4849 18.02 92500 0.4654
0.4848 18.51 95000 0.4649
0.4831 18.99 97500 0.4646
0.4816 19.48 100000 0.4644
0.4808 19.97 102500 0.4637
0.4812 20.46 105000 0.4634
0.4813 20.94 107500 0.4633
0.4818 21.43 110000 0.4631
0.4813 21.92 112500 0.4629
0.4782 22.4 115000 0.4628
0.4804 22.89 117500 0.4626
0.4815 23.38 120000 0.4625
0.4812 23.87 122500 0.4625
0.4785 24.35 125000 0.4624
0.4795 24.84 127500 0.4624

Framework versions

  • Transformers 4.24.0
  • Pytorch 1.13.0
  • Datasets 2.7.0
  • Tokenizers 0.13.0.dev0