sub1-wsp / README.md
lulygavri's picture
Training in progress epoch 5
409e04d
metadata
base_model: dccuchile/bert-base-spanish-wwm-uncased
tags:
  - generated_from_keras_callback
model-index:
  - name: lulygavri/sub1-wsp
    results: []

lulygavri/sub1-wsp

This model is a fine-tuned version of dccuchile/bert-base-spanish-wwm-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Train Loss: 0.0192
  • Validation Loss: 0.2130
  • Train Accuracy: 0.9492
  • Train Precision: [0.96024845 0.88152174]
  • Train Precision W: 0.9478
  • Train Recall: [0.98025362 0.78357488]
  • Train Recall W: 0.9492
  • Train F1: [0.97014792 0.82966752]
  • Train F1 W: 0.9480
  • Epoch: 5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'transformers.optimization_tf', 'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-05, 'decay_schedule_fn': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 340, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'warmup_steps': 500, 'power': 1.0, 'name': None}, 'registered_name': 'WarmUp'}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Validation Loss Train Accuracy Train Precision Train Precision W Train Recall Train Recall W Train F1 Train F1 W Epoch
0.4028 0.2959 0.8429 [0.84274809 1. ] 0.8676 [1. 0.00483092] 0.8429 [0.91466446 0.00961538] 0.7718 1
0.1572 0.1686 0.9315 [0.94412331 0.8463357 ] 0.9287 [0.97644928 0.69178744] 0.9315 [0.96001425 0.76129718] 0.9286 2
0.1028 0.2179 0.9222 [0.92357119 0.90951638] 0.9214 [0.98949275 0.56328502] 0.9222 [0.95539619 0.69570406] 0.9144 3
0.0554 0.1492 0.9475 [0.96816208 0.83641675] 0.9474 [0.96956522 0.82995169] 0.9475 [0.96886314 0.83317168] 0.9474 4
0.0192 0.2130 0.9492 [0.96024845 0.88152174] 0.9478 [0.98025362 0.78357488] 0.9492 [0.97014792 0.82966752] 0.9480 5

Framework versions

  • Transformers 4.38.1
  • TensorFlow 2.15.0
  • Datasets 2.18.0
  • Tokenizers 0.15.2