Hugging Face's logo

LovenOO
/
distilBERT_without_preprocessing_grid_search

Text Classification
Transformers
PyTorch
TensorFlow
TensorBoard
distilbert
Generated from Trainer
Inference Endpoints
Model card Files Metrics Community
LovenOO commited on
Commit
4fc2462
•
1 Parent(s): 1d62c5a

update model card README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -26
README.md CHANGED
@@ -1,26 +1,30 @@
1
  ---
2
  license: apache-2.0
 
3
  tags:
4
- - generated_from_keras_callback
 
 
 
 
 
5
  model-index:
6
- - name: LovenOO/distilBERT_without_preprocessing_grid_search
7
  results: []
8
  ---
9
 
10
- <!-- This model card has been generated automatically according to the information Keras had access to. You should
11
- probably proofread and complete it, then remove this comment. -->
12
 
13
- # LovenOO/distilBERT_without_preprocessing_grid_search
14
 
15
- This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
16
  It achieves the following results on the evaluation set:
17
- - Train Loss: 0.2317
18
- - Validation Loss: 0.5081
19
- - Train Precision: 0.6613
20
- - Train Recall: 0.6531
21
- - Train F1: 0.6566
22
- - Train Accuracy: 0.8581
23
- - Epoch: 4
24
 
25
  ## Model description
26
 
@@ -39,23 +43,33 @@ More information needed
39
  ### Training hyperparameters
40
 
41
  The following hyperparameters were used during training:
42
- - optimizer: {'name': 'Adam', 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-05, 'decay_steps': 5140, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False}
43
- - training_precision: float32
 
 
 
 
 
44
 
45
  ### Training results
46
 
47
- | Train Loss | Validation Loss | Train Precision | Train Recall | Train F1 | Train Accuracy | Epoch |
48
- |:----------:|:---------------:|:---------------:|:------------:|:--------:|:--------------:|:-----:|
49
- | 1.2437 | 0.6349 | 0.5675 | 0.5420 | 0.5499 | 0.8343 | 0 |
50
- | 0.5456 | 0.5544 | 0.6316 | 0.6172 | 0.6214 | 0.8435 | 1 |
51
- | 0.3904 | 0.5303 | 0.6384 | 0.6191 | 0.6239 | 0.8523 | 2 |
52
- | 0.3005 | 0.5042 | 0.6594 | 0.6486 | 0.6531 | 0.8610 | 3 |
53
- | 0.2317 | 0.5081 | 0.6613 | 0.6531 | 0.6566 | 0.8581 | 4 |
 
 
 
 
 
54
 
55
 
56
  ### Framework versions
57
 
58
- - Transformers 4.24.0
59
- - TensorFlow 2.13.0
60
- - Datasets 2.14.2
61
- - Tokenizers 0.11.0
 
1
  ---
2
  license: apache-2.0
3
+ base_model: distilbert-base-uncased
4
  tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - precision
8
+ - recall
9
+ - f1
10
+ - accuracy
11
  model-index:
12
+ - name: distilBERT_without_preprocessing_grid_search
13
  results: []
14
  ---
15
 
16
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
17
+ should probably proofread and complete it, then remove this comment. -->
18
 
19
+ # distilBERT_without_preprocessing_grid_search
20
 
21
+ This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.
22
  It achieves the following results on the evaluation set:
23
+ - Loss: 0.8740
24
+ - Precision: 0.8582
25
+ - Recall: 0.8441
26
+ - F1: 0.8491
27
+ - Accuracy: 0.8896
 
 
28
 
29
  ## Model description
30
 
 
43
  ### Training hyperparameters
44
 
45
  The following hyperparameters were used during training:
46
+ - learning_rate: 5e-05
47
+ - train_batch_size: 16
48
+ - eval_batch_size: 16
49
+ - seed: 42
50
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
+ - lr_scheduler_type: linear
52
+ - num_epochs: 10
53
 
54
  ### Training results
55
 
56
+ | Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
57
+ |:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
58
+ | 0.8195 | 1.0 | 514 | 0.5442 | 0.7965 | 0.8464 | 0.8071 | 0.8638 |
59
+ | 0.4249 | 2.0 | 1028 | 0.6446 | 0.8539 | 0.8236 | 0.8306 | 0.8769 |
60
+ | 0.3014 | 3.0 | 1542 | 0.6167 | 0.8484 | 0.8472 | 0.8463 | 0.8818 |
61
+ | 0.2268 | 4.0 | 2056 | 0.6262 | 0.8493 | 0.8594 | 0.8523 | 0.8896 |
62
+ | 0.1549 | 5.0 | 2570 | 0.6261 | 0.8443 | 0.8585 | 0.8501 | 0.8862 |
63
+ | 0.124 | 6.0 | 3084 | 0.8133 | 0.8566 | 0.8454 | 0.8503 | 0.8876 |
64
+ | 0.1057 | 7.0 | 3598 | 0.7241 | 0.8645 | 0.8596 | 0.8584 | 0.8925 |
65
+ | 0.0955 | 8.0 | 4112 | 0.8449 | 0.8532 | 0.8334 | 0.8421 | 0.8862 |
66
+ | 0.0744 | 9.0 | 4626 | 0.8140 | 0.8544 | 0.8536 | 0.8527 | 0.8901 |
67
+ | 0.0493 | 10.0 | 5140 | 0.8740 | 0.8582 | 0.8441 | 0.8491 | 0.8896 |
68
 
69
 
70
  ### Framework versions
71
 
72
+ - Transformers 4.31.0
73
+ - Pytorch 2.0.1+cu118
74
+ - Datasets 2.14.4
75
+ - Tokenizers 0.13.3