Hugging Face's logo

MUmairAB
/
bert-based-MaskedLM

Fill-Mask
Transformers
TensorFlow
distilbert
generated_from_keras_callback
Inference Endpoints
Model card Files Community
1
MUmairAB commited on
Commit
238be91
•
1 Parent(s): b1e955d

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -16
README.md CHANGED
@@ -5,6 +5,9 @@ tags:
5
  model-index:
6
  - name: MUmairAB/bert-based-MaskedLM
7
  results: []
 
 
 
8
  ---
9
 
10
  <!-- This model card has been generated automatically according to the information Keras had access to. You should
@@ -12,23 +15,45 @@ probably proofread and complete it, then remove this comment. -->
12
 
13
  # MUmairAB/bert-based-MaskedLM
14
 
15
- This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
16
  It achieves the following results on the evaluation set:
17
  - Train Loss: 2.4360
18
  - Validation Loss: 2.3284
19
- - Epoch: 9
20
 
21
  ## Model description
22
 
23
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
 
25
  ## Intended uses & limitations
26
 
27
- More information needed
28
 
29
  ## Training and evaluation data
30
 
31
- More information needed
32
 
33
  ## Training procedure
34
 
@@ -42,20 +67,31 @@ The following hyperparameters were used during training:
42
 
43
  | Train Loss | Validation Loss | Epoch |
44
  |:----------:|:---------------:|:-----:|
45
- | 2.4443 | 2.2974 | 0 |
46
- | 2.4497 | 2.3317 | 1 |
47
- | 2.4371 | 2.3317 | 2 |
48
- | 2.4377 | 2.3237 | 3 |
49
- | 2.4369 | 2.3338 | 4 |
50
- | 2.4350 | 2.3021 | 5 |
51
- | 2.4267 | 2.3264 | 6 |
52
- | 2.4557 | 2.3280 | 7 |
53
- | 2.4461 | 2.3165 | 8 |
54
- | 2.4360 | 2.3284 | 9 |
 
 
 
 
 
 
 
 
 
 
 
55
 
56
 
57
  ### Framework versions
58
 
59
  - Transformers 4.30.2
60
  - TensorFlow 2.12.0
61
- - Tokenizers 0.13.3
 
5
  model-index:
6
  - name: MUmairAB/bert-based-MaskedLM
7
  results: []
8
+ datasets:
9
+ - imdb
10
+ pipeline_tag: fill-mask
11
  ---
12
 
13
  <!-- This model card has been generated automatically according to the information Keras had access to. You should
 
15
 
16
  # MUmairAB/bert-based-MaskedLM
17
 
18
+ This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on [IMDB Movies Review](https://huggingface.co/datasets/imdb) dataset.
19
  It achieves the following results on the evaluation set:
20
  - Train Loss: 2.4360
21
  - Validation Loss: 2.3284
22
+ - Epoch: 20
23
 
24
  ## Model description
25
 
26
+ [DistilBERT-base-uncased](https://huggingface.co/distilbert-base-uncased)
27
+ ```
28
+ Model: "tf_distil_bert_for_masked_lm"
29
+ _________________________________________________________________
30
+ Layer (type) Output Shape Param #
31
+ =================================================================
32
+ distilbert (TFDistilBertMai multiple 66362880
33
+ nLayer)
34
+
35
+ vocab_transform (Dense) multiple 590592
36
+
37
+ vocab_layer_norm (LayerNorm multiple 1536
38
+ alization)
39
+
40
+ vocab_projector (TFDistilBe multiple 23866170
41
+ rtLMHead)
42
+
43
+ =================================================================
44
+ Total params: 66,985,530
45
+ Trainable params: 66,985,530
46
+ Non-trainable params: 0
47
+ _________________________________________________________________
48
+ ```
49
 
50
  ## Intended uses & limitations
51
 
52
+ The model was trained on IMDB movies review dataset. So, it inherits the language biases from the dataset.
53
 
54
  ## Training and evaluation data
55
 
56
+ The model was trained on [IMDB Movies Review](https://huggingface.co/datasets/imdb) dataset.
57
 
58
  ## Training procedure
59
 
 
67
 
68
  | Train Loss | Validation Loss | Epoch |
69
  |:----------:|:---------------:|:-----:|
70
+ | 3.0754 | 2.7548 | 0 |
71
+ | 2.7969 | 2.6209 | 1 |
72
+ | 2.7214 | 2.5588 | 2 |
73
+ | 2.6626 | 2.5554 | 3 |
74
+ | 2.6466 | 2.4881 | 4 |
75
+ | 2.6238 | 2.4775 | 5 |
76
+ | 2.5696 | 2.4280 | 6 |
77
+ | 2.5504 | 2.3924 | 7 |
78
+ | 2.5171 | 2.3725 | 8 |
79
+ | 2.5180 | 2.3142 | 9 |
80
+ | 2.4443 | 2.2974 | 10 |
81
+ | 2.4497 | 2.3317 | 11 |
82
+ | 2.4371 | 2.3317 | 12 |
83
+ | 2.4377 | 2.3237 | 13 |
84
+ | 2.4369 | 2.3338 | 14 |
85
+ | 2.4350 | 2.3021 | 15 |
86
+ | 2.4267 | 2.3264 | 16 |
87
+ | 2.4557 | 2.3280 | 17 |
88
+ | 2.4461 | 2.3165 | 18 |
89
+ | 2.4360 | 2.3284 | 19 |
90
+
91
 
92
 
93
  ### Framework versions
94
 
95
  - Transformers 4.30.2
96
  - TensorFlow 2.12.0
97
+ - Tokenizers 0.13.3