Upload TFBertForMaskedLM

README.md

@@ -1,6 +1,6 @@
 ---
 license: apache-2.0
-base_model: raygx/NepaliLM-BERT
+base_model: bert-base-uncased
 tags:
 - generated_from_keras_callback
 model-index:
@@ -13,11 +13,11 @@ probably proofread and complete it, then remove this comment. -->
 
 # NepaliLM-BERT
 
-This model is a fine-tuned version of [raygx/NepaliLM-BERT](https://huggingface.co/raygx/NepaliLM-BERT) on an unknown dataset.
+This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Train Loss: 5.6802
-- Validation Loss: 5.5575
-- Epoch: 1
+- Train Loss: 6.4102
+- Validation Loss: 6.3584
+- Epoch: 0
 
 ## Model description
 
@@ -36,20 +36,19 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 2e-06, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 2e-06, 'decay_steps': 12205, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.065}
+- optimizer: {'name': 'AdamWeightDecay', 'learning_rate': {'class_name': 'WarmUp', 'config': {'initial_learning_rate': 1e-06, 'decay_schedule_fn': {'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 1e-06, 'decay_steps': 40996, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, '__passive_serialization__': True}, 'warmup_steps': 1000, 'power': 1.0, 'name': None}}, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-08, 'amsgrad': False, 'weight_decay_rate': 0.0001}
 - training_precision: mixed_bfloat16
 
 ### Training results
 
 | Train Loss | Validation Loss | Epoch |
 |:----------:|:---------------:|:-----:|
-| 5.6988     | 5.5615          | 0     |
-| 5.6802     | 5.5575          | 1     |
+| 6.4102     | 6.3584          | 0     |
 
 
 ### Framework versions
 
-- Transformers 4.31.0
+- Transformers 4.32.0
 - TensorFlow 2.12.0
 - Datasets 2.14.4
 - Tokenizers 0.13.3

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "raygx/NepaliLM-BERT",
+  "_name_or_path": "bert-base-uncased",
   "architectures": [
     "BertForMaskedLM"
   ],
@@ -19,7 +19,7 @@
   "pad_token_id": 3,
   "position_embedding_type": "absolute",
   "sep_token_id": 2,
-  "transformers_version": "4.31.0",
+  "transformers_version": "4.32.0",
   "type_vocab_size": 2,
   "use_cache": true,
   "vocab_size": 50000

tf_model.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:296aa4f983116b4f393ca5acb0510c628593f4eff106b33e819dca68f8bbfff7
+oid sha256:48ae65bee1a082db2f9549ceb87d8a58014d3a7d62f8e4166add0a6444b4446b
 size 653438360