Hugging Face's logo

fadhilarkan
/
qa-indo-math-k-v2

Text2Text Generation
Transformers
PyTorch
t5
text-generation-inference
Inference Endpoints
Model card Files Community
1
Muhammad Fadhil Arkan commited on
Commit
e1c3e77
·
1 Parent(s): cf57bce

add model

Browse files
Files changed (7) hide show
  1. README.md +153 -0
  2. config.json +40 -0
  3. pytorch_model.bin +3 -0
  4. special_tokens_map.json +1 -0
  5. spiece.model +3 -0
  6. tokenizer.json +0 -0
  7. tokenizer_config.json +1 -0
README.md ADDED
@@ -0,0 +1,153 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+
3
+ model-index:
4
+ - name: qa-indo-math-k-v2
5
+ ---
6
+
7
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
8
+ should probably proofread and complete it, then remove this comment. -->
9
+
10
+ # qa-indo-math-k-v2
11
+
12
+ This model was trained from scratch on an unkown dataset.
13
+ It achieves the following results on the evaluation set:
14
+ - Loss: 1.9328
15
+
16
+ ## Model description
17
+
18
+ More information needed
19
+
20
+ ## Intended uses & limitations
21
+
22
+ More information needed
23
+
24
+ ## Training and evaluation data
25
+
26
+ More information needed
27
+
28
+ ## Training procedure
29
+
30
+ ### Training hyperparameters
31
+
32
+ The following hyperparameters were used during training:
33
+ - learning_rate: 2e-05
34
+ - train_batch_size: 16
35
+ - eval_batch_size: 16
36
+ - seed: 42
37
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
38
+ - lr_scheduler_type: linear
39
+ - num_epochs: 100
40
+ - mixed_precision_training: Native AMP
41
+
42
+ ### Training results
43
+
44
+ | Training Loss | Epoch | Step | Validation Loss |
45
+ |:-------------:|:-----:|:----:|:---------------:|
46
+ | No log | 1.0 | 80 | 0.7969 |
47
+ | No log | 2.0 | 160 | 0.7612 |
48
+ | No log | 3.0 | 240 | 0.7624 |
49
+ | No log | 4.0 | 320 | 0.7424 |
50
+ | No log | 5.0 | 400 | 0.7634 |
51
+ | No log | 6.0 | 480 | 0.7415 |
52
+ | 0.9241 | 7.0 | 560 | 0.7219 |
53
+ | 0.9241 | 8.0 | 640 | 0.7792 |
54
+ | 0.9241 | 9.0 | 720 | 0.7803 |
55
+ | 0.9241 | 10.0 | 800 | 0.7666 |
56
+ | 0.9241 | 11.0 | 880 | 0.7614 |
57
+ | 0.9241 | 12.0 | 960 | 0.7616 |
58
+ | 0.6373 | 13.0 | 1040 | 0.7673 |
59
+ | 0.6373 | 14.0 | 1120 | 0.7818 |
60
+ | 0.6373 | 15.0 | 1200 | 0.8030 |
61
+ | 0.6373 | 16.0 | 1280 | 0.8021 |
62
+ | 0.6373 | 17.0 | 1360 | 0.8025 |
63
+ | 0.6373 | 18.0 | 1440 | 0.8628 |
64
+ | 0.5614 | 19.0 | 1520 | 0.8616 |
65
+ | 0.5614 | 20.0 | 1600 | 0.8739 |
66
+ | 0.5614 | 21.0 | 1680 | 0.8647 |
67
+ | 0.5614 | 22.0 | 1760 | 0.9006 |
68
+ | 0.5614 | 23.0 | 1840 | 0.9560 |
69
+ | 0.5614 | 24.0 | 1920 | 0.9395 |
70
+ | 0.486 | 25.0 | 2000 | 0.9453 |
71
+ | 0.486 | 26.0 | 2080 | 0.9569 |
72
+ | 0.486 | 27.0 | 2160 | 1.0208 |
73
+ | 0.486 | 28.0 | 2240 | 0.9860 |
74
+ | 0.486 | 29.0 | 2320 | 0.9806 |
75
+ | 0.486 | 30.0 | 2400 | 1.0681 |
76
+ | 0.486 | 31.0 | 2480 | 1.1085 |
77
+ | 0.4126 | 32.0 | 2560 | 1.1028 |
78
+ | 0.4126 | 33.0 | 2640 | 1.1110 |
79
+ | 0.4126 | 34.0 | 2720 | 1.1573 |
80
+ | 0.4126 | 35.0 | 2800 | 1.1387 |
81
+ | 0.4126 | 36.0 | 2880 | 1.2067 |
82
+ | 0.4126 | 37.0 | 2960 | 1.2079 |
83
+ | 0.3559 | 38.0 | 3040 | 1.2152 |
84
+ | 0.3559 | 39.0 | 3120 | 1.2418 |
85
+ | 0.3559 | 40.0 | 3200 | 1.2023 |
86
+ | 0.3559 | 41.0 | 3280 | 1.2679 |
87
+ | 0.3559 | 42.0 | 3360 | 1.3178 |
88
+ | 0.3559 | 43.0 | 3440 | 1.3419 |
89
+ | 0.3084 | 44.0 | 3520 | 1.4702 |
90
+ | 0.3084 | 45.0 | 3600 | 1.3824 |
91
+ | 0.3084 | 46.0 | 3680 | 1.4227 |
92
+ | 0.3084 | 47.0 | 3760 | 1.3925 |
93
+ | 0.3084 | 48.0 | 3840 | 1.4940 |
94
+ | 0.3084 | 49.0 | 3920 | 1.4110 |
95
+ | 0.2686 | 50.0 | 4000 | 1.4534 |
96
+ | 0.2686 | 51.0 | 4080 | 1.4749 |
97
+ | 0.2686 | 52.0 | 4160 | 1.5351 |
98
+ | 0.2686 | 53.0 | 4240 | 1.5479 |
99
+ | 0.2686 | 54.0 | 4320 | 1.4755 |
100
+ | 0.2686 | 55.0 | 4400 | 1.5207 |
101
+ | 0.2686 | 56.0 | 4480 | 1.5075 |
102
+ | 0.2388 | 57.0 | 4560 | 1.5470 |
103
+ | 0.2388 | 58.0 | 4640 | 1.5361 |
104
+ | 0.2388 | 59.0 | 4720 | 1.5914 |
105
+ | 0.2388 | 60.0 | 4800 | 1.6430 |
106
+ | 0.2388 | 61.0 | 4880 | 1.6249 |
107
+ | 0.2388 | 62.0 | 4960 | 1.5503 |
108
+ | 0.2046 | 63.0 | 5040 | 1.6441 |
109
+ | 0.2046 | 64.0 | 5120 | 1.6789 |
110
+ | 0.2046 | 65.0 | 5200 | 1.6174 |
111
+ | 0.2046 | 66.0 | 5280 | 1.6175 |
112
+ | 0.2046 | 67.0 | 5360 | 1.6947 |
113
+ | 0.2046 | 68.0 | 5440 | 1.6299 |
114
+ | 0.1891 | 69.0 | 5520 | 1.7419 |
115
+ | 0.1891 | 70.0 | 5600 | 1.8442 |
116
+ | 0.1891 | 71.0 | 5680 | 1.8802 |
117
+ | 0.1891 | 72.0 | 5760 | 1.8233 |
118
+ | 0.1891 | 73.0 | 5840 | 1.8172 |
119
+ | 0.1891 | 74.0 | 5920 | 1.8181 |
120
+ | 0.1664 | 75.0 | 6000 | 1.8399 |
121
+ | 0.1664 | 76.0 | 6080 | 1.8128 |
122
+ | 0.1664 | 77.0 | 6160 | 1.8423 |
123
+ | 0.1664 | 78.0 | 6240 | 1.8380 |
124
+ | 0.1664 | 79.0 | 6320 | 1.8941 |
125
+ | 0.1664 | 80.0 | 6400 | 1.8636 |
126
+ | 0.1664 | 81.0 | 6480 | 1.7949 |
127
+ | 0.1614 | 82.0 | 6560 | 1.8342 |
128
+ | 0.1614 | 83.0 | 6640 | 1.8123 |
129
+ | 0.1614 | 84.0 | 6720 | 1.8639 |
130
+ | 0.1614 | 85.0 | 6800 | 1.8580 |
131
+ | 0.1614 | 86.0 | 6880 | 1.8816 |
132
+ | 0.1614 | 87.0 | 6960 | 1.8579 |
133
+ | 0.1487 | 88.0 | 7040 | 1.8783 |
134
+ | 0.1487 | 89.0 | 7120 | 1.9175 |
135
+ | 0.1487 | 90.0 | 7200 | 1.9025 |
136
+ | 0.1487 | 91.0 | 7280 | 1.9207 |
137
+ | 0.1487 | 92.0 | 7360 | 1.9195 |
138
+ | 0.1487 | 93.0 | 7440 | 1.9142 |
139
+ | 0.1355 | 94.0 | 7520 | 1.9333 |
140
+ | 0.1355 | 95.0 | 7600 | 1.9238 |
141
+ | 0.1355 | 96.0 | 7680 | 1.9256 |
142
+ | 0.1355 | 97.0 | 7760 | 1.9305 |
143
+ | 0.1355 | 98.0 | 7840 | 1.9294 |
144
+ | 0.1355 | 99.0 | 7920 | 1.9301 |
145
+ | 0.1297 | 100.0 | 8000 | 1.9328 |
146
+
147
+
148
+ ### Framework versions
149
+
150
+ - Transformers 4.6.1
151
+ - Pytorch 1.7.0
152
+ - Datasets 1.11.0
153
+ - Tokenizers 0.10.3
config.json ADDED
@@ -0,0 +1,40 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "panggi/t5-small-indonesian-summarization-cased",
3
+ "architectures": [
4
+ "T5ForConditionalGeneration"
5
+ ],
6
+ "d_ff": 2048,
7
+ "d_kv": 64,
8
+ "d_model": 512,
9
+ "decoder_start_token_id": 0,
10
+ "dropout_rate": 0.1,
11
+ "eos_token_id": 1,
12
+ "feed_forward_proj": "relu",
13
+ "gradient_checkpointing": false,
14
+ "initializer_factor": 1.0,
15
+ "inputs_length": 1024,
16
+ "is_encoder_decoder": true,
17
+ "layer_norm_epsilon": 1e-06,
18
+ "model_type": "t5",
19
+ "n_positions": 1024,
20
+ "num_decoder_layers": 6,
21
+ "num_heads": 8,
22
+ "num_layers": 6,
23
+ "pad_token_id": 0,
24
+ "relative_attention_num_buckets": 32,
25
+ "task_specific_params": {
26
+ "summarization": {
27
+ "early_stopping": true,
28
+ "length_penalty": 2.0,
29
+ "max_length": 100,
30
+ "min_length": 50,
31
+ "no_repeat_ngram_size": 2,
32
+ "num_beams": 2,
33
+ "prefix": "summarize: ",
34
+ "repetition_penalty": 2.5
35
+ }
36
+ },
37
+ "transformers_version": "4.6.1",
38
+ "use_cache": true,
39
+ "vocab_size": 32128
40
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3a3cebec013a7346960ae4706b4249ec89033f19ec8b4feb7843c9d89660062b
3
+ size 242089354
special_tokens_map.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"]}
spiece.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ec91e24db6b3ab052b7a93bd6ac3fc0d06727ff3a57d462cada3c00783430173
3
+ size 793027
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"eos_token": "</s>", "unk_token": "<unk>", "pad_token": "<pad>", "extra_ids": 100, "additional_special_tokens": ["<extra_id_0>", "<extra_id_1>", "<extra_id_2>", "<extra_id_3>", "<extra_id_4>", "<extra_id_5>", "<extra_id_6>", "<extra_id_7>", "<extra_id_8>", "<extra_id_9>", "<extra_id_10>", "<extra_id_11>", "<extra_id_12>", "<extra_id_13>", "<extra_id_14>", "<extra_id_15>", "<extra_id_16>", "<extra_id_17>", "<extra_id_18>", "<extra_id_19>", "<extra_id_20>", "<extra_id_21>", "<extra_id_22>", "<extra_id_23>", "<extra_id_24>", "<extra_id_25>", "<extra_id_26>", "<extra_id_27>", "<extra_id_28>", "<extra_id_29>", "<extra_id_30>", "<extra_id_31>", "<extra_id_32>", "<extra_id_33>", "<extra_id_34>", "<extra_id_35>", "<extra_id_36>", "<extra_id_37>", "<extra_id_38>", "<extra_id_39>", "<extra_id_40>", "<extra_id_41>", "<extra_id_42>", "<extra_id_43>", "<extra_id_44>", "<extra_id_45>", "<extra_id_46>", "<extra_id_47>", "<extra_id_48>", "<extra_id_49>", "<extra_id_50>", "<extra_id_51>", "<extra_id_52>", "<extra_id_53>", "<extra_id_54>", "<extra_id_55>", "<extra_id_56>", "<extra_id_57>", "<extra_id_58>", "<extra_id_59>", "<extra_id_60>", "<extra_id_61>", "<extra_id_62>", "<extra_id_63>", "<extra_id_64>", "<extra_id_65>", "<extra_id_66>", "<extra_id_67>", "<extra_id_68>", "<extra_id_69>", "<extra_id_70>", "<extra_id_71>", "<extra_id_72>", "<extra_id_73>", "<extra_id_74>", "<extra_id_75>", "<extra_id_76>", "<extra_id_77>", "<extra_id_78>", "<extra_id_79>", "<extra_id_80>", "<extra_id_81>", "<extra_id_82>", "<extra_id_83>", "<extra_id_84>", "<extra_id_85>", "<extra_id_86>", "<extra_id_87>", "<extra_id_88>", "<extra_id_89>", "<extra_id_90>", "<extra_id_91>", "<extra_id_92>", "<extra_id_93>", "<extra_id_94>", "<extra_id_95>", "<extra_id_96>", "<extra_id_97>", "<extra_id_98>", "<extra_id_99>"], "volatile": true, "special_tokens_map_file": "/root/.cache/huggingface/transformers/c9ef00b3383d33979f3034d492ec78ac482a9335c4589a46b750be52d484ff9f.294ebaa4cd17bb284635004c92d2c4d522ec488c828dcce0c2471b6f28e3fe82", "name_or_path": "panggi/t5-small-indonesian-summarization-cased"}