aishanur/HV_Roberta_Large_2

README.md

@@ -0,0 +1,124 @@
+---
+license: mit
+base_model: microsoft/deberta-v3-large
+tags:
+- generated_from_trainer
+model-index:
+- name: deberta_large_hv_6
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# deberta_large_hv_6
+
+This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0584
+- F1-macro subtask 1: 0.2413
+- F1-micro subtask 1: 0.3199
+- Roc auc macro subtask 1: 0.5924
+- F1-macro subtask 2: 0.0495
+- F1-micro subtask 2: 0.0516
+- Roc auc macro subtask 2: 0.6138
+- Self-direction: thought1: 0.0440
+- Self-direction: action1: 0.1345
+- Stimulation1: 0.2982
+- Hedonism1: 0.3182
+- Achievement1: 0.3198
+- Power: dominance1: 0.1377
+- Power: resources1: 0.1513
+- Face1: 0.2367
+- Security: personal1: 0.1859
+- Security: societal1: 0.4505
+- Tradition1: 0.4289
+- Conformity: rules1: 0.4634
+- Conformity: interpersonal1: 0.0623
+- Humility1: 0.0533
+- Benevolence: caring1: 0.0588
+- Benevolence: dependability1: 0.1609
+- Universalism: concern1: 0.3206
+- Universalism: nature1: 0.5426
+- Universalism: tolerance1: 0.2165
+- Self-direction: thought attained2: 0.0187
+- Self-direction: thought constrained2: 0.0730
+- Self-direction: action attained2: 0.0523
+- Self-direction: action constrained2: 0.0374
+- Stimulation attained2: 0.0582
+- Stimulation constrained2: 0.0110
+- Hedonism attained2: 0.0164
+- Hedonism constrained2: 0.0067
+- Achievement attained2: 0.1265
+- Achievement constrained2: 0.0800
+- Power: dominance attained2: 0.0702
+- Power: dominance constrained2: 0.0731
+- Power: resources attained2: 0.0814
+- Power: resources constrained2: 0.0820
+- Face attained2: 0.0273
+- Face constrained2: 0.0297
+- Security: personal attained2: 0.0157
+- Security: personal constrained2: 0.0571
+- Security: societal attained2: 0.1026
+- Security: societal constrained2: 0.1784
+- Tradition attained2: 0.0352
+- Tradition constrained2: 0.0173
+- Conformity: rules attained2: 0.1159
+- Conformity: rules constrained2: 0.0828
+- Conformity: interpersonal attained2: 0.0171
+- Conformity: interpersonal constrained2: 0.0267
+- Humility attained2: 0.0066
+- Humility constrained2: 0.0017
+- Benevolence: caring attained2: 0.0424
+- Benevolence: caring constrained2: 0.0182
+- Benevolence: dependability attained2: 0.0346
+- Benevolence: dependability constrained2: 0.0274
+- Universalism: concern attained2: 0.0865
+- Universalism: concern constrained2: 0.0568
+- Universalism: nature attained2: 0.0464
+- Universalism: nature constrained2: 0.0430
+- Universalism: tolerance attained2: 0.0128
+- Universalism: tolerance constrained2: 0.0134
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.2
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | F1-macro subtask 1 | F1-micro subtask 1 | Roc auc macro subtask 1 | F1-macro subtask 2 | F1-micro subtask 2 | Roc auc macro subtask 2 | Self-direction: thought1 | Self-direction: action1 | Stimulation1 | Hedonism1 | Achievement1 | Power: dominance1 | Power: resources1 | Face1  | Security: personal1 | Security: societal1 | Tradition1 | Conformity: rules1 | Conformity: interpersonal1 | Humility1 | Benevolence: caring1 | Benevolence: dependability1 | Universalism: concern1 | Universalism: nature1 | Universalism: tolerance1 | Self-direction: thought attained2 | Self-direction: thought constrained2 | Self-direction: action attained2 | Self-direction: action constrained2 | Stimulation attained2 | Stimulation constrained2 | Hedonism attained2 | Hedonism constrained2 | Achievement attained2 | Achievement constrained2 | Power: dominance attained2 | Power: dominance constrained2 | Power: resources attained2 | Power: resources constrained2 | Face attained2 | Face constrained2 | Security: personal attained2 | Security: personal constrained2 | Security: societal attained2 | Security: societal constrained2 | Tradition attained2 | Tradition constrained2 | Conformity: rules attained2 | Conformity: rules constrained2 | Conformity: interpersonal attained2 | Conformity: interpersonal constrained2 | Humility attained2 | Humility constrained2 | Benevolence: caring attained2 | Benevolence: caring constrained2 | Benevolence: dependability attained2 | Benevolence: dependability constrained2 | Universalism: concern attained2 | Universalism: concern constrained2 | Universalism: nature attained2 | Universalism: nature constrained2 | Universalism: tolerance attained2 | Universalism: tolerance constrained2 |
+|:-------------:|:-----:|:-----:|:---------------:|:------------------:|:------------------:|:-----------------------:|:------------------:|:------------------:|:-----------------------:|:------------------------:|:-----------------------:|:------------:|:---------:|:------------:|:-----------------:|:-----------------:|:------:|:-------------------:|:-------------------:|:----------:|:------------------:|:--------------------------:|:---------:|:--------------------:|:---------------------------:|:----------------------:|:---------------------:|:------------------------:|:---------------------------------:|:------------------------------------:|:--------------------------------:|:-----------------------------------:|:---------------------:|:------------------------:|:------------------:|:---------------------:|:---------------------:|:------------------------:|:--------------------------:|:-----------------------------:|:--------------------------:|:-----------------------------:|:--------------:|:-----------------:|:----------------------------:|:-------------------------------:|:----------------------------:|:-------------------------------:|:-------------------:|:----------------------:|:---------------------------:|:------------------------------:|:-----------------------------------:|:--------------------------------------:|:------------------:|:---------------------:|:-----------------------------:|:--------------------------------:|:------------------------------------:|:---------------------------------------:|:-------------------------------:|:----------------------------------:|:------------------------------:|:---------------------------------:|:---------------------------------:|:------------------------------------:|
+| 0.0601        | 1.0   | 7183  | 0.0610          | 0.1579             | 0.2399             | 0.5599                  | 0.0481             | 0.0505             | 0.6300                  | 0.0                      | 0.0526                  | 0.0235       | 0.0       | 0.2704       | 0.2361            | 0.2080            | 0.2443 | 0.0207              | 0.3901              | 0.2647     | 0.3168             | 0.0867                     | 0.0       | 0.0270               | 0.0381                      | 0.0687                 | 0.5338                | 0.2180                   | 0.0173                            | 0.0169                               | 0.0517                           | 0.0521                              | 0.0598                | 0.0132                   | 0.0134             | 0.0074                | 0.1371                | 0.0682                   | 0.0618                     | 0.0615                        | 0.0860                     | 0.0705                        | 0.0215         | 0.0463            | 0.0164                       | 0.0486                          | 0.1362                       | 0.1329                          | 0.0370              | 0.0166                 | 0.0960                      | 0.1105                         | 0.0161                              | 0.0334                                 | 0.0061             | 0.0018                | 0.0522                        | 0.0137                           | 0.0441                               | 0.0145                                  | 0.0783                          | 0.0643                             | 0.0403                         | 0.0545                            | 0.0149                            | 0.0136                               |
+| 0.038         | 2.0   | 14366 | 0.0584          | 0.2413             | 0.3199             | 0.5924                  | 0.0495             | 0.0516             | 0.6138                  | 0.0440                   | 0.1345                  | 0.2982       | 0.3182    | 0.3198       | 0.1377            | 0.1513            | 0.2367 | 0.1859              | 0.4505              | 0.4289     | 0.4634             | 0.0623                     | 0.0533    | 0.0588               | 0.1609                      | 0.3206                 | 0.5426                | 0.2165                   | 0.0187                            | 0.0730                               | 0.0523                           | 0.0374                              | 0.0582                | 0.0110                   | 0.0164             | 0.0067                | 0.1265                | 0.0800                   | 0.0702                     | 0.0731                        | 0.0814                     | 0.0820                        | 0.0273         | 0.0297            | 0.0157                       | 0.0571                          | 0.1026                       | 0.1784                          | 0.0352              | 0.0173                 | 0.1159                      | 0.0828                         | 0.0171                              | 0.0267                                 | 0.0066             | 0.0017                | 0.0424                        | 0.0182                           | 0.0346                               | 0.0274                                  | 0.0865                          | 0.0568                             | 0.0464                         | 0.0430                            | 0.0128                            | 0.0134                               |
+| 0.0239        | 3.0   | 21549 | 0.0623          | 0.2723             | 0.3465             | 0.6074                  | 0.0523             | 0.0512             | 0.6170                  | 0.0788                   | 0.2017                  | 0.2390       | 0.2920    | 0.3128       | 0.3410            | 0.2956            | 0.1950 | 0.2689              | 0.4408              | 0.4435     | 0.4632             | 0.0556                     | 0.0392    | 0.1615               | 0.2495                      | 0.3972                 | 0.5443                | 0.1548                   | 0.0187                            | 0.1471                               | 0.0525                           | 0.0518                              | 0.0692                | 0.0117                   | 0.0133             | 0.0089                | 0.1289                | 0.0778                   | 0.0698                     | 0.0581                        | 0.0804                     | 0.0794                        | 0.0229         | 0.0372            | 0.0190                       | 0.0462                          | 0.0967                       | 0.1903                          | 0.0328              | 0.0260                 | 0.1065                      | 0.0924                         | 0.0169                              | 0.0281                                 | 0.0071             | 0.0019                | 0.0428                        | 0.0128                           | 0.0336                               | 0.0345                                  | 0.0766                          | 0.0714                             | 0.0409                         | 0.0535                            | 0.0186                            | 0.0106                               |
+| 0.0208        | 4.0   | 28732 | 0.0699          | 0.2913             | 0.3585             | 0.6210                  | 0.0489             | 0.0513             | 0.6155                  | 0.1195                   | 0.2460                  | 0.3053       | 0.3660    | 0.3784       | 0.3366            | 0.3114            | 0.1915 | 0.3010              | 0.4442              | 0.4366     | 0.4609             | 0.0549                     | 0.0385    | 0.2372               | 0.2329                      | 0.3927                 | 0.5479                | 0.1325                   | 0.0188                            | 0.1034                               | 0.0534                           | 0.0356                              | 0.0673                | 0.0121                   | 0.0144             | 0.0075                | 0.1251                | 0.0795                   | 0.0799                     | 0.0255                        | 0.0837                     | 0.0753                        | 0.0215         | 0.0427            | 0.0190                       | 0.0418                          | 0.1069                       | 0.1659                          | 0.0347              | 0.0160                 | 0.1180                      | 0.0823                         | 0.0212                              | 0.0228                                 | 0.0064             | 0.0017                | 0.0455                        | 0.0101                           | 0.0354                               | 0.0253                                  | 0.0788                          | 0.0662                             | 0.0450                         | 0.0430                            | 0.0140                            | 0.0120                               |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.3.0
+- Datasets 2.19.0
+- Tokenizers 0.15.1

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 128000
+}

config.json

@@ -0,0 +1,116 @@
+{
+  "_name_or_path": "microsoft/deberta-v3-large",
+  "architectures": [
+    "DebertaV2ForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "Self-direction: thought attained",
+    "1": "Self-direction: thought constrained",
+    "2": "Self-direction: action attained",
+    "3": "Self-direction: action constrained",
+    "4": "Stimulation attained",
+    "5": "Stimulation constrained",
+    "6": "Hedonism attained",
+    "7": "Hedonism constrained",
+    "8": "Achievement attained",
+    "9": "Achievement constrained",
+    "10": "Power: dominance attained",
+    "11": "Power: dominance constrained",
+    "12": "Power: resources attained",
+    "13": "Power: resources constrained",
+    "14": "Face attained",
+    "15": "Face constrained",
+    "16": "Security: personal attained",
+    "17": "Security: personal constrained",
+    "18": "Security: societal attained",
+    "19": "Security: societal constrained",
+    "20": "Tradition attained",
+    "21": "Tradition constrained",
+    "22": "Conformity: rules attained",
+    "23": "Conformity: rules constrained",
+    "24": "Conformity: interpersonal attained",
+    "25": "Conformity: interpersonal constrained",
+    "26": "Humility attained",
+    "27": "Humility constrained",
+    "28": "Benevolence: caring attained",
+    "29": "Benevolence: caring constrained",
+    "30": "Benevolence: dependability attained",
+    "31": "Benevolence: dependability constrained",
+    "32": "Universalism: concern attained",
+    "33": "Universalism: concern constrained",
+    "34": "Universalism: nature attained",
+    "35": "Universalism: nature constrained",
+    "36": "Universalism: tolerance attained",
+    "37": "Universalism: tolerance constrained"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "Achievement attained": 8,
+    "Achievement constrained": 9,
+    "Benevolence: caring attained": 28,
+    "Benevolence: caring constrained": 29,
+    "Benevolence: dependability attained": 30,
+    "Benevolence: dependability constrained": 31,
+    "Conformity: interpersonal attained": 24,
+    "Conformity: interpersonal constrained": 25,
+    "Conformity: rules attained": 22,
+    "Conformity: rules constrained": 23,
+    "Face attained": 14,
+    "Face constrained": 15,
+    "Hedonism attained": 6,
+    "Hedonism constrained": 7,
+    "Humility attained": 26,
+    "Humility constrained": 27,
+    "Power: dominance attained": 10,
+    "Power: dominance constrained": 11,
+    "Power: resources attained": 12,
+    "Power: resources constrained": 13,
+    "Security: personal attained": 16,
+    "Security: personal constrained": 17,
+    "Security: societal attained": 18,
+    "Security: societal constrained": 19,
+    "Self-direction: action attained": 2,
+    "Self-direction: action constrained": 3,
+    "Self-direction: thought attained": 0,
+    "Self-direction: thought constrained": 1,
+    "Stimulation attained": 4,
+    "Stimulation constrained": 5,
+    "Tradition attained": 20,
+    "Tradition constrained": 21,
+    "Universalism: concern attained": 32,
+    "Universalism: concern constrained": 33,
+    "Universalism: nature attained": 34,
+    "Universalism: nature constrained": 35,
+    "Universalism: tolerance attained": 36,
+    "Universalism: tolerance constrained": 37
+  },
+  "layer_norm_eps": 1e-07,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 1024,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "problem_type": "multi_label_classification",
+  "relative_attention": true,
+  "share_att_key": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.37.2",
+  "type_vocab_size": 0,
+  "vocab_size": 128100
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37b7678f6f8170df1baaae66a8fd12bd59b34abf7bc392d9ec501c40d7b4d575
+size 1740452056

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
+size 2464616

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:89c4b6e7da53cc7e100f474b10719b2cd45a046b2fb4fdcfa01199cbd07fbe68
+size 4728