aishanur/deberta_large_hv_no_upsampling

README.md

@@ -0,0 +1,124 @@
+---
+license: mit
+base_model: microsoft/deberta-v3-large
+tags:
+- generated_from_trainer
+model-index:
+- name: deberta_large_hv_no_upsampling
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# deberta_large_hv_no_upsampling
+
+This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co/microsoft/deberta-v3-large) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.0566
+- F1-macro subtask 1: 0.2178
+- F1-micro subtask 1: 0.3264
+- Roc auc macro subtask 1: 0.5833
+- F1-macro subtask 2: 0.0469
+- F1-micro subtask 2: 0.0515
+- Roc auc macro subtask 2: 0.6196
+- Self-direction: thought1: 0.0
+- Self-direction: action1: 0.1049
+- Stimulation1: 0.1055
+- Hedonism1: 0.2167
+- Achievement1: 0.3279
+- Power: dominance1: 0.3324
+- Power: resources1: 0.1996
+- Face1: 0.1995
+- Security: personal1: 0.2380
+- Security: societal1: 0.4408
+- Tradition1: 0.3850
+- Conformity: rules1: 0.4826
+- Conformity: interpersonal1: 0.0
+- Humility1: 0.0
+- Benevolence: caring1: 0.1362
+- Benevolence: dependability1: 0.0390
+- Universalism: concern1: 0.3558
+- Universalism: nature1: 0.5436
+- Universalism: tolerance1: 0.0310
+- Self-direction: thought attained2: 0.0192
+- Self-direction: thought constrained2: 0.0086
+- Self-direction: action attained2: 0.0609
+- Self-direction: action constrained2: 0.0225
+- Stimulation attained2: 0.0618
+- Stimulation constrained2: 0.0130
+- Hedonism attained2: 0.0152
+- Hedonism constrained2: 0.0071
+- Achievement attained2: 0.1285
+- Achievement constrained2: 0.0769
+- Power: dominance attained2: 0.0732
+- Power: dominance constrained2: 0.0505
+- Power: resources attained2: 0.0837
+- Power: resources constrained2: 0.0743
+- Face attained2: 0.0216
+- Face constrained2: 0.0448
+- Security: personal attained2: 0.0173
+- Security: personal constrained2: 0.0441
+- Security: societal attained2: 0.1013
+- Security: societal constrained2: 0.1787
+- Tradition attained2: 0.0399
+- Tradition constrained2: 0.0085
+- Conformity: rules attained2: 0.1024
+- Conformity: rules constrained2: 0.0964
+- Conformity: interpersonal attained2: 0.0146
+- Conformity: interpersonal constrained2: 0.0302
+- Humility attained2: 0.0050
+- Humility constrained2: 0.0019
+- Benevolence: caring attained2: 0.0500
+- Benevolence: caring constrained2: 0.0101
+- Benevolence: dependability attained2: 0.0407
+- Benevolence: dependability constrained2: 0.0199
+- Universalism: concern attained2: 0.0816
+- Universalism: concern constrained2: 0.0610
+- Universalism: nature attained2: 0.0473
+- Universalism: nature constrained2: 0.0393
+- Universalism: tolerance attained2: 0.0118
+- Universalism: tolerance constrained2: 0.0195
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_ratio: 0.2
+- num_epochs: 4
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | F1-macro subtask 1 | F1-micro subtask 1 | Roc auc macro subtask 1 | F1-macro subtask 2 | F1-micro subtask 2 | Roc auc macro subtask 2 | Self-direction: thought1 | Self-direction: action1 | Stimulation1 | Hedonism1 | Achievement1 | Power: dominance1 | Power: resources1 | Face1  | Security: personal1 | Security: societal1 | Tradition1 | Conformity: rules1 | Conformity: interpersonal1 | Humility1 | Benevolence: caring1 | Benevolence: dependability1 | Universalism: concern1 | Universalism: nature1 | Universalism: tolerance1 | Self-direction: thought attained2 | Self-direction: thought constrained2 | Self-direction: action attained2 | Self-direction: action constrained2 | Stimulation attained2 | Stimulation constrained2 | Hedonism attained2 | Hedonism constrained2 | Achievement attained2 | Achievement constrained2 | Power: dominance attained2 | Power: dominance constrained2 | Power: resources attained2 | Power: resources constrained2 | Face attained2 | Face constrained2 | Security: personal attained2 | Security: personal constrained2 | Security: societal attained2 | Security: societal constrained2 | Tradition attained2 | Tradition constrained2 | Conformity: rules attained2 | Conformity: rules constrained2 | Conformity: interpersonal attained2 | Conformity: interpersonal constrained2 | Humility attained2 | Humility constrained2 | Benevolence: caring attained2 | Benevolence: caring constrained2 | Benevolence: dependability attained2 | Benevolence: dependability constrained2 | Universalism: concern attained2 | Universalism: concern constrained2 | Universalism: nature attained2 | Universalism: nature constrained2 | Universalism: tolerance attained2 | Universalism: tolerance constrained2 |
+|:-------------:|:-----:|:-----:|:---------------:|:------------------:|:------------------:|:-----------------------:|:------------------:|:------------------:|:-----------------------:|:------------------------:|:-----------------------:|:------------:|:---------:|:------------:|:-----------------:|:-----------------:|:------:|:-------------------:|:-------------------:|:----------:|:------------------:|:--------------------------:|:---------:|:--------------------:|:---------------------------:|:----------------------:|:---------------------:|:------------------------:|:---------------------------------:|:------------------------------------:|:--------------------------------:|:-----------------------------------:|:---------------------:|:------------------------:|:------------------:|:---------------------:|:---------------------:|:------------------------:|:--------------------------:|:-----------------------------:|:--------------------------:|:-----------------------------:|:--------------:|:-----------------:|:----------------------------:|:-------------------------------:|:----------------------------:|:-------------------------------:|:-------------------:|:----------------------:|:---------------------------:|:------------------------------:|:-----------------------------------:|:--------------------------------------:|:------------------:|:---------------------:|:-----------------------------:|:--------------------------------:|:------------------------------------:|:---------------------------------------:|:-------------------------------:|:----------------------------------:|:------------------------------:|:---------------------------------:|:---------------------------------:|:------------------------------------:|
+| 0.0635        | 1.0   | 5595  | 0.0605          | 0.0659             | 0.1339             | 0.5221                  | 0.0458             | 0.0501             | 0.5899                  | 0.0                      | 0.0                     | 0.0          | 0.0       | 0.0812       | 0.3342            | 0.1051            | 0.0    | 0.0072              | 0.2917              | 0.0        | 0.0919             | 0.0                        | 0.0       | 0.0                  | 0.0                         | 0.0030                 | 0.3383                | 0.0                      | 0.0190                            | 0.0                                  | 0.0512                           | 0.0                                 | 0.0525                | 0.0083                   | 0.0146             | 0.0068                | 0.1117                | 0.1019                   | 0.0704                     | 0.0095                        | 0.0782                     | 0.0900                        | 0.0185         | 0.1068            | 0.0173                       | 0.0453                          | 0.1012                       | 0.1626                          | 0.0351              | 0.0034                 | 0.0967                      | 0.1016                         | 0.0161                              | 0.0300                                 | 0.0044             | 0.0                   | 0.0444                        | 0.0173                           | 0.0403                               | 0.0159                                  | 0.0742                          | 0.0693                             | 0.0412                         | 0.0551                            | 0.0158                            | 0.0131                               |
+| 0.0579        | 2.0   | 11190 | 0.0566          | 0.2178             | 0.3264             | 0.5833                  | 0.0469             | 0.0515             | 0.6196                  | 0.0                      | 0.1049                  | 0.1055       | 0.2167    | 0.3279       | 0.3324            | 0.1996            | 0.1995 | 0.2380              | 0.4408              | 0.3850     | 0.4826             | 0.0                        | 0.0       | 0.1362               | 0.0390                      | 0.3558                 | 0.5436                | 0.0310                   | 0.0192                            | 0.0086                               | 0.0609                           | 0.0225                              | 0.0618                | 0.0130                   | 0.0152             | 0.0071                | 0.1285                | 0.0769                   | 0.0732                     | 0.0505                        | 0.0837                     | 0.0743                        | 0.0216         | 0.0448            | 0.0173                       | 0.0441                          | 0.1013                       | 0.1787                          | 0.0399              | 0.0085                 | 0.1024                      | 0.0964                         | 0.0146                              | 0.0302                                 | 0.0050             | 0.0019                | 0.0500                        | 0.0101                           | 0.0407                               | 0.0199                                  | 0.0816                          | 0.0610                             | 0.0473                         | 0.0393                            | 0.0118                            | 0.0195                               |
+| 0.0494        | 3.0   | 16785 | 0.0593          | 0.2890             | 0.3592             | 0.6203                  | 0.0477             | 0.0517             | 0.6389                  | 0.0326                   | 0.2200                  | 0.3066       | 0.3494    | 0.3684       | 0.3375            | 0.2974            | 0.2748 | 0.3153              | 0.4421              | 0.4372     | 0.4851             | 0.0097                     | 0.0       | 0.2210               | 0.2649                      | 0.3284                 | 0.5909                | 0.2099                   | 0.0210                            | 0.0147                               | 0.0642                           | 0.0226                              | 0.0684                | 0.0111                   | 0.0150             | 0.0069                | 0.1390                | 0.0675                   | 0.0769                     | 0.0533                        | 0.0862                     | 0.0681                        | 0.0234         | 0.0406            | 0.0225                       | 0.0387                          | 0.1132                       | 0.1578                          | 0.0461              | 0.0105                 | 0.1287                      | 0.0736                         | 0.0176                              | 0.0290                                 | 0.0067             | 0.0017                | 0.0550                        | 0.0100                           | 0.0440                               | 0.0168                                  | 0.0898                          | 0.0538                             | 0.0513                         | 0.0369                            | 0.0127                            | 0.0168                               |
+| 0.028         | 4.0   | 22380 | 0.0640          | 0.2936             | 0.3626             | 0.6191                  | 0.0479             | 0.0516             | 0.6370                  | 0.0609                   | 0.2275                  | 0.3042       | 0.3444    | 0.3601       | 0.3445            | 0.3163            | 0.2599 | 0.3326              | 0.4429              | 0.4259     | 0.4933             | 0.075                      | 0.0       | 0.2313               | 0.2688                      | 0.3698                 | 0.5570                | 0.1639                   | 0.0202                            | 0.0213                               | 0.0640                           | 0.0253                              | 0.0699                | 0.0114                   | 0.0151             | 0.0070                | 0.1318                | 0.0736                   | 0.0776                     | 0.0491                        | 0.0847                     | 0.0708                        | 0.0234         | 0.0426            | 0.0210                       | 0.0400                          | 0.1080                       | 0.1655                          | 0.0419              | 0.0115                 | 0.1199                      | 0.0767                         | 0.0161                              | 0.0337                                 | 0.0069             | 0.0016                | 0.0564                        | 0.0105                           | 0.0407                               | 0.0187                                  | 0.0811                          | 0.0635                             | 0.0459                         | 0.0429                            | 0.0131                            | 0.0158                               |
+
+
+### Framework versions
+
+- Transformers 4.41.2
+- Pytorch 2.3.0
+- Datasets 2.19.0
+- Tokenizers 0.19.1

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "[MASK]": 128000
+}

config.json

@@ -0,0 +1,116 @@
+{
+  "_name_or_path": "microsoft/deberta-v3-large",
+  "architectures": [
+    "DebertaV2ForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "id2label": {
+    "0": "Self-direction: thought attained",
+    "1": "Self-direction: thought constrained",
+    "2": "Self-direction: action attained",
+    "3": "Self-direction: action constrained",
+    "4": "Stimulation attained",
+    "5": "Stimulation constrained",
+    "6": "Hedonism attained",
+    "7": "Hedonism constrained",
+    "8": "Achievement attained",
+    "9": "Achievement constrained",
+    "10": "Power: dominance attained",
+    "11": "Power: dominance constrained",
+    "12": "Power: resources attained",
+    "13": "Power: resources constrained",
+    "14": "Face attained",
+    "15": "Face constrained",
+    "16": "Security: personal attained",
+    "17": "Security: personal constrained",
+    "18": "Security: societal attained",
+    "19": "Security: societal constrained",
+    "20": "Tradition attained",
+    "21": "Tradition constrained",
+    "22": "Conformity: rules attained",
+    "23": "Conformity: rules constrained",
+    "24": "Conformity: interpersonal attained",
+    "25": "Conformity: interpersonal constrained",
+    "26": "Humility attained",
+    "27": "Humility constrained",
+    "28": "Benevolence: caring attained",
+    "29": "Benevolence: caring constrained",
+    "30": "Benevolence: dependability attained",
+    "31": "Benevolence: dependability constrained",
+    "32": "Universalism: concern attained",
+    "33": "Universalism: concern constrained",
+    "34": "Universalism: nature attained",
+    "35": "Universalism: nature constrained",
+    "36": "Universalism: tolerance attained",
+    "37": "Universalism: tolerance constrained"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "label2id": {
+    "Achievement attained": 8,
+    "Achievement constrained": 9,
+    "Benevolence: caring attained": 28,
+    "Benevolence: caring constrained": 29,
+    "Benevolence: dependability attained": 30,
+    "Benevolence: dependability constrained": 31,
+    "Conformity: interpersonal attained": 24,
+    "Conformity: interpersonal constrained": 25,
+    "Conformity: rules attained": 22,
+    "Conformity: rules constrained": 23,
+    "Face attained": 14,
+    "Face constrained": 15,
+    "Hedonism attained": 6,
+    "Hedonism constrained": 7,
+    "Humility attained": 26,
+    "Humility constrained": 27,
+    "Power: dominance attained": 10,
+    "Power: dominance constrained": 11,
+    "Power: resources attained": 12,
+    "Power: resources constrained": 13,
+    "Security: personal attained": 16,
+    "Security: personal constrained": 17,
+    "Security: societal attained": 18,
+    "Security: societal constrained": 19,
+    "Self-direction: action attained": 2,
+    "Self-direction: action constrained": 3,
+    "Self-direction: thought attained": 0,
+    "Self-direction: thought constrained": 1,
+    "Stimulation attained": 4,
+    "Stimulation constrained": 5,
+    "Tradition attained": 20,
+    "Tradition constrained": 21,
+    "Universalism: concern attained": 32,
+    "Universalism: concern constrained": 33,
+    "Universalism: nature attained": 34,
+    "Universalism: nature constrained": 35,
+    "Universalism: tolerance attained": 36,
+    "Universalism: tolerance constrained": 37
+  },
+  "layer_norm_eps": 1e-07,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 1024,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "problem_type": "multi_label_classification",
+  "relative_attention": true,
+  "share_att_key": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.41.2",
+  "type_vocab_size": 0,
+  "vocab_size": 128100
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:63439b5a164114548ea286f8f109a667649b7cc68737aff8f38452de6093e87d
+size 1740452056

special_tokens_map.json

@@ -0,0 +1,15 @@
+{
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

spm.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c679fbf93643d19aab7ee10c0b99e460bdbc02fedf34b92b05af343b4af586fd
+size 2464616

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "128000": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "sp_model_kwargs": {},
+  "split_by_punct": false,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c85730afe25aa3518ad16eb8a5cb7af17a9e84e75e59fc97fe412f3cb278c60d
+size 5112