railwayBK-sentiment-analysis

README.md

@@ -0,0 +1,101 @@
+---
+base_model: ProsusAI/finbert
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+- f1
+- precision
+- recall
+model-index:
+- name: finBert
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# finBert
+
+This model is a fine-tuned version of [ProsusAI/finbert](https://huggingface.co/ProsusAI/finbert) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.3623
+- Accuracy: 0.868
+- F1: 0.8666
+- Precision: 0.8671
+- Recall: 0.8685
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 32
+- eval_batch_size: 32
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 100
+- num_epochs: 3
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step | Validation Loss | Accuracy | F1     | Precision | Recall |
+|:-------------:|:------:|:----:|:---------------:|:--------:|:------:|:---------:|:------:|
+| 1.2152        | 0.0820 | 50   | 0.6140          | 0.7004   | 0.6545 | 0.7339    | 0.7020 |
+| 0.5357        | 0.1639 | 100  | 0.4736          | 0.8265   | 0.8216 | 0.8270    | 0.8268 |
+| 0.4421        | 0.2459 | 150  | 0.4189          | 0.8501   | 0.8466 | 0.8494    | 0.8505 |
+| 0.4591        | 0.3279 | 200  | 0.4177          | 0.8390   | 0.8336 | 0.8417    | 0.8395 |
+| 0.4382        | 0.4098 | 250  | 0.3876          | 0.8549   | 0.8537 | 0.8538    | 0.8550 |
+| 0.4002        | 0.4918 | 300  | 0.4063          | 0.8434   | 0.8454 | 0.8503    | 0.8433 |
+| 0.4085        | 0.5738 | 350  | 0.3818          | 0.8576   | 0.8574 | 0.8572    | 0.8579 |
+| 0.3692        | 0.6557 | 400  | 0.3700          | 0.8608   | 0.8592 | 0.8598    | 0.8611 |
+| 0.3686        | 0.7377 | 450  | 0.3730          | 0.8610   | 0.8604 | 0.8600    | 0.8613 |
+| 0.3966        | 0.8197 | 500  | 0.3760          | 0.8566   | 0.8573 | 0.8584    | 0.8569 |
+| 0.3729        | 0.9016 | 550  | 0.3533          | 0.8673   | 0.8667 | 0.8664    | 0.8675 |
+| 0.3605        | 0.9836 | 600  | 0.3582          | 0.8632   | 0.8637 | 0.8645    | 0.8632 |
+| 0.3471        | 1.0656 | 650  | 0.3671          | 0.8644   | 0.8643 | 0.8646    | 0.8645 |
+| 0.3397        | 1.1475 | 700  | 0.3632          | 0.8658   | 0.8658 | 0.8656    | 0.8660 |
+| 0.2977        | 1.2295 | 750  | 0.3737          | 0.8631   | 0.8619 | 0.8620    | 0.8634 |
+| 0.3047        | 1.3115 | 800  | 0.3609          | 0.8656   | 0.8664 | 0.8673    | 0.8657 |
+| 0.2817        | 1.3934 | 850  | 0.3816          | 0.8682   | 0.8661 | 0.8675    | 0.8685 |
+| 0.2667        | 1.4754 | 900  | 0.3734          | 0.8690   | 0.8680 | 0.8686    | 0.8692 |
+| 0.2809        | 1.5574 | 950  | 0.3523          | 0.8702   | 0.8706 | 0.8710    | 0.8703 |
+| 0.2964        | 1.6393 | 1000 | 0.3621          | 0.8709   | 0.8693 | 0.8705    | 0.8711 |
+| 0.2993        | 1.7213 | 1050 | 0.3502          | 0.8731   | 0.8728 | 0.8729    | 0.8732 |
+| 0.3153        | 1.8033 | 1100 | 0.3532          | 0.8723   | 0.8703 | 0.8718    | 0.8726 |
+| 0.3089        | 1.8852 | 1150 | 0.3581          | 0.8721   | 0.8710 | 0.8711    | 0.8724 |
+| 0.3027        | 1.9672 | 1200 | 0.3513          | 0.8725   | 0.8712 | 0.8719    | 0.8727 |
+| 0.2294        | 2.0492 | 1250 | 0.3673          | 0.8714   | 0.8715 | 0.8715    | 0.8715 |
+| 0.2321        | 2.1311 | 1300 | 0.3630          | 0.8716   | 0.8718 | 0.8727    | 0.8716 |
+| 0.2003        | 2.2131 | 1350 | 0.3951          | 0.872    | 0.8714 | 0.8711    | 0.8722 |
+| 0.1948        | 2.2951 | 1400 | 0.3912          | 0.8724   | 0.8721 | 0.8719    | 0.8726 |
+| 0.1939        | 2.3770 | 1450 | 0.3871          | 0.8733   | 0.8734 | 0.8734    | 0.8735 |
+| 0.1897        | 2.4590 | 1500 | 0.3937          | 0.8738   | 0.8732 | 0.8730    | 0.8741 |
+| 0.1951        | 2.5410 | 1550 | 0.3915          | 0.8711   | 0.8702 | 0.8704    | 0.8713 |
+| 0.193         | 2.6230 | 1600 | 0.3902          | 0.8732   | 0.8729 | 0.8727    | 0.8734 |
+| 0.1982        | 2.7049 | 1650 | 0.3903          | 0.8734   | 0.8734 | 0.8733    | 0.8736 |
+| 0.1836        | 2.7869 | 1700 | 0.3929          | 0.8734   | 0.8732 | 0.8731    | 0.8736 |
+| 0.2112        | 2.8689 | 1750 | 0.3927          | 0.8742   | 0.8738 | 0.8738    | 0.8743 |
+| 0.1937        | 2.9508 | 1800 | 0.3899          | 0.8738   | 0.8739 | 0.8741    | 0.8740 |
+
+
+### Framework versions
+
+- Transformers 4.42.4
+- Pytorch 2.3.1+cu121
+- Tokenizers 0.19.1

config.json

@@ -0,0 +1,37 @@
+{
+  "_name_or_path": "ProsusAI/finbert",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "Positive",
+    "1": "Neutral",
+    "2": "Negative"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "Negative": 2,
+    "Neutral": 1,
+    "Positive": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "problem_type": "single_label_classification",
+  "torch_dtype": "float32",
+  "transformers_version": "4.42.4",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:38680c0a8ea1fb6d962d52faaae32bfa1ee5cb80fa4cbe42fdf0e885d755baf3
+size 437961724

multi-class-logs/events.out.tfevents.1721280146.4867d8d32b72.4228.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f37af7f3e26eaf7217ce2f2c9a7f94f29a7e80aa8ffde8261595eb88d784920a
+size 30015

multi-class-logs/events.out.tfevents.1721284078.4867d8d32b72.4228.1

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55cb682c05184ec91f0c512c2875abe1f70811969b2e38d5d1c8a438b63df02f
+size 1504

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96fc7d766cdf02412a5d1b261185556de7cc08c52e37956b0f2e385760c094c2
+size 5176