Model save

README.md

@@ -1,6 +1,6 @@
 ---
 license: apache-2.0
-base_model: binh230/mambaformer
+base_model: OuteAI/Lite-Oute-2-Mamba2Attn-Base
 tags:
 - generated_from_trainer
 metrics:
@@ -16,19 +16,17 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/truonggiabjnh2003-fpt-university/Detect%20AI%20Generated%20Text/runs/z6i92ua2)
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/truonggiabjnh2003-fpt-university/Detect%20AI%20Generated%20Text/runs/g6m1faks)
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/truonggiabjnh2003-fpt-university/Detect%20AI%20Generated%20Text/runs/6afk83hw)
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/truonggiabjnh2003-fpt-university/Detect%20AI%20Generated%20Text/runs/vbdymxf4)
 # mambaformer
 
-This model is a fine-tuned version of [binh230/mambaformer](https://huggingface.co/binh230/mambaformer) on the None dataset.
+This model is a fine-tuned version of [OuteAI/Lite-Oute-2-Mamba2Attn-Base](https://huggingface.co/OuteAI/Lite-Oute-2-Mamba2Attn-Base) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.4950
-- Accuracy: 0.7747
-- Precision: 0.8314
-- Recall: 0.7747
-- F1: 0.7647
-- Auroc: 0.9429
+- Loss: 0.1639
+- Accuracy: 0.9607
+- Precision: 0.9628
+- Recall: 0.9607
+- F1: 0.9613
+- Auroc: 0.9925
 
 ## Model description
 
@@ -48,42 +46,30 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 5e-05
-- train_batch_size: 64
+- train_batch_size: 32
 - eval_batch_size: 4
 - seed: 42
 - gradient_accumulation_steps: 4
-- total_train_batch_size: 256
+- total_train_batch_size: 128
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
 - num_epochs: 1
-- mixed_precision_training: Native AMP
 - label_smoothing_factor: 0.03
 
 ### Training results
 
 | Training Loss | Epoch  | Step | Validation Loss | Accuracy | Precision | Recall | F1     | Auroc  |
 |:-------------:|:------:|:----:|:---------------:|:--------:|:---------:|:------:|:------:|:------:|
-| 0.6026        | 0.0471 | 256  | 0.5927          | 0.6776   | 0.7982    | 0.6776 | 0.6414 | 0.9217 |
-| 0.3481        | 0.0941 | 512  | 0.4428          | 0.8167   | 0.8239    | 0.8167 | 0.8157 | 0.9015 |
-| 0.2434        | 0.1412 | 768  | 0.4749          | 0.7833   | 0.8043    | 0.7833 | 0.7795 | 0.8934 |
-| 0.1975        | 0.1882 | 1024 | 0.5786          | 0.7304   | 0.7949    | 0.7304 | 0.7149 | 0.8979 |
-| 0.1749        | 0.2353 | 1280 | 0.6214          | 0.7157   | 0.7952    | 0.7157 | 0.6952 | 0.9004 |
-| 0.1644        | 0.2824 | 1536 | 0.6323          | 0.7107   | 0.7984    | 0.7107 | 0.6877 | 0.9123 |
-| 0.1556        | 0.3294 | 1792 | 0.6491          | 0.7046   | 0.7990    | 0.7046 | 0.6793 | 0.9161 |
-| 0.1476        | 0.3765 | 2048 | 0.6989          | 0.6884   | 0.7955    | 0.6884 | 0.6573 | 0.9203 |
-| 0.1466        | 0.4235 | 2304 | 0.6633          | 0.7014   | 0.8016    | 0.7014 | 0.6744 | 0.9241 |
-| 0.1405        | 0.4706 | 2560 | 0.6076          | 0.7229   | 0.8071    | 0.7229 | 0.7026 | 0.9250 |
-| 0.1399        | 0.5177 | 2816 | 0.6221          | 0.7164   | 0.8042    | 0.7164 | 0.6943 | 0.9248 |
-| 0.135         | 0.5647 | 3072 | 0.6249          | 0.7150   | 0.8064    | 0.7150 | 0.6920 | 0.9290 |
-| 0.1339        | 0.6118 | 3328 | 0.6109          | 0.7233   | 0.8108    | 0.7233 | 0.7024 | 0.9340 |
-| 0.1297        | 0.6589 | 3584 | 0.5931          | 0.7306   | 0.8127    | 0.7306 | 0.7117 | 0.9360 |
-| 0.1298        | 0.7059 | 3840 | 0.5644          | 0.7439   | 0.8170    | 0.7439 | 0.7282 | 0.9357 |
-| 0.1275        | 0.7530 | 4096 | 0.5526          | 0.7475   | 0.8209    | 0.7475 | 0.7322 | 0.9416 |
-| 0.1268        | 0.8000 | 4352 | 0.5564          | 0.7470   | 0.8203    | 0.7470 | 0.7317 | 0.9412 |
-| 0.1235        | 0.8471 | 4608 | 0.5439          | 0.7537   | 0.8238    | 0.7537 | 0.7396 | 0.9436 |
-| 0.1231        | 0.8942 | 4864 | 0.5051          | 0.7693   | 0.8292    | 0.7693 | 0.7583 | 0.9446 |
-| 0.1222        | 0.9412 | 5120 | 0.5254          | 0.7611   | 0.8241    | 0.7611 | 0.7488 | 0.9405 |
-| 0.1198        | 0.9883 | 5376 | 0.5439          | 0.7534   | 0.8230    | 0.7534 | 0.7394 | 0.9408 |
+| 0.8973        | 0.0988 | 128  | 0.6661          | 0.6897   | 0.6807    | 0.6897 | 0.6850 | 0.5552 |
+| 0.5525        | 0.1976 | 256  | 0.4682          | 0.7898   | 0.7526    | 0.7898 | 0.7413 | 0.7643 |
+| 0.4086        | 0.2965 | 384  | 0.3500          | 0.8523   | 0.8452    | 0.8523 | 0.8472 | 0.9024 |
+| 0.3067        | 0.3953 | 512  | 0.2573          | 0.9107   | 0.9085    | 0.9107 | 0.9091 | 0.9620 |
+| 0.2477        | 0.4941 | 640  | 0.2234          | 0.9309   | 0.9298    | 0.9309 | 0.9288 | 0.9761 |
+| 0.2283        | 0.5929 | 768  | 0.2074          | 0.9404   | 0.9396    | 0.9404 | 0.9398 | 0.9804 |
+| 0.2035        | 0.6918 | 896  | 0.1875          | 0.9529   | 0.9530    | 0.9529 | 0.9530 | 0.9853 |
+| 0.1963        | 0.7906 | 1024 | 0.1809          | 0.9464   | 0.9458    | 0.9464 | 0.9460 | 0.9867 |
+| 0.1798        | 0.8894 | 1152 | 0.1638          | 0.9601   | 0.9610    | 0.9601 | 0.9604 | 0.9900 |
+| 0.1749        | 0.9882 | 1280 | 0.1652          | 0.9583   | 0.9579    | 0.9583 | 0.9581 | 0.9894 |
 
 
 ### Framework versions