|
--- |
|
license: apache-2.0 |
|
library_name: peft |
|
tags: |
|
- generated_from_trainer |
|
base_model: mistralai/Mistral-7B-v0.1 |
|
datasets: |
|
- mbe |
|
metrics: |
|
- accuracy |
|
model-index: |
|
- name: Mistral-7B-v0.1_mbe_positive |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# Mistral-7B-v0.1_mbe_positive |
|
|
|
This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co./mistralai/Mistral-7B-v0.1) on the mbe dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.0233 |
|
- Accuracy: 0.6809 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 3e-05 |
|
- train_batch_size: 4 |
|
- eval_batch_size: 4 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 4 |
|
- total_train_batch_size: 16 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: constant |
|
- lr_scheduler_warmup_ratio: 0.03 |
|
- num_epochs: 5.0 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Accuracy | |
|
|:-------------:|:-----:|:----:|:---------------:|:--------:| |
|
| 2.3217 | 0.07 | 10 | 0.7263 | 0.4901 | |
|
| 0.56 | 0.13 | 20 | 0.6898 | 0.5526 | |
|
| 0.5281 | 0.2 | 30 | 0.6465 | 0.5888 | |
|
| 0.994 | 0.27 | 40 | 0.7351 | 0.5987 | |
|
| 0.4785 | 0.33 | 50 | 0.6004 | 0.6118 | |
|
| 0.4732 | 0.4 | 60 | 0.5783 | 0.6349 | |
|
| 0.4466 | 0.47 | 70 | 0.5714 | 0.6414 | |
|
| 0.8737 | 0.53 | 80 | 0.5673 | 0.6184 | |
|
| 0.4471 | 0.6 | 90 | 0.5631 | 0.6283 | |
|
| 0.46 | 0.67 | 100 | 0.5504 | 0.6349 | |
|
| 0.3294 | 0.73 | 110 | 0.6010 | 0.625 | |
|
| 0.6526 | 0.8 | 120 | 0.5731 | 0.6283 | |
|
| 0.3712 | 0.87 | 130 | 0.5379 | 0.6447 | |
|
| 0.3341 | 0.93 | 140 | 0.5409 | 0.6283 | |
|
| 0.552 | 1.0 | 150 | 0.5311 | 0.6382 | |
|
| 0.4681 | 1.07 | 160 | 0.5371 | 0.6414 | |
|
| 0.3119 | 1.14 | 170 | 0.6172 | 0.6283 | |
|
| 0.3082 | 1.2 | 180 | 0.5361 | 0.6513 | |
|
| 0.5217 | 1.27 | 190 | 0.5468 | 0.625 | |
|
| 0.3888 | 1.34 | 200 | 0.5891 | 0.6316 | |
|
| 0.2841 | 1.4 | 210 | 0.5429 | 0.6283 | |
|
| 0.2728 | 1.47 | 220 | 0.5247 | 0.6382 | |
|
| 0.5563 | 1.54 | 230 | 0.5004 | 0.6513 | |
|
| 0.2862 | 1.6 | 240 | 0.4741 | 0.6546 | |
|
| 0.2289 | 1.67 | 250 | 0.5441 | 0.6513 | |
|
| 0.2481 | 1.74 | 260 | 0.5171 | 0.6513 | |
|
| 0.329 | 1.8 | 270 | 0.5371 | 0.6546 | |
|
| 0.1741 | 1.87 | 280 | 0.5412 | 0.6678 | |
|
| 0.2888 | 1.94 | 290 | 0.5131 | 0.6711 | |
|
| 0.4157 | 2.0 | 300 | 0.4555 | 0.6447 | |
|
| 0.1982 | 2.07 | 310 | 0.5670 | 0.6612 | |
|
| 0.106 | 2.14 | 320 | 0.7943 | 0.6678 | |
|
| 0.1718 | 2.2 | 330 | 0.7496 | 0.6645 | |
|
| 0.214 | 2.27 | 340 | 0.6264 | 0.6842 | |
|
| 0.1571 | 2.34 | 350 | 0.6139 | 0.6316 | |
|
| 0.1432 | 2.4 | 360 | 0.6199 | 0.6842 | |
|
| 0.1038 | 2.47 | 370 | 0.6368 | 0.6974 | |
|
| 0.1728 | 2.54 | 380 | 0.7889 | 0.6678 | |
|
| 0.14 | 2.6 | 390 | 0.7952 | 0.6546 | |
|
| 0.1522 | 2.67 | 400 | 0.7745 | 0.6579 | |
|
| 0.1345 | 2.74 | 410 | 0.7231 | 0.6513 | |
|
| 0.1587 | 2.8 | 420 | 0.7154 | 0.6480 | |
|
| 0.1391 | 2.87 | 430 | 0.6923 | 0.6513 | |
|
| 0.129 | 2.94 | 440 | 0.6484 | 0.6711 | |
|
| 0.2092 | 3.01 | 450 | 0.5822 | 0.6743 | |
|
| 0.015 | 3.07 | 460 | 1.1217 | 0.6579 | |
|
| 0.051 | 3.14 | 470 | 1.5790 | 0.6480 | |
|
| 0.0999 | 3.21 | 480 | 1.5168 | 0.6678 | |
|
| 0.1776 | 3.27 | 490 | 1.2342 | 0.6875 | |
|
| 0.0612 | 3.34 | 500 | 1.0371 | 0.6974 | |
|
| 0.0858 | 3.41 | 510 | 1.0277 | 0.6776 | |
|
| 0.0316 | 3.47 | 520 | 1.0387 | 0.6809 | |
|
| 0.1899 | 3.54 | 530 | 0.8185 | 0.6908 | |
|
| 0.1517 | 3.61 | 540 | 0.7054 | 0.6842 | |
|
| 0.0324 | 3.67 | 550 | 0.8505 | 0.6842 | |
|
| 0.0646 | 3.74 | 560 | 1.0057 | 0.6612 | |
|
| 0.1038 | 3.81 | 570 | 1.0027 | 0.6645 | |
|
| 0.0844 | 3.87 | 580 | 0.9926 | 0.6513 | |
|
| 0.0986 | 3.94 | 590 | 0.9246 | 0.6579 | |
|
| 0.0627 | 4.01 | 600 | 0.8539 | 0.6546 | |
|
| 0.0513 | 4.07 | 610 | 0.9247 | 0.6513 | |
|
| 0.0484 | 4.14 | 620 | 1.1128 | 0.6546 | |
|
| 0.0244 | 4.21 | 630 | 1.2702 | 0.6480 | |
|
| 0.0672 | 4.27 | 640 | 1.7169 | 0.6414 | |
|
| 0.0824 | 4.34 | 650 | 1.6627 | 0.6414 | |
|
| 0.0068 | 4.41 | 660 | 1.3425 | 0.6349 | |
|
| 0.044 | 4.47 | 670 | 1.2208 | 0.6612 | |
|
| 0.0378 | 4.54 | 680 | 1.2891 | 0.6447 | |
|
| 0.0411 | 4.61 | 690 | 1.3528 | 0.6612 | |
|
| 0.0215 | 4.67 | 700 | 1.2606 | 0.6678 | |
|
| 0.0438 | 4.74 | 710 | 1.2515 | 0.6546 | |
|
| 0.0936 | 4.81 | 720 | 1.0858 | 0.6645 | |
|
| 0.0305 | 4.87 | 730 | 0.9839 | 0.6579 | |
|
| 0.0282 | 4.94 | 740 | 1.0233 | 0.6809 | |
|
|
|
|
|
### Framework versions |
|
|
|
- PEFT 0.7.1 |
|
- Transformers 4.37.2 |
|
- Pytorch 2.1.2+cu121 |
|
- Datasets 2.17.1 |
|
- Tokenizers 0.15.1 |
