---
base_model: MBZUAI/swiftformer-xs
tags:
- generated_from_trainer
metrics:
- accuracy
model-index:
- name: swiftformer-xs-dmae-va-U5-42C
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# swiftformer-xs-dmae-va-U5-42C

This model is a fine-tuned version of [MBZUAI/swiftformer-xs](https://huggingface.co./MBZUAI/swiftformer-xs) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6744
- Accuracy: 0.8167

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- gradient_accumulation_steps: 4
- total_train_batch_size: 128
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 80

### Training results

| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:--------:|
| No log        | 0.9   | 7    | 1.3856          | 0.4667   |
| 1.3855        | 1.94  | 15   | 1.3819          | 0.4833   |
| 1.3855        | 2.97  | 23   | 1.3687          | 0.4333   |
| 1.3742        | 4.0   | 31   | 1.3189          | 0.3167   |
| 1.3004        | 4.9   | 38   | 1.2501          | 0.4833   |
| 1.3004        | 5.94  | 46   | 1.2268          | 0.4833   |
| 1.1716        | 6.97  | 54   | 1.2115          | 0.5      |
| 1.0686        | 8.0   | 62   | 1.2243          | 0.5333   |
| 1.0686        | 8.9   | 69   | 1.1432          | 0.55     |
| 0.9764        | 9.94  | 77   | 1.0205          | 0.55     |
| 0.873         | 10.97 | 85   | 0.9721          | 0.6      |
| 0.873         | 12.0  | 93   | 0.9221          | 0.5667   |
| 0.7822        | 12.9  | 100  | 0.8593          | 0.6167   |
| 0.664         | 13.94 | 108  | 0.7775          | 0.7      |
| 0.664         | 14.97 | 116  | 0.8117          | 0.6167   |
| 0.5439        | 16.0  | 124  | 0.7553          | 0.6833   |
| 0.5439        | 16.9  | 131  | 0.6697          | 0.7167   |
| 0.496         | 17.94 | 139  | 0.6480          | 0.7333   |
| 0.4563        | 18.97 | 147  | 0.7115          | 0.7333   |
| 0.4563        | 20.0  | 155  | 0.6777          | 0.7167   |
| 0.3831        | 20.9  | 162  | 0.6416          | 0.7667   |
| 0.339         | 21.94 | 170  | 0.7040          | 0.7      |
| 0.339         | 22.97 | 178  | 0.6859          | 0.7167   |
| 0.3033        | 24.0  | 186  | 0.6012          | 0.7      |
| 0.2655        | 24.9  | 193  | 0.5440          | 0.7833   |
| 0.2655        | 25.94 | 201  | 0.6174          | 0.75     |
| 0.2269        | 26.97 | 209  | 0.5745          | 0.7333   |
| 0.2472        | 28.0  | 217  | 0.5688          | 0.8      |
| 0.2472        | 28.9  | 224  | 0.6578          | 0.75     |
| 0.2004        | 29.94 | 232  | 0.5811          | 0.7833   |
| 0.2099        | 30.97 | 240  | 0.6672          | 0.75     |
| 0.2099        | 32.0  | 248  | 0.5927          | 0.75     |
| 0.1834        | 32.9  | 255  | 0.6193          | 0.7667   |
| 0.1834        | 33.94 | 263  | 0.7505          | 0.7167   |
| 0.2248        | 34.97 | 271  | 0.7730          | 0.7333   |
| 0.1571        | 36.0  | 279  | 0.6211          | 0.7333   |
| 0.1571        | 36.9  | 286  | 0.6228          | 0.7167   |
| 0.1983        | 37.94 | 294  | 0.6088          | 0.7167   |
| 0.1629        | 38.97 | 302  | 0.7009          | 0.75     |
| 0.1629        | 40.0  | 310  | 0.7285          | 0.75     |
| 0.1547        | 40.9  | 317  | 0.6401          | 0.7667   |
| 0.1548        | 41.94 | 325  | 0.6123          | 0.7833   |
| 0.1548        | 42.97 | 333  | 0.6317          | 0.8      |
| 0.1566        | 44.0  | 341  | 0.7579          | 0.7167   |
| 0.1361        | 44.9  | 348  | 0.6653          | 0.7167   |
| 0.1361        | 45.94 | 356  | 0.7401          | 0.7333   |
| 0.1273        | 46.97 | 364  | 0.8404          | 0.7333   |
| 0.1312        | 48.0  | 372  | 0.8388          | 0.75     |
| 0.1312        | 48.9  | 379  | 0.7823          | 0.7667   |
| 0.1307        | 49.94 | 387  | 0.6980          | 0.7167   |
| 0.1307        | 50.97 | 395  | 0.7589          | 0.75     |
| 0.1061        | 52.0  | 403  | 0.6644          | 0.7333   |
| 0.1186        | 52.9  | 410  | 0.7057          | 0.7333   |
| 0.1186        | 53.94 | 418  | 0.6744          | 0.8167   |
| 0.1108        | 54.97 | 426  | 0.6328          | 0.7667   |
| 0.1014        | 56.0  | 434  | 0.6402          | 0.7833   |
| 0.1014        | 56.9  | 441  | 0.6631          | 0.75     |
| 0.1082        | 57.94 | 449  | 0.7001          | 0.7333   |
| 0.1118        | 58.97 | 457  | 0.7898          | 0.7333   |
| 0.1118        | 60.0  | 465  | 0.7644          | 0.7333   |
| 0.1051        | 60.9  | 472  | 0.7767          | 0.7333   |
| 0.0979        | 61.94 | 480  | 0.7440          | 0.7333   |
| 0.0979        | 62.97 | 488  | 0.6827          | 0.7333   |
| 0.0834        | 64.0  | 496  | 0.7008          | 0.7333   |
| 0.0834        | 64.9  | 503  | 0.7243          | 0.7167   |
| 0.0963        | 65.94 | 511  | 0.7656          | 0.7333   |
| 0.0989        | 66.97 | 519  | 0.7332          | 0.7333   |
| 0.0989        | 68.0  | 527  | 0.7624          | 0.7333   |
| 0.107         | 68.9  | 534  | 0.7292          | 0.75     |
| 0.0987        | 69.94 | 542  | 0.7169          | 0.7333   |
| 0.0987        | 70.97 | 550  | 0.7462          | 0.7333   |
| 0.0956        | 72.0  | 558  | 0.6656          | 0.75     |
| 0.0956        | 72.26 | 560  | 0.6873          | 0.7333   |


### Framework versions

- Transformers 4.36.2
- Pytorch 2.1.2+cu118
- Datasets 2.16.1
- Tokenizers 0.15.0