---
license: apache-2.0
tags:
- generated_from_trainer
datasets:
- banking77
metrics:
- f1
model-index:
- name: bert-base-banking77-pt2
  results:
  - task:
      type: text-classification
      name: Text Classification
    dataset:
      name: banking77
      type: banking77
      config: default
      split: test
      args: default
    metrics:
    - type: f1
      value: 0.9311734343722707
      name: F1
    - type: accuracy
      value: 0.9308441558441558
      name: Accuracy
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWI2NWQ5YmM2NTM0ZmRkNWNmNTEwNWEzZTk4NmYzZGI1NDdiMmMwYjQ3MDdhYjM1NjQzMGY0MTNjY2VhYzVlMiIsInZlcnNpb24iOjF9.TfCsa8zqx0Y4L_w3P39VeVHiZC5voU7Fgnrc_tLErWuXNkLyDq-bUML35f8vXMQg-AJjk4bqTYH3VAvKBZ7JAA
    - type: f1
      value: 0.9308950029410715
      name: F1 Macro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiODdkOWViMGU4NmQyMGQ4NjczMzI0OWYwY2VmN2EyOTZlNjVlYzMxYjkxOTI1NGI2OTYzY2UzN2I5OTQ2NzM3MiIsInZlcnNpb24iOjF9.P0XNL76QzZ7F9r6yDDpLX-D0Gbl6O5aow6_A-uytnbMFQZTo7NyUZuCBvh0VfG5x6QMCL4X4Crqn1NJTZxzqAA
    - type: f1
      value: 0.9308441558441558
      name: F1 Micro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOWNlMzc2Yjc2MWY5OGUxMTYwZGY5ZTk3MTEzMjBhMmFiZWM4MmJmNjc3MDY3ZjA1NTdkYTNiMTg5OWU0ODNmYiIsInZlcnNpb24iOjF9.pgOX-MmThMpEwFKrrGvqTBg0hkwWSMpF2HMXrR_brfPLKSFO77P6F7FAOi6aj-Wd_3Y1Kr0eEuwFTVpZebZsAg
    - type: f1
      value: 0.9308950029410716
      name: F1 Weighted
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGNlMDRlOTc3YzBjN2IyZDE2NzZhYjMyZWIzYjFkOTY5N2VjZDY1MjIxYzc2OTAwNDRjZjljZjk3NmRmODViYiIsInZlcnNpb24iOjF9._MqnpQPUGwGm1qLnNI5v28X7E-NU5poQw8624VCUGhucjyj4HbQPXQyN07TxL6CMkZjhweA6EUTpu8whIuteAQ
    - type: precision
      value: 0.9335975349016483
      name: Precision Macro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiNGJmYzYwOGY5NDFkYTk1ZmU2YTAzNWJmNDFhOTcxYjkyNGUyMzgzNjg3NDhiY2ExMGE0N2FkOGZjNDE4MzAwNCIsInZlcnNpb24iOjF9.d1ON4n0wz8ac4SEsSzahXfp-p72kBRXpqfZ2BeTHmW-Fz_LW6ORe3cC6XXIabS8Odtl4oPSBJj6cVkfjAFI5AA
    - type: precision
      value: 0.9308441558441558
      name: Precision Micro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYTc4M2ExMDQ4YWZmZmI1Mzg2NTIwY2RkOTUyODA4OGIyYTdjNzA5OGI2MGZhOWNiN2VhYzc4ZmRiMmYyNTNiZCIsInZlcnNpb24iOjF9.Y-BeyFh6Eg_b0Im1T9PoM-MKxZkr_236iYlMLFISY2K5pnYRa2DNvJdy77kWUaCqXNzF3P6gCxx5fq6zPdO6BA
    - type: precision
      value: 0.9335975349016484
      name: Precision Weighted
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiYWI3NDAyZjg5ZDJmODg3MTczZDcwNzRiZTA3NzY0YmNmMzg3ZWQ1MTEwMDYwOWY2NGMxMWI4ODhhMzU3OTgwMCIsInZlcnNpb24iOjF9.QeDOz4wAW_MFyY9iMAcX6uBWjXA2Pvtnw4SXOKXaqrhrK0i4PFdICSaL5BwEphRvzS9kkg7KXKFd0NCxG_XCDQ
    - type: recall
      value: 0.930844155844156
      name: Recall Macro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiOGMxNWM0N2VhOTQ5OTg5ODUyNWNjZjQwNmY4MTdjNWM2ZGVlMDkzYjlkYzhlY2U3ZDUxODNjMzMxNzVhOGNhZiIsInZlcnNpb24iOjF9.mj-4bcIZfa-rQl_oNHBY5h7EPPhdNZnbtQILToHZX8Q48X0pr2J2tYufzF9IYo5ALp_SKFE4TBoNKsVxXG3HDQ
    - type: recall
      value: 0.9308441558441558
      name: Recall Micro
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDU2MzQ3ZTEyZDQ4ZjVkM2FhZGE4MTYxOTMwOTZmNGEyZWQ1Mjc1NjMwODNmZmU3MGFhMGIwMmYzODMyNmUzZiIsInZlcnNpb24iOjF9.EU4rTzibAgbs1lb7ScKJ_qn8DTkUmd9KBCy16a8QClDU0Dg_NmiEnyJZ_7q9joepzQiFw1pfJmVwY6ETF_IYBw
    - type: recall
      value: 0.9308441558441558
      name: Recall Weighted
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZjFhZmY3ZmUwZDAyMzEzZjhlNjAwMDhjZTAzNmVmOTQ3NjVkZTY5YzU3ZTBlMjQzMDQxOGRlY2ZiMjFhNmVkNSIsInZlcnNpb24iOjF9.20WZVIETmsV5ervv4GLiNLaf0SWInHjGW9v7vBVkOFccm98hNHGiCpbSBDqWHGvXGckuh1B6bpdp0wkPr45YBQ
    - type: loss
      value: 0.28434908390045166
      name: loss
      verified: true
      verifyToken: eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJoYXNoIjoiZDZjMDU3ZWQ0ZWNhYjE3Mjg4NGMxM2Q5ODhlYWE4ZTc1YmExYTExMmVkNmQ4NGJhZGM3N2Y0NzI0YjQ5NmI0OSIsInZlcnNpb24iOjF9.wB9AEZE-lQm_EE9411lfOz-xRogbIqbY7_QedNLIppP5bhE_NosSqS930YA8lWhFguDxJsUJmfy3jIWC5QWOBA
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bert-base-banking77-pt2

This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co./bert-base-uncased) on the banking77 dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2844
- F1: 0.9312

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1     |
|:-------------:|:-----:|:----:|:---------------:|:------:|
| 3.2215        | 1.0   | 313  | 1.1811          | 0.7646 |
| 0.6252        | 2.0   | 626  | 0.4665          | 0.9120 |
| 0.3323        | 3.0   | 939  | 0.3294          | 0.9281 |
| 0.1446        | 4.0   | 1252 | 0.3051          | 0.9267 |
| 0.0994        | 5.0   | 1565 | 0.2844          | 0.9312 |


### Framework versions

- Transformers 4.27.1
- Pytorch 2.0.1
- Datasets 2.9.0
- Tokenizers 0.13.3