---
license: apache-2.0
base_model: GerMedBERT/medbert-512
tags:
- generated_from_trainer
metrics:
- precision
- recall
- accuracy
model-index:
- name: GerMedBert_ATTR_V02_BRONCO
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# GerMedBert_ATTR_V02_BRONCO

This model is a fine-tuned version of [GerMedBERT/medbert-512](https://huggingface.co./GerMedBERT/medbert-512) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.0593
- F1 Score: 0.8187
- Precision: 0.8235
- Recall: 0.8140
- Accuracy: 0.8993
- Num Input Tokens Seen: 4204246

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 6

### Training results

| Training Loss | Epoch | Step | Validation Loss | F1 Score | Precision | Recall | Accuracy | Input Tokens Seen |
|:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:--------:|:-----------------:|
| No log        | 0.25  | 81   | 0.1949          | 0.0      | 1.0       | 0.0    | 0.7240   | 175856            |
| 0.206         | 0.5   | 162  | 0.1196          | 0.4123   | 0.8393    | 0.2733 | 0.7847   | 349280            |
| 0.206         | 0.75  | 243  | 0.0945          | 0.6296   | 0.8673    | 0.4942 | 0.8351   | 526096            |
| 0.1019        | 1.0   | 324  | 0.0792          | 0.7584   | 0.8       | 0.7209 | 0.8698   | 702407            |
| 0.1019        | 1.25  | 405  | 0.0697          | 0.7874   | 0.7784    | 0.7965 | 0.8785   | 877047            |
| 0.0715        | 1.5   | 486  | 0.0697          | 0.7778   | 0.7447    | 0.8140 | 0.8715   | 1052647           |
| 0.0715        | 1.75  | 567  | 0.0673          | 0.7568   | 0.7826    | 0.7326 | 0.8715   | 1227287           |
| 0.0638        | 2.0   | 648  | 0.0680          | 0.7781   | 0.7358    | 0.8256 | 0.8715   | 1400330           |
| 0.0638        | 2.25  | 729  | 0.0622          | 0.7965   | 0.8084    | 0.7849 | 0.8941   | 1577786           |
| 0.0445        | 2.5   | 810  | 0.0593          | 0.8012   | 0.7943    | 0.8081 | 0.8906   | 1751722           |
| 0.0445        | 2.75  | 891  | 0.0583          | 0.8023   | 0.8023    | 0.8023 | 0.8906   | 1927258           |
| 0.0396        | 3.0   | 972  | 0.0579          | 0.8161   | 0.8068    | 0.8256 | 0.8993   | 2100749           |
| 0.0396        | 3.25  | 1053 | 0.0598          | 0.8125   | 0.7944    | 0.8314 | 0.8941   | 2276989           |
| 0.0289        | 3.5   | 1134 | 0.0592          | 0.8036   | 0.8232    | 0.7849 | 0.8941   | 2451501           |
| 0.0289        | 3.75  | 1215 | 0.0585          | 0.7954   | 0.7886    | 0.8023 | 0.8906   | 2628573           |
| 0.0271        | 4.0   | 1296 | 0.0571          | 0.8171   | 0.8034    | 0.8314 | 0.8993   | 2802576           |
| 0.0271        | 4.25  | 1377 | 0.0581          | 0.8235   | 0.8333    | 0.8140 | 0.9045   | 2978368           |
| 0.0194        | 4.5   | 1458 | 0.0619          | 0.7978   | 0.7717    | 0.8256 | 0.8837   | 3154544           |
| 0.0194        | 4.75  | 1539 | 0.0612          | 0.8048   | 0.8323    | 0.7791 | 0.8958   | 3330400           |
| 0.0193        | 5.0   | 1620 | 0.0585          | 0.8059   | 0.8155    | 0.7965 | 0.8958   | 3505555           |
| 0.0193        | 5.25  | 1701 | 0.0587          | 0.8187   | 0.8235    | 0.8140 | 0.9010   | 3680771           |
| 0.0153        | 5.5   | 1782 | 0.0592          | 0.8242   | 0.8171    | 0.8314 | 0.9010   | 3856947           |
| 0.0153        | 5.75  | 1863 | 0.0592          | 0.8163   | 0.8187    | 0.8140 | 0.8993   | 4030371           |
| 0.0146        | 6.0   | 1944 | 0.0593          | 0.8187   | 0.8235    | 0.8140 | 0.8993   | 4204246           |


### Framework versions

- Transformers 4.40.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.0
- Tokenizers 0.19.1