Hugging Face's logo

snoels
/
FinGEITje-7B-dpo

Text Generation
PEFT
TensorBoard
Safetensors
Dutch
mistral
alignment-handbook
trl
dpo
Generated from Trainer
geitje
fingeitje
dutch
finance
conversational
4-bit precision
bitsandbytes
Model card Files Metrics Community
FinGEITje-7B-dpo / README.md
snoels's picture
snoels
Update README.md
9638104 verified
preview code
|
raw
history blame
3.61 kB
metadata
base_model: BramVanroy/GEITje-7B-ultra
datasets:
  - BramVanroy/ultra_feedback_dutch
library_name: peft
tags:
  - alignment-handbook
  - trl
  - dpo
  - generated_from_trainer
model-index:
  - name: FinGEITje-7B-dpo
    results: []
language:
  - nl

Visualize in Weights & Biases

FinGEITje-7B-dpo

This model is a fine-tuned version of /mnt/trained_models/fingeitje on the BramVanroy/ultra_feedback_dutch dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0279
  • Rewards/chosen: -3.8986
  • Rewards/rejected: -15.9713
  • Rewards/accuracies: 0.9836
  • Rewards/margins: 12.0727
  • Logps/rejected: -1952.6360
  • Logps/chosen: -789.0983
  • Logits/rejected: -1.7369
  • Logits/chosen: -1.8936

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-06
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 64
  • total_eval_batch_size: 4
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.1029 0.1327 100 0.1099 -1.8067 -5.3683 0.9679 3.5616 -892.3373 -579.9115 -2.4775 -2.3705
0.042 0.2654 200 0.0430 -3.5129 -10.6778 0.9828 7.1649 -1423.2883 -750.5289 -1.9744 -1.9895
0.0278 0.3981 300 0.0344 -3.7335 -13.5153 0.9828 9.7818 -1707.0360 -772.5893 -1.7454 -1.8191
0.0223 0.5308 400 0.0308 -3.6554 -13.7712 0.9858 10.1158 -1732.6289 -764.7831 -1.8020 -1.9184
0.0378 0.6635 500 0.0297 -4.0018 -16.3285 0.9851 12.3266 -1988.3542 -799.4221 -1.6924 -1.8650
0.0352 0.7962 600 0.0278 -3.8104 -15.6430 0.9836 11.8327 -1919.8119 -780.2752 -1.7437 -1.8978
0.0238 0.9289 700 0.0279 -3.8974 -15.9642 0.9828 12.0668 -1951.9310 -788.9780 -1.7371 -1.8937

Framework versions

  • PEFT 0.11.1
  • Transformers 4.42.4
  • Pytorch 2.3.1
  • Datasets 2.20.0
  • Tokenizers 0.19.1