lomahony's picture
Update README.md
8f97744 verified
metadata
language:
  - en
tags:
  - pytorch
  - causal-lm
  - pythia
license: apache-2.0
datasets:
  - Anthropic/hh-rlhf

Pythia-1.4b supervised finetuned using TRLx library with the helpful subset of Anthropic-hh-rlhf dataset for 1 epoch.

Checkpoints are also uploaded.

Fully reproducible finetuning code is available on GitHub

wandb log

See Pythia-1.4b for model details (paper).

See further details of these models in the paper Attributing Mode Collapse in the Fine-Tuning of Large Language Models.

You can cite these models if they are helpful as follows:

@inproceedings{o2024attributing,
  title={Attributing Mode Collapse in the Fine-Tuning of Large Language Models},
  author={O’Mahony, Laura and Grinsztajn, Leo and Schoelkopf, Hailey and Biderman, Stella},
  booktitle={ICLR 2024, Mathematical and Empirical Understanding of Foundation Models (ME-FoMo) workshop},
  year={2024}
}

hf (pretrained=lomahony/pythia-1.4b-helpful-sft), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 16

Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 0 acc 0.2679 ± 0.0129
none 0 acc_norm 0.2978 ± 0.0134
arc_easy 1 none 0 acc 0.6120 ± 0.0100
none 0 acc_norm 0.5282 ± 0.0102
boolq 2 none 0 acc 0.6260 ± 0.0085
hellaswag 1 none 0 acc 0.4097 ± 0.0049
none 0 acc_norm 0.5212 ± 0.0050
lambada_openai 1 none 0 perplexity 6.4836 ± 0.1838
none 0 acc 0.5789 ± 0.0069
openbookqa 1 none 0 acc 0.2120 ± 0.0183
none 0 acc_norm 0.3340 ± 0.0211
piqa 1 none 0 acc 0.7100 ± 0.0106
none 0 acc_norm 0.7144 ± 0.0105
sciq 1 none 0 acc 0.8540 ± 0.0112
none 0 acc_norm 0.7830 ± 0.0130
wikitext 2 none 0 word_perplexity 15.8394 ± N/A
none 0 byte_perplexity 1.6763 ± N/A
none 0 bits_per_byte 0.7453 ± N/A
winogrande 1 none 0 acc 0.5872 ± 0.0138

hf (pretrained=lomahony/pythia-1.4b-helpful-sft), gen_kwargs: (None), limit: None, num_fewshot: 5, batch_size: 16

Tasks Version Filter n-shot Metric Value Stderr
arc_challenge 1 none 5 acc 0.2892 ± 0.0133
none 5 acc_norm 0.3097 ± 0.0135
arc_easy 1 none 5 acc 0.6444 ± 0.0098
none 5 acc_norm 0.6309 ± 0.0099
boolq 2 none 5 acc 0.6333 ± 0.0084
hellaswag 1 none 5 acc 0.4065 ± 0.0049
none 5 acc_norm 0.5215 ± 0.0050
lambada_openai 1 none 5 perplexity 9.7040 ± 0.2887
none 5 acc 0.4951 ± 0.0070
openbookqa 1 none 5 acc 0.2220 ± 0.0186
none 5 acc_norm 0.3100 ± 0.0207
piqa 1 none 5 acc 0.7029 ± 0.0107
none 5 acc_norm 0.7127 ± 0.0106
sciq 1 none 5 acc 0.9170 ± 0.0087
none 5 acc_norm 0.9160 ± 0.0088
wikitext 2 none 5 word_perplexity 15.8394 ± N/A
none 5 byte_perplexity 1.6763 ± N/A
none 5 bits_per_byte 0.7453 ± N/A
winogrande 1 none 5 acc 0.5699 ± 0.0139