allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm

Text Generation • Updated Jun 14 • 366 • 6

Note Our overall best model, a 13B Tulu 2 model trained using PPO with a 70B reward model trained on UltraFeedback! We also release the value and reward models associated with this model - see the model card for details

Note The datasets used for training PPO, DPO, and reward models in our paper.

Note The prompt sets used during PPO training in our paper. Below, see all our PPO-trained models!

allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm-mixed-prompts

Text Generation • Updated Jun 14 • 22

Note Below is our PPO data ablations.

allenai/tulu-v2.5-ppo-13b-nectar-60k

Text Generation • Updated Jun 14 • 43

Note Below is our DPO data ablations.

allenai/tulu-v2.5-dpo-13b-nectar-60k

Text Generation • Updated Jun 14 • 48 • 1

Note Below are our reward models!

allenai/tulu-v2.5-13b-stackexchange-60k-rm

Text Classification • Updated Jun 14 • 21

Note Below are our value models.

allenai/tulu-v2.5-ppo-13b-uf-mean-70b-uf-rm-mixed-prompts-value

Token Classification • Updated Jun 14 • 22

Note Below is llama 3 models: