metadata
library_name: transformers
license: mit
This repository contains the model release for the use of CPO, please find more details in our github!
@inproceedings{
xu2024contrastive,
title={Contrastive Preference Optimization: Pushing the Boundaries of {LLM} Performance in Machine Translation},
author={Haoran Xu and Amr Sharaf and Yunmo Chen and Weiting Tan and Lingfeng Shen and Benjamin Van Durme and Kenton Murray and Young Jin Kim},
booktitle={Forty-first International Conference on Machine Learning},
year={2024},
url={https://openreview.net/forum?id=51iwkioZpn}
}
Here are released models for CPO and SimPO. The code is based on SimPO github. We focus on highlighting reference-free preference learning and demonstrating the effectiveness of SimPO. Additionally, we integrate length normalization and target reward margin into CPO, showing promising results and the potential benefits of combining them together. CPO adds a BC-regularizer to prevent the model from deviating too much from the preferred data distribution.
| models | AE2 LC | AE2 WR | |
|---|---|---|---|
| Llama3 Instruct 8B SimPO (reported) | princeton-nlp/Llama-3-Instruct-8B-SimPO | 44.7 | 40.5 |
| Llama3 Instruct 8B SimPO (reproduced) | haoranxu/Llama-3-Instruct-8B-SimPO | 43.3 | 40.6 |
| Llama3 Instruct 8B CPO | haoranxu/Llama-3-Instruct-8B-CPO | 36.07 | 40.06 |
| Llama3 Instruct 8B CPO-SimPO | haoranxu/Llama-3-Instruct-8B-CPO-SimPO | 46.94 | 44.72 |