cloudyu

/

Mixtral_7Bx4_MOE_DPO

like 1

fine-tune can improve the score again if the dataset is great.
https://huggingface.co./cloudyu/Pluto_24B_DPO_200/blob/main/dpo-metrics.jpg

Thanks for sharing, i would like to know if have you done the sft before doing dpo, and if yes on which dataset? if not, then could you tell me which comparison training you are using during dpo training ? thanks again for you sharing