microsoft/DialogRPT-human-vs-machine

Demo

Please try this ➤➤➤ Colab Notebook Demo (click me!)

Context	Response	human_vs_machine score
I love NLP!	I'm not sure if it's a good idea.	0.000
I love NLP!	Me too!	0.605

The human_vs_machine score predicts how likely the response is from a human rather than a machine.

DialogRPT-human-vs-machine

Dialog Ranking Pretrained Transformers

How likely a dialog response is upvoted 👍 and/or gets replied 💬?

This is what DialogRPT is learned to predict. It is a set of dialog response ranking models proposed by Microsoft Research NLP Group trained on 100 + millions of human feedback data. It can be used to improve existing dialog generation model (e.g., DialoGPT) by re-ranking the generated response candidates.

Quick Links:

• EMNLP'20 Paper
• Dataset, training, and evaluation
• Colab Notebook Demo

We considered the following tasks and provided corresponding pretrained models.

Task	Description	Pretrained model
Human feedback	given a context and its two human responses, predict...
updown	... which gets more upvotes?	model card
width	... which gets more direct replies?	model card
depth	... which gets longer follow-up thread?	model card
Human-like (human vs fake)	given a context and one human response, distinguish it with...
human_vs_rand	... a random human response	model card
human_vs_machine	... a machine generated response	this model

Contact:

Please create an issue on our repo

Citation:

@inproceedings{gao2020dialogrpt,
    title={Dialogue Response RankingTraining with Large-Scale Human Feedback Data},
    author={Xiang Gao and Yizhe Zhang and Michel Galley and Chris Brockett and Bill Dolan},
    year={2020},
    booktitle={EMNLP}
}