For example, in some situations we might have datasets that have human assessments of translation quality. Such datasets consists of tuples (x,x, r), where x = (x 1 , . . . , x n ) is a reference translation,x = (x 1 , . . . ,x m ) is a candidate machine translation, and r ∈ R is a human rating that expresses the quality ofx with respect to x. Given such data, algorithms like COMET (Rei et al., 2020) BLEURT (Sellam et al., 2020) train a predictor on the human-labeled datasets, for example by passing x andx through a version of BERT (trained with extra pretraining, and then finetuned on the human-labeled sentences), followed by a linear layer that is trained to predict r. The output of such models correlates highly with human labels.