In building systems we often need to compare the performance of two systems. How can we know if the new system we just built is better than our old one? Or better than the some other system described in the literature? This is the domain of statistical hypothesis testing, and in this section we introduce tests for statistical significance for NLP classifiers, drawing especially on the work of Dror et al. 2020 such as F 1 , or accuracy. Perhaps we want to know if our logistic regression sentiment classifier A (Chapter 5) gets a higher F 1 score than our naive Bayes sentiment classifier B on a particular test set x. Let's call M(A, x) the score that system A gets on test set x, and δ (x) the performance difference between A and B on x: