In BERT, 15% of the input tokens in a training sequence are sampled for learning. Of these, 80% are replaced with [MASK] , 10% are replaced with randomly selected tokens, and the remaining 10% are left unchanged.