Update README.md

README.md

@@ -8,4 +8,4 @@ tags:
   - fill-mask
 library_name: transformers
 ---
-This model is derived from the bert-base-uncased checkpoint by replacing the GELU with ReLU activation function and further pre-trained through several iterations to adapt it to the change of the activation function.
+This model was derived from the bert-base-uncased checkpoint by replacing the GELU with ReLU activation function and further pre-training through several iterations to adapt it to the change of the activation function.