Update README.md
Browse files
README.md
CHANGED
|
@@ -4,12 +4,13 @@ language: de
|
|
| 4 |
|
| 5 |
### Welcome to ParlBERT-German!
|
| 6 |
|
| 7 |
-
🏷 **Model description
|
| 8 |
|
| 9 |
**ParlBERT-German** is a domain-specific language model. The model was created through a process of continuous pre-training, which involved using a generic German language model (GermanBERT) as the foundation and further enhancing it with domain-specific knowledge. We used [DeuParl](https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/2889?show=full) as the domain-specific dataset for continuous pre-training, which provided **ParlBERT-German** with an better understanding of the language and context used in parliamentary debates. The result is a specialized language model that can be used in related scenarios.
|
| 10 |
|
| 11 |
|
| 12 |
🤖 **Model training**
|
|
|
|
| 13 |
During the model training process, a masked language modeling approach was used with a token masking probability of 15\%. The training was performed for a single epoch, which means that the entire dataset was passed through the model once during the training process.
|
| 14 |
|
| 15 |
⚠️ **Limitations**
|
|
|
|
| 4 |
|
| 5 |
### Welcome to ParlBERT-German!
|
| 6 |
|
| 7 |
+
🏷 **Model description**:
|
| 8 |
|
| 9 |
**ParlBERT-German** is a domain-specific language model. The model was created through a process of continuous pre-training, which involved using a generic German language model (GermanBERT) as the foundation and further enhancing it with domain-specific knowledge. We used [DeuParl](https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/2889?show=full) as the domain-specific dataset for continuous pre-training, which provided **ParlBERT-German** with an better understanding of the language and context used in parliamentary debates. The result is a specialized language model that can be used in related scenarios.
|
| 10 |
|
| 11 |
|
| 12 |
🤖 **Model training**
|
| 13 |
+
|
| 14 |
During the model training process, a masked language modeling approach was used with a token masking probability of 15\%. The training was performed for a single epoch, which means that the entire dataset was passed through the model once during the training process.
|
| 15 |
|
| 16 |
⚠️ **Limitations**
|