Commit
•
5cba176
1
Parent(s):
ca91748
Update README.md
Browse files
README.md
CHANGED
|
@@ -23,7 +23,7 @@ The details are available at [Github:FS-ABSA](https://github.com/nustm/fs-absa)
|
|
| 23 |
|
| 24 |
To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `restaurant` in this repo), we conducted *domain-adaptive pre-training*,
|
| 25 |
i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `restaurant`) with the *text-infilling objective*
|
| 26 |
-
(corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Yelp for the restaurant domain
|
| 27 |
For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 80 and a learning rate of 1e-4.
|
| 28 |
|
| 29 |
Our model can be seen as an enhanced T5 model in the restaurant domain, which can be used for various NLP tasks related to the restaurant domain,
|
|
|
|
| 23 |
|
| 24 |
To bridge the domain gap between general pre-training and the task of interest in a specific domain (i.e., `restaurant` in this repo), we conducted *domain-adaptive pre-training*,
|
| 25 |
i.e., continuing pre-training the language model (i.e., T5) on the unlabeled corpus of the domain of interest (i.e., `restaurant`) with the *text-infilling objective*
|
| 26 |
+
(corruption rate of 15% and average span length of 1). We collect relevant 100k unlabeled reviews from Yelp for the restaurant domain.
|
| 27 |
For pre-training, we employ the [Adafactor](https://arxiv.org/abs/1804.04235) optimizer with a batch size of 80 and a learning rate of 1e-4.
|
| 28 |
|
| 29 |
Our model can be seen as an enhanced T5 model in the restaurant domain, which can be used for various NLP tasks related to the restaurant domain,
|