Evaluation results for hadifar/tqa_qg_t5 model as a base model for other tasks
#1
by
eladven
- opened
README.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# hadifar/tqa_qg_t5 model
|
| 2 |
+
This model is based on t5-base pretrained model.
|
| 3 |
+
|
| 4 |
+
|
| 5 |
+
## Model Recycling
|
| 6 |
+
|
| 7 |
+
[Evaluation on 36 datasets](https://ibm.github.io/model-recycling/model_gain_chart?avg=2.23&mnli_lp=nan&20_newsgroup=7.08&ag_news=-18.15&amazon_reviews_multi=19.06&anli=1.48&boolq=21.14&cb=13.39&cola=1.12&copa=6.23&dbpedia=9.43&esnli=-41.19&financial_phrasebank=-39.30&imdb=-3.77&isear=21.34&mnli=8.51&mrpc=-15.91&multirc=33.87&poem_sentiment=18.40&qnli=-10.96&qqp=0.50&rotten_tomatoes=-32.27&rte=-24.39&sst2=-27.36&sst_5bins=38.14&stsb=6.69&trec_coarse=-14.52&trec_fine=5.67&tweet_ev_emoji=37.17&tweet_ev_emotion=-26.95&tweet_ev_hate=23.16&tweet_ev_irony=10.33&tweet_ev_offensive=-13.39&tweet_ev_sentiment=15.58&wic=-15.24&wnli=35.35&wsc=23.60&yahoo_answers=6.41&model_name=hadifar%2Ftqa_qg_t5&base_name=t5-base) using hadifar/tqa_qg_t5 as a base model yields average score of 77.68 in comparison to 75.45 by t5-base.
|
| 8 |
+
|
| 9 |
+
The model is ranked 3rd among all tested models for the t5-base architecture as of 21/12/2022
|
| 10 |
+
Results:
|
| 11 |
+
|
| 12 |
+
| 20_newsgroup | ag_news | amazon_reviews_multi | anli | boolq | cb | cola | copa | dbpedia | esnli | financial_phrasebank | imdb | isear | mnli | mrpc | multirc | poem_sentiment | qnli | qqp | rotten_tomatoes | rte | sst2 | sst_5bins | stsb | trec_coarse | trec_fine | tweet_ev_emoji | tweet_ev_emotion | tweet_ev_hate | tweet_ev_irony | tweet_ev_offensive | tweet_ev_sentiment | wic | wnli | wsc | yahoo_answers |
|
| 13 |
+
|---------------:|----------:|-----------------------:|--------:|--------:|--------:|--------:|--------:|----------:|--------:|-----------------------:|--------:|--------:|--------:|--------:|----------:|-----------------:|--------:|--------:|------------------:|--------:|-------:|------------:|-------:|--------------:|------------:|-----------------:|-------------------:|----------------:|-----------------:|---------------------:|---------------------:|--------:|--------:|--------:|----------------:|
|
| 14 |
+
| 92.2 | 71.2716 | 85.6 | 48.5312 | 97.8 | 88.9306 | 83.0297 | 55.8824 | 85.8338 | 48.5312 | 46.008 | 88.5553 | 92.6231 | 92.3119 | 69.7492 | 94.1514 | 92.82 | 79.4224 | 89.4333 | 56.3348 | 49.2958 | 66.48 | 93.6927 | 92 | 82.6923 | 98 | 82.0549 | 52.5589 | 75.8929 | 84.0698 | 70.6447 | 85.7913 | 51.9529 | 90.7064 | 83.6047 | 78 |
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
For more information, see: [Model Recycling](https://ibm.github.io/model-recycling/)
|