Huertas97
commited on
Commit
•
5b95be3
1
Parent(s):
5329fcf
README avg results
Browse files
README.md
CHANGED
@@ -79,7 +79,19 @@ print(sentence_embeddings)
|
|
79 |
## Evaluation Results
|
80 |
|
81 |
<!--- Describe how your model was evaluated -->
|
82 |
-
Check the test results in the Semantic Textual Similarity Tasks. The 15 languages available at the [Multilingual STSB](https://github.com/Huertas97/Multilingual-STSB) have been combined into monolingual and cross-lingual tasks, giving a total of 31 tasks. Monolingual tasks have both sentences from the same language source (e.g., Ar-Ar, Es-Es), while cross-lingual tasks have two sentences, each in a different language being one of them English (e.g., en-ar, en-es).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
83 |
|
84 |
| Monolingual Task | Pearson Cosine test | Spearman Cosine test |
|
85 |
|------------------|---------------------|-----------------------|
|
|
|
79 |
## Evaluation Results
|
80 |
|
81 |
<!--- Describe how your model was evaluated -->
|
82 |
+
Check the test results in the Semantic Textual Similarity Tasks. The 15 languages available at the [Multilingual STSB](https://github.com/Huertas97/Multilingual-STSB) have been combined into monolingual and cross-lingual tasks, giving a total of 31 tasks. Monolingual tasks have both sentences from the same language source (e.g., Ar-Ar, Es-Es), while cross-lingual tasks have two sentences, each in a different language being one of them English (e.g., en-ar, en-es).
|
83 |
+
|
84 |
+
|
85 |
+
Here we compare the average multilingual semantic textual similairty capabilities between the `paraphrase-multilingual-mpnet-base-v2` based model and the `mstsb-paraphrase-multilingual-mpnet-base-v2` fine-tuned model across the 31 tasks. It is worth noting that both models are multilingual, but the second model is adjusted with multilingual data for semantic similarity. The average of correlation coefficients is computed by transforming each correlation coefficient to a Fisher's z value, averaging them, and then back-transforming to a correlation coefficient.
|
86 |
+
|
87 |
+
|
88 |
+
| Model | Average Spearman Cosine Test |
|
89 |
+
|---------------------------------------------|------------------------------|
|
90 |
+
| mstsb-paraphrase-multilingual-mpnet-base-v2 | 0.835890 |
|
91 |
+
| paraphrase-multilingual-mpnet-base-v2 | 0.818896 |
|
92 |
+
|
93 |
+
|
94 |
+
For the sake of readability tasks have been splitted into monolingual and cross-lingual tasks.
|
95 |
|
96 |
| Monolingual Task | Pearson Cosine test | Spearman Cosine test |
|
97 |
|------------------|---------------------|-----------------------|
|