Update README.md
Browse files
README.md
CHANGED
@@ -36,15 +36,15 @@ Retrieval:
|
|
36 |
|
37 |
This is adapted and truncated (to keep only the most popular models) from [oshizo's benchmarking github repo](https://github.com/oshizo/JapaneseEmbeddingEval), please check it out for more information and give it a star as it was very useful!
|
38 |
|
39 |
-
Italic denotes best model for its size (base/large | 768/1024), bold denotes best overall.
|
40 |
|
41 |
| Model | JSTS valid-v1.1 | JSICK test | MIRACL dev | Average |
|
42 |
|-------------------------------------------------|-----------------|------------|------------|---------|
|
43 |
| bclavie/fio-base-japanese-v0.1 | **_0.863_** | **_0.894_** | 0.715 | _0.824_ |
|
44 |
| cl-nagoya/sup-simcse-ja-base | 0.809 | 0.827 | 0.527 | 0.721 |
|
45 |
-
| cl-nagoya/sup-simcse-ja-large |
|
46 |
| colorfulscoop/sbert-base-ja | 0.742 | 0.657 | 0.254 | 0.551 |
|
47 |
-
| intfloat/multilingual-e5-base | 0.796 | 0.806 |
|
48 |
| intfloat/multilingual-e5-large | 0.819 | 0.794 | **0.883** | **_0.832_** |
|
49 |
| pkshatech/GLuCoSE-base-ja | 0.818 | 0.757 | 0.692 | 0.755 |
|
50 |
| text-embedding-ada-002 | 0.790 | 0.789 | 0.7232 | 0.768 |
|
|
|
36 |
|
37 |
This is adapted and truncated (to keep only the most popular models) from [oshizo's benchmarking github repo](https://github.com/oshizo/JapaneseEmbeddingEval), please check it out for more information and give it a star as it was very useful!
|
38 |
|
39 |
+
Italic denotes best model for its size when a smaller model outperforms a bigger one (base/large | 768/1024), bold denotes best overall.
|
40 |
|
41 |
| Model | JSTS valid-v1.1 | JSICK test | MIRACL dev | Average |
|
42 |
|-------------------------------------------------|-----------------|------------|------------|---------|
|
43 |
| bclavie/fio-base-japanese-v0.1 | **_0.863_** | **_0.894_** | 0.715 | _0.824_ |
|
44 |
| cl-nagoya/sup-simcse-ja-base | 0.809 | 0.827 | 0.527 | 0.721 |
|
45 |
+
| cl-nagoya/sup-simcse-ja-large | _0.831_ | _0.831_ | 0.507 | 0.723 |
|
46 |
| colorfulscoop/sbert-base-ja | 0.742 | 0.657 | 0.254 | 0.551 |
|
47 |
+
| intfloat/multilingual-e5-base | 0.796 | 0.806 | __0.845__ | 0.816 |
|
48 |
| intfloat/multilingual-e5-large | 0.819 | 0.794 | **0.883** | **_0.832_** |
|
49 |
| pkshatech/GLuCoSE-base-ja | 0.818 | 0.757 | 0.692 | 0.755 |
|
50 |
| text-embedding-ada-002 | 0.790 | 0.789 | 0.7232 | 0.768 |
|