avsolatorio commited on
Commit
9b911f6
1 Parent(s): 3df2a88

Upload large model

Browse files

Signed-off-by: Aivin V. Solatorio <[email protected]>

1_Pooling/config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 1024,
3
+ "pooling_mode_cls_token": true,
4
+ "pooling_mode_mean_tokens": false,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false
7
+ }
README.md CHANGED
@@ -1,3 +1,2685 @@
1
  ---
 
 
 
2
  license: mit
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ library_name: sentence-transformers
5
  license: mit
6
+ pipeline_tag: sentence-similarity
7
+ tags:
8
+ - feature-extraction
9
+ - mteb
10
+ - sentence-similarity
11
+ - sentence-transformers
12
+
13
+ model-index:
14
+ - name: GIST-large-Embedding-v0
15
+ results:
16
+ - task:
17
+ type: Classification
18
+ dataset:
19
+ type: mteb/amazon_counterfactual
20
+ name: MTEB AmazonCounterfactualClassification (en)
21
+ config: en
22
+ split: test
23
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
24
+ metrics:
25
+ - type: accuracy
26
+ value: 75.5820895522388
27
+ - type: ap
28
+ value: 38.32190121241783
29
+ - type: f1
30
+ value: 69.44777155231054
31
+ - task:
32
+ type: Classification
33
+ dataset:
34
+ type: mteb/amazon_polarity
35
+ name: MTEB AmazonPolarityClassification
36
+ config: default
37
+ split: test
38
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
39
+ metrics:
40
+ - type: accuracy
41
+ value: 93.40514999999998
42
+ - type: ap
43
+ value: 90.2011565132406
44
+ - type: f1
45
+ value: 93.39486246843605
46
+ - task:
47
+ type: Classification
48
+ dataset:
49
+ type: mteb/amazon_reviews_multi
50
+ name: MTEB AmazonReviewsClassification (en)
51
+ config: en
52
+ split: test
53
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
54
+ metrics:
55
+ - type: accuracy
56
+ value: 49.05999999999999
57
+ - type: f1
58
+ value: 48.58702718571088
59
+ - task:
60
+ type: Retrieval
61
+ dataset:
62
+ type: arguana
63
+ name: MTEB ArguAna
64
+ config: default
65
+ split: test
66
+ revision: None
67
+ metrics:
68
+ - type: map_at_1
69
+ value: 38.407000000000004
70
+ - type: map_at_10
71
+ value: 54.822
72
+ - type: map_at_100
73
+ value: 55.387
74
+ - type: map_at_1000
75
+ value: 55.388999999999996
76
+ - type: map_at_3
77
+ value: 50.308
78
+ - type: map_at_5
79
+ value: 53.199
80
+ - type: mrr_at_1
81
+ value: 39.900000000000006
82
+ - type: mrr_at_10
83
+ value: 55.385
84
+ - type: mrr_at_100
85
+ value: 55.936
86
+ - type: mrr_at_1000
87
+ value: 55.93900000000001
88
+ - type: mrr_at_3
89
+ value: 50.853
90
+ - type: mrr_at_5
91
+ value: 53.738
92
+ - type: ndcg_at_1
93
+ value: 38.407000000000004
94
+ - type: ndcg_at_10
95
+ value: 63.38
96
+ - type: ndcg_at_100
97
+ value: 65.52900000000001
98
+ - type: ndcg_at_1000
99
+ value: 65.58800000000001
100
+ - type: ndcg_at_3
101
+ value: 54.26
102
+ - type: ndcg_at_5
103
+ value: 59.488
104
+ - type: precision_at_1
105
+ value: 38.407000000000004
106
+ - type: precision_at_10
107
+ value: 9.04
108
+ - type: precision_at_100
109
+ value: 0.992
110
+ - type: precision_at_1000
111
+ value: 0.1
112
+ - type: precision_at_3
113
+ value: 21.906
114
+ - type: precision_at_5
115
+ value: 15.690000000000001
116
+ - type: recall_at_1
117
+ value: 38.407000000000004
118
+ - type: recall_at_10
119
+ value: 90.398
120
+ - type: recall_at_100
121
+ value: 99.21799999999999
122
+ - type: recall_at_1000
123
+ value: 99.644
124
+ - type: recall_at_3
125
+ value: 65.718
126
+ - type: recall_at_5
127
+ value: 78.45
128
+ - task:
129
+ type: Clustering
130
+ dataset:
131
+ type: mteb/arxiv-clustering-p2p
132
+ name: MTEB ArxivClusteringP2P
133
+ config: default
134
+ split: test
135
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
136
+ metrics:
137
+ - type: v_measure
138
+ value: 48.49766333679089
139
+ - task:
140
+ type: Clustering
141
+ dataset:
142
+ type: mteb/arxiv-clustering-s2s
143
+ name: MTEB ArxivClusteringS2S
144
+ config: default
145
+ split: test
146
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
147
+ metrics:
148
+ - type: v_measure
149
+ value: 42.57731111438094
150
+ - task:
151
+ type: Reranking
152
+ dataset:
153
+ type: mteb/askubuntudupquestions-reranking
154
+ name: MTEB AskUbuntuDupQuestions
155
+ config: default
156
+ split: test
157
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
158
+ metrics:
159
+ - type: map
160
+ value: 64.70120072857361
161
+ - type: mrr
162
+ value: 77.86714593501297
163
+ - task:
164
+ type: STS
165
+ dataset:
166
+ type: mteb/biosses-sts
167
+ name: MTEB BIOSSES
168
+ config: default
169
+ split: test
170
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
171
+ metrics:
172
+ - type: cos_sim_pearson
173
+ value: 90.73821860690765
174
+ - type: cos_sim_spearman
175
+ value: 89.17070651383446
176
+ - type: euclidean_pearson
177
+ value: 88.28303958293029
178
+ - type: euclidean_spearman
179
+ value: 88.81889126856979
180
+ - type: manhattan_pearson
181
+ value: 88.09080621828731
182
+ - type: manhattan_spearman
183
+ value: 88.55924679817751
184
+ - task:
185
+ type: Classification
186
+ dataset:
187
+ type: mteb/banking77
188
+ name: MTEB Banking77Classification
189
+ config: default
190
+ split: test
191
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
192
+ metrics:
193
+ - type: accuracy
194
+ value: 88.10064935064933
195
+ - type: f1
196
+ value: 88.08460758973867
197
+ - task:
198
+ type: Clustering
199
+ dataset:
200
+ type: mteb/biorxiv-clustering-p2p
201
+ name: MTEB BiorxivClusteringP2P
202
+ config: default
203
+ split: test
204
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
205
+ metrics:
206
+ - type: v_measure
207
+ value: 39.338228337929976
208
+ - task:
209
+ type: Clustering
210
+ dataset:
211
+ type: mteb/biorxiv-clustering-s2s
212
+ name: MTEB BiorxivClusteringS2S
213
+ config: default
214
+ split: test
215
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
216
+ metrics:
217
+ - type: v_measure
218
+ value: 36.179156232378226
219
+ - task:
220
+ type: Retrieval
221
+ dataset:
222
+ type: BeIR/cqadupstack
223
+ name: MTEB CQADupstackAndroidRetrieval
224
+ config: default
225
+ split: test
226
+ revision: None
227
+ metrics:
228
+ - type: map_at_1
229
+ value: 33.440999999999995
230
+ - type: map_at_10
231
+ value: 45.495000000000005
232
+ - type: map_at_100
233
+ value: 47.132000000000005
234
+ - type: map_at_1000
235
+ value: 47.253
236
+ - type: map_at_3
237
+ value: 41.766
238
+ - type: map_at_5
239
+ value: 43.873
240
+ - type: mrr_at_1
241
+ value: 40.772999999999996
242
+ - type: mrr_at_10
243
+ value: 51.627
244
+ - type: mrr_at_100
245
+ value: 52.364
246
+ - type: mrr_at_1000
247
+ value: 52.397000000000006
248
+ - type: mrr_at_3
249
+ value: 48.951
250
+ - type: mrr_at_5
251
+ value: 50.746
252
+ - type: ndcg_at_1
253
+ value: 40.772999999999996
254
+ - type: ndcg_at_10
255
+ value: 52.306
256
+ - type: ndcg_at_100
257
+ value: 57.753
258
+ - type: ndcg_at_1000
259
+ value: 59.36900000000001
260
+ - type: ndcg_at_3
261
+ value: 47.177
262
+ - type: ndcg_at_5
263
+ value: 49.71
264
+ - type: precision_at_1
265
+ value: 40.772999999999996
266
+ - type: precision_at_10
267
+ value: 10.129000000000001
268
+ - type: precision_at_100
269
+ value: 1.617
270
+ - type: precision_at_1000
271
+ value: 0.208
272
+ - type: precision_at_3
273
+ value: 22.985
274
+ - type: precision_at_5
275
+ value: 16.652
276
+ - type: recall_at_1
277
+ value: 33.440999999999995
278
+ - type: recall_at_10
279
+ value: 65.121
280
+ - type: recall_at_100
281
+ value: 87.55199999999999
282
+ - type: recall_at_1000
283
+ value: 97.41300000000001
284
+ - type: recall_at_3
285
+ value: 49.958999999999996
286
+ - type: recall_at_5
287
+ value: 57.14900000000001
288
+ - task:
289
+ type: Retrieval
290
+ dataset:
291
+ type: BeIR/cqadupstack
292
+ name: MTEB CQADupstackEnglishRetrieval
293
+ config: default
294
+ split: test
295
+ revision: None
296
+ metrics:
297
+ - type: map_at_1
298
+ value: 32.126
299
+ - type: map_at_10
300
+ value: 42.856
301
+ - type: map_at_100
302
+ value: 44.134
303
+ - type: map_at_1000
304
+ value: 44.274
305
+ - type: map_at_3
306
+ value: 39.594
307
+ - type: map_at_5
308
+ value: 41.504999999999995
309
+ - type: mrr_at_1
310
+ value: 40.127
311
+ - type: mrr_at_10
312
+ value: 48.736000000000004
313
+ - type: mrr_at_100
314
+ value: 49.303999999999995
315
+ - type: mrr_at_1000
316
+ value: 49.356
317
+ - type: mrr_at_3
318
+ value: 46.263
319
+ - type: mrr_at_5
320
+ value: 47.878
321
+ - type: ndcg_at_1
322
+ value: 40.127
323
+ - type: ndcg_at_10
324
+ value: 48.695
325
+ - type: ndcg_at_100
326
+ value: 52.846000000000004
327
+ - type: ndcg_at_1000
328
+ value: 54.964
329
+ - type: ndcg_at_3
330
+ value: 44.275
331
+ - type: ndcg_at_5
332
+ value: 46.54
333
+ - type: precision_at_1
334
+ value: 40.127
335
+ - type: precision_at_10
336
+ value: 9.229
337
+ - type: precision_at_100
338
+ value: 1.473
339
+ - type: precision_at_1000
340
+ value: 0.19499999999999998
341
+ - type: precision_at_3
342
+ value: 21.444
343
+ - type: precision_at_5
344
+ value: 15.389
345
+ - type: recall_at_1
346
+ value: 32.126
347
+ - type: recall_at_10
348
+ value: 58.971
349
+ - type: recall_at_100
350
+ value: 76.115
351
+ - type: recall_at_1000
352
+ value: 89.556
353
+ - type: recall_at_3
354
+ value: 45.891
355
+ - type: recall_at_5
356
+ value: 52.242
357
+ - task:
358
+ type: Retrieval
359
+ dataset:
360
+ type: BeIR/cqadupstack
361
+ name: MTEB CQADupstackGamingRetrieval
362
+ config: default
363
+ split: test
364
+ revision: None
365
+ metrics:
366
+ - type: map_at_1
367
+ value: 41.312
368
+ - type: map_at_10
369
+ value: 54.510000000000005
370
+ - type: map_at_100
371
+ value: 55.544000000000004
372
+ - type: map_at_1000
373
+ value: 55.593
374
+ - type: map_at_3
375
+ value: 50.859
376
+ - type: map_at_5
377
+ value: 52.839999999999996
378
+ - type: mrr_at_1
379
+ value: 47.147
380
+ - type: mrr_at_10
381
+ value: 57.678
382
+ - type: mrr_at_100
383
+ value: 58.287
384
+ - type: mrr_at_1000
385
+ value: 58.312
386
+ - type: mrr_at_3
387
+ value: 55.025999999999996
388
+ - type: mrr_at_5
389
+ value: 56.55
390
+ - type: ndcg_at_1
391
+ value: 47.147
392
+ - type: ndcg_at_10
393
+ value: 60.672000000000004
394
+ - type: ndcg_at_100
395
+ value: 64.411
396
+ - type: ndcg_at_1000
397
+ value: 65.35499999999999
398
+ - type: ndcg_at_3
399
+ value: 54.643
400
+ - type: ndcg_at_5
401
+ value: 57.461
402
+ - type: precision_at_1
403
+ value: 47.147
404
+ - type: precision_at_10
405
+ value: 9.881
406
+ - type: precision_at_100
407
+ value: 1.27
408
+ - type: precision_at_1000
409
+ value: 0.13799999999999998
410
+ - type: precision_at_3
411
+ value: 24.556
412
+ - type: precision_at_5
413
+ value: 16.814999999999998
414
+ - type: recall_at_1
415
+ value: 41.312
416
+ - type: recall_at_10
417
+ value: 75.62299999999999
418
+ - type: recall_at_100
419
+ value: 91.388
420
+ - type: recall_at_1000
421
+ value: 98.08
422
+ - type: recall_at_3
423
+ value: 59.40299999999999
424
+ - type: recall_at_5
425
+ value: 66.43900000000001
426
+ - task:
427
+ type: Retrieval
428
+ dataset:
429
+ type: BeIR/cqadupstack
430
+ name: MTEB CQADupstackGisRetrieval
431
+ config: default
432
+ split: test
433
+ revision: None
434
+ metrics:
435
+ - type: map_at_1
436
+ value: 27.609
437
+ - type: map_at_10
438
+ value: 37.614
439
+ - type: map_at_100
440
+ value: 38.584
441
+ - type: map_at_1000
442
+ value: 38.652
443
+ - type: map_at_3
444
+ value: 34.731
445
+ - type: map_at_5
446
+ value: 36.308
447
+ - type: mrr_at_1
448
+ value: 29.944
449
+ - type: mrr_at_10
450
+ value: 39.829
451
+ - type: mrr_at_100
452
+ value: 40.659
453
+ - type: mrr_at_1000
454
+ value: 40.709
455
+ - type: mrr_at_3
456
+ value: 37.269000000000005
457
+ - type: mrr_at_5
458
+ value: 38.625
459
+ - type: ndcg_at_1
460
+ value: 29.944
461
+ - type: ndcg_at_10
462
+ value: 43.082
463
+ - type: ndcg_at_100
464
+ value: 47.857
465
+ - type: ndcg_at_1000
466
+ value: 49.612
467
+ - type: ndcg_at_3
468
+ value: 37.578
469
+ - type: ndcg_at_5
470
+ value: 40.135
471
+ - type: precision_at_1
472
+ value: 29.944
473
+ - type: precision_at_10
474
+ value: 6.678000000000001
475
+ - type: precision_at_100
476
+ value: 0.951
477
+ - type: precision_at_1000
478
+ value: 0.11399999999999999
479
+ - type: precision_at_3
480
+ value: 16.045
481
+ - type: precision_at_5
482
+ value: 11.073
483
+ - type: recall_at_1
484
+ value: 27.609
485
+ - type: recall_at_10
486
+ value: 57.718
487
+ - type: recall_at_100
488
+ value: 79.768
489
+ - type: recall_at_1000
490
+ value: 92.868
491
+ - type: recall_at_3
492
+ value: 42.876
493
+ - type: recall_at_5
494
+ value: 49.104
495
+ - task:
496
+ type: Retrieval
497
+ dataset:
498
+ type: BeIR/cqadupstack
499
+ name: MTEB CQADupstackMathematicaRetrieval
500
+ config: default
501
+ split: test
502
+ revision: None
503
+ metrics:
504
+ - type: map_at_1
505
+ value: 18.071
506
+ - type: map_at_10
507
+ value: 27.471
508
+ - type: map_at_100
509
+ value: 28.71
510
+ - type: map_at_1000
511
+ value: 28.833
512
+ - type: map_at_3
513
+ value: 24.698
514
+ - type: map_at_5
515
+ value: 26.461000000000002
516
+ - type: mrr_at_1
517
+ value: 22.387999999999998
518
+ - type: mrr_at_10
519
+ value: 32.522
520
+ - type: mrr_at_100
521
+ value: 33.393
522
+ - type: mrr_at_1000
523
+ value: 33.455
524
+ - type: mrr_at_3
525
+ value: 29.830000000000002
526
+ - type: mrr_at_5
527
+ value: 31.472
528
+ - type: ndcg_at_1
529
+ value: 22.387999999999998
530
+ - type: ndcg_at_10
531
+ value: 33.278999999999996
532
+ - type: ndcg_at_100
533
+ value: 39.043
534
+ - type: ndcg_at_1000
535
+ value: 41.763
536
+ - type: ndcg_at_3
537
+ value: 28.310999999999996
538
+ - type: ndcg_at_5
539
+ value: 31.007
540
+ - type: precision_at_1
541
+ value: 22.387999999999998
542
+ - type: precision_at_10
543
+ value: 6.157
544
+ - type: precision_at_100
545
+ value: 1.042
546
+ - type: precision_at_1000
547
+ value: 0.14200000000000002
548
+ - type: precision_at_3
549
+ value: 13.972000000000001
550
+ - type: precision_at_5
551
+ value: 10.274
552
+ - type: recall_at_1
553
+ value: 18.071
554
+ - type: recall_at_10
555
+ value: 46.025
556
+ - type: recall_at_100
557
+ value: 71.153
558
+ - type: recall_at_1000
559
+ value: 90.232
560
+ - type: recall_at_3
561
+ value: 32.311
562
+ - type: recall_at_5
563
+ value: 39.296
564
+ - task:
565
+ type: Retrieval
566
+ dataset:
567
+ type: BeIR/cqadupstack
568
+ name: MTEB CQADupstackPhysicsRetrieval
569
+ config: default
570
+ split: test
571
+ revision: None
572
+ metrics:
573
+ - type: map_at_1
574
+ value: 30.813000000000002
575
+ - type: map_at_10
576
+ value: 42.594
577
+ - type: map_at_100
578
+ value: 43.949
579
+ - type: map_at_1000
580
+ value: 44.052
581
+ - type: map_at_3
582
+ value: 39.1
583
+ - type: map_at_5
584
+ value: 41.111
585
+ - type: mrr_at_1
586
+ value: 37.824999999999996
587
+ - type: mrr_at_10
588
+ value: 48.06
589
+ - type: mrr_at_100
590
+ value: 48.91
591
+ - type: mrr_at_1000
592
+ value: 48.946
593
+ - type: mrr_at_3
594
+ value: 45.509
595
+ - type: mrr_at_5
596
+ value: 47.073
597
+ - type: ndcg_at_1
598
+ value: 37.824999999999996
599
+ - type: ndcg_at_10
600
+ value: 48.882
601
+ - type: ndcg_at_100
602
+ value: 54.330999999999996
603
+ - type: ndcg_at_1000
604
+ value: 56.120999999999995
605
+ - type: ndcg_at_3
606
+ value: 43.529
607
+ - type: ndcg_at_5
608
+ value: 46.217999999999996
609
+ - type: precision_at_1
610
+ value: 37.824999999999996
611
+ - type: precision_at_10
612
+ value: 8.845
613
+ - type: precision_at_100
614
+ value: 1.34
615
+ - type: precision_at_1000
616
+ value: 0.168
617
+ - type: precision_at_3
618
+ value: 20.757
619
+ - type: precision_at_5
620
+ value: 14.802999999999999
621
+ - type: recall_at_1
622
+ value: 30.813000000000002
623
+ - type: recall_at_10
624
+ value: 61.895999999999994
625
+ - type: recall_at_100
626
+ value: 84.513
627
+ - type: recall_at_1000
628
+ value: 95.817
629
+ - type: recall_at_3
630
+ value: 47.099000000000004
631
+ - type: recall_at_5
632
+ value: 54.031
633
+ - task:
634
+ type: Retrieval
635
+ dataset:
636
+ type: BeIR/cqadupstack
637
+ name: MTEB CQADupstackProgrammersRetrieval
638
+ config: default
639
+ split: test
640
+ revision: None
641
+ metrics:
642
+ - type: map_at_1
643
+ value: 25.735999999999997
644
+ - type: map_at_10
645
+ value: 36.799
646
+ - type: map_at_100
647
+ value: 38.246
648
+ - type: map_at_1000
649
+ value: 38.353
650
+ - type: map_at_3
651
+ value: 33.133
652
+ - type: map_at_5
653
+ value: 34.954
654
+ - type: mrr_at_1
655
+ value: 31.849
656
+ - type: mrr_at_10
657
+ value: 41.928
658
+ - type: mrr_at_100
659
+ value: 42.846000000000004
660
+ - type: mrr_at_1000
661
+ value: 42.894
662
+ - type: mrr_at_3
663
+ value: 39.117000000000004
664
+ - type: mrr_at_5
665
+ value: 40.521
666
+ - type: ndcg_at_1
667
+ value: 31.849
668
+ - type: ndcg_at_10
669
+ value: 43.143
670
+ - type: ndcg_at_100
671
+ value: 48.963
672
+ - type: ndcg_at_1000
673
+ value: 51.041000000000004
674
+ - type: ndcg_at_3
675
+ value: 37.218
676
+ - type: ndcg_at_5
677
+ value: 39.542
678
+ - type: precision_at_1
679
+ value: 31.849
680
+ - type: precision_at_10
681
+ value: 8.231
682
+ - type: precision_at_100
683
+ value: 1.277
684
+ - type: precision_at_1000
685
+ value: 0.164
686
+ - type: precision_at_3
687
+ value: 18.037
688
+ - type: precision_at_5
689
+ value: 12.945
690
+ - type: recall_at_1
691
+ value: 25.735999999999997
692
+ - type: recall_at_10
693
+ value: 56.735
694
+ - type: recall_at_100
695
+ value: 81.04
696
+ - type: recall_at_1000
697
+ value: 94.845
698
+ - type: recall_at_3
699
+ value: 40.239999999999995
700
+ - type: recall_at_5
701
+ value: 46.378
702
+ - task:
703
+ type: Retrieval
704
+ dataset:
705
+ type: BeIR/cqadupstack
706
+ name: MTEB CQADupstackRetrieval
707
+ config: default
708
+ split: test
709
+ revision: None
710
+ metrics:
711
+ - type: map_at_1
712
+ value: 27.580333333333336
713
+ - type: map_at_10
714
+ value: 37.70558333333334
715
+ - type: map_at_100
716
+ value: 38.94941666666667
717
+ - type: map_at_1000
718
+ value: 39.062083333333334
719
+ - type: map_at_3
720
+ value: 34.63333333333334
721
+ - type: map_at_5
722
+ value: 36.35241666666666
723
+ - type: mrr_at_1
724
+ value: 32.64866666666667
725
+ - type: mrr_at_10
726
+ value: 42.018499999999996
727
+ - type: mrr_at_100
728
+ value: 42.83391666666666
729
+ - type: mrr_at_1000
730
+ value: 42.884166666666665
731
+ - type: mrr_at_3
732
+ value: 39.476499999999994
733
+ - type: mrr_at_5
734
+ value: 40.96983333333334
735
+ - type: ndcg_at_1
736
+ value: 32.64866666666667
737
+ - type: ndcg_at_10
738
+ value: 43.43866666666667
739
+ - type: ndcg_at_100
740
+ value: 48.569833333333335
741
+ - type: ndcg_at_1000
742
+ value: 50.6495
743
+ - type: ndcg_at_3
744
+ value: 38.327166666666656
745
+ - type: ndcg_at_5
746
+ value: 40.76941666666667
747
+ - type: precision_at_1
748
+ value: 32.64866666666667
749
+ - type: precision_at_10
750
+ value: 7.652333333333332
751
+ - type: precision_at_100
752
+ value: 1.2066666666666666
753
+ - type: precision_at_1000
754
+ value: 0.15841666666666668
755
+ - type: precision_at_3
756
+ value: 17.75108333333333
757
+ - type: precision_at_5
758
+ value: 12.641916666666669
759
+ - type: recall_at_1
760
+ value: 27.580333333333336
761
+ - type: recall_at_10
762
+ value: 56.02591666666667
763
+ - type: recall_at_100
764
+ value: 78.317
765
+ - type: recall_at_1000
766
+ value: 92.52608333333332
767
+ - type: recall_at_3
768
+ value: 41.84283333333333
769
+ - type: recall_at_5
770
+ value: 48.105666666666664
771
+ - task:
772
+ type: Retrieval
773
+ dataset:
774
+ type: BeIR/cqadupstack
775
+ name: MTEB CQADupstackStatsRetrieval
776
+ config: default
777
+ split: test
778
+ revision: None
779
+ metrics:
780
+ - type: map_at_1
781
+ value: 27.876
782
+ - type: map_at_10
783
+ value: 34.521
784
+ - type: map_at_100
785
+ value: 35.581
786
+ - type: map_at_1000
787
+ value: 35.674
788
+ - type: map_at_3
789
+ value: 32.501000000000005
790
+ - type: map_at_5
791
+ value: 33.602
792
+ - type: mrr_at_1
793
+ value: 31.441999999999997
794
+ - type: mrr_at_10
795
+ value: 37.669999999999995
796
+ - type: mrr_at_100
797
+ value: 38.523
798
+ - type: mrr_at_1000
799
+ value: 38.59
800
+ - type: mrr_at_3
801
+ value: 35.762
802
+ - type: mrr_at_5
803
+ value: 36.812
804
+ - type: ndcg_at_1
805
+ value: 31.441999999999997
806
+ - type: ndcg_at_10
807
+ value: 38.46
808
+ - type: ndcg_at_100
809
+ value: 43.479
810
+ - type: ndcg_at_1000
811
+ value: 45.858
812
+ - type: ndcg_at_3
813
+ value: 34.668
814
+ - type: ndcg_at_5
815
+ value: 36.416
816
+ - type: precision_at_1
817
+ value: 31.441999999999997
818
+ - type: precision_at_10
819
+ value: 5.782
820
+ - type: precision_at_100
821
+ value: 0.91
822
+ - type: precision_at_1000
823
+ value: 0.11900000000000001
824
+ - type: precision_at_3
825
+ value: 14.417
826
+ - type: precision_at_5
827
+ value: 9.876999999999999
828
+ - type: recall_at_1
829
+ value: 27.876
830
+ - type: recall_at_10
831
+ value: 47.556
832
+ - type: recall_at_100
833
+ value: 70.39699999999999
834
+ - type: recall_at_1000
835
+ value: 87.969
836
+ - type: recall_at_3
837
+ value: 37.226
838
+ - type: recall_at_5
839
+ value: 41.43
840
+ - task:
841
+ type: Retrieval
842
+ dataset:
843
+ type: BeIR/cqadupstack
844
+ name: MTEB CQADupstackTexRetrieval
845
+ config: default
846
+ split: test
847
+ revision: None
848
+ metrics:
849
+ - type: map_at_1
850
+ value: 18.854000000000003
851
+ - type: map_at_10
852
+ value: 26.632
853
+ - type: map_at_100
854
+ value: 27.849
855
+ - type: map_at_1000
856
+ value: 27.977
857
+ - type: map_at_3
858
+ value: 24.089
859
+ - type: map_at_5
860
+ value: 25.477
861
+ - type: mrr_at_1
862
+ value: 22.987
863
+ - type: mrr_at_10
864
+ value: 30.781999999999996
865
+ - type: mrr_at_100
866
+ value: 31.746000000000002
867
+ - type: mrr_at_1000
868
+ value: 31.818
869
+ - type: mrr_at_3
870
+ value: 28.43
871
+ - type: mrr_at_5
872
+ value: 29.791
873
+ - type: ndcg_at_1
874
+ value: 22.987
875
+ - type: ndcg_at_10
876
+ value: 31.585
877
+ - type: ndcg_at_100
878
+ value: 37.32
879
+ - type: ndcg_at_1000
880
+ value: 40.072
881
+ - type: ndcg_at_3
882
+ value: 27.058
883
+ - type: ndcg_at_5
884
+ value: 29.137999999999998
885
+ - type: precision_at_1
886
+ value: 22.987
887
+ - type: precision_at_10
888
+ value: 5.76
889
+ - type: precision_at_100
890
+ value: 1.018
891
+ - type: precision_at_1000
892
+ value: 0.14400000000000002
893
+ - type: precision_at_3
894
+ value: 12.767000000000001
895
+ - type: precision_at_5
896
+ value: 9.257
897
+ - type: recall_at_1
898
+ value: 18.854000000000003
899
+ - type: recall_at_10
900
+ value: 42.349
901
+ - type: recall_at_100
902
+ value: 68.15299999999999
903
+ - type: recall_at_1000
904
+ value: 87.44
905
+ - type: recall_at_3
906
+ value: 29.715999999999998
907
+ - type: recall_at_5
908
+ value: 35.085
909
+ - task:
910
+ type: Retrieval
911
+ dataset:
912
+ type: BeIR/cqadupstack
913
+ name: MTEB CQADupstackUnixRetrieval
914
+ config: default
915
+ split: test
916
+ revision: None
917
+ metrics:
918
+ - type: map_at_1
919
+ value: 28.094
920
+ - type: map_at_10
921
+ value: 38.22
922
+ - type: map_at_100
923
+ value: 39.352
924
+ - type: map_at_1000
925
+ value: 39.452
926
+ - type: map_at_3
927
+ value: 35.339
928
+ - type: map_at_5
929
+ value: 36.78
930
+ - type: mrr_at_1
931
+ value: 33.022
932
+ - type: mrr_at_10
933
+ value: 42.466
934
+ - type: mrr_at_100
935
+ value: 43.3
936
+ - type: mrr_at_1000
937
+ value: 43.356
938
+ - type: mrr_at_3
939
+ value: 40.159
940
+ - type: mrr_at_5
941
+ value: 41.272999999999996
942
+ - type: ndcg_at_1
943
+ value: 33.022
944
+ - type: ndcg_at_10
945
+ value: 43.976
946
+ - type: ndcg_at_100
947
+ value: 49.008
948
+ - type: ndcg_at_1000
949
+ value: 51.154999999999994
950
+ - type: ndcg_at_3
951
+ value: 38.891
952
+ - type: ndcg_at_5
953
+ value: 40.897
954
+ - type: precision_at_1
955
+ value: 33.022
956
+ - type: precision_at_10
957
+ value: 7.396999999999999
958
+ - type: precision_at_100
959
+ value: 1.1199999999999999
960
+ - type: precision_at_1000
961
+ value: 0.14200000000000002
962
+ - type: precision_at_3
963
+ value: 17.724
964
+ - type: precision_at_5
965
+ value: 12.239
966
+ - type: recall_at_1
967
+ value: 28.094
968
+ - type: recall_at_10
969
+ value: 57.162
970
+ - type: recall_at_100
971
+ value: 78.636
972
+ - type: recall_at_1000
973
+ value: 93.376
974
+ - type: recall_at_3
975
+ value: 43.328
976
+ - type: recall_at_5
977
+ value: 48.252
978
+ - task:
979
+ type: Retrieval
980
+ dataset:
981
+ type: BeIR/cqadupstack
982
+ name: MTEB CQADupstackWebmastersRetrieval
983
+ config: default
984
+ split: test
985
+ revision: None
986
+ metrics:
987
+ - type: map_at_1
988
+ value: 24.937
989
+ - type: map_at_10
990
+ value: 34.82
991
+ - type: map_at_100
992
+ value: 36.405
993
+ - type: map_at_1000
994
+ value: 36.626
995
+ - type: map_at_3
996
+ value: 31.548
997
+ - type: map_at_5
998
+ value: 33.355000000000004
999
+ - type: mrr_at_1
1000
+ value: 30.435000000000002
1001
+ - type: mrr_at_10
1002
+ value: 39.946
1003
+ - type: mrr_at_100
1004
+ value: 40.873
1005
+ - type: mrr_at_1000
1006
+ value: 40.910000000000004
1007
+ - type: mrr_at_3
1008
+ value: 37.088
1009
+ - type: mrr_at_5
1010
+ value: 38.808
1011
+ - type: ndcg_at_1
1012
+ value: 30.435000000000002
1013
+ - type: ndcg_at_10
1014
+ value: 41.25
1015
+ - type: ndcg_at_100
1016
+ value: 47.229
1017
+ - type: ndcg_at_1000
1018
+ value: 49.395
1019
+ - type: ndcg_at_3
1020
+ value: 35.801
1021
+ - type: ndcg_at_5
1022
+ value: 38.457
1023
+ - type: precision_at_1
1024
+ value: 30.435000000000002
1025
+ - type: precision_at_10
1026
+ value: 8.083
1027
+ - type: precision_at_100
1028
+ value: 1.601
1029
+ - type: precision_at_1000
1030
+ value: 0.247
1031
+ - type: precision_at_3
1032
+ value: 17.061999999999998
1033
+ - type: precision_at_5
1034
+ value: 12.767000000000001
1035
+ - type: recall_at_1
1036
+ value: 24.937
1037
+ - type: recall_at_10
1038
+ value: 53.905
1039
+ - type: recall_at_100
1040
+ value: 80.607
1041
+ - type: recall_at_1000
1042
+ value: 93.728
1043
+ - type: recall_at_3
1044
+ value: 38.446000000000005
1045
+ - type: recall_at_5
1046
+ value: 45.188
1047
+ - task:
1048
+ type: Retrieval
1049
+ dataset:
1050
+ type: BeIR/cqadupstack
1051
+ name: MTEB CQADupstackWordpressRetrieval
1052
+ config: default
1053
+ split: test
1054
+ revision: None
1055
+ metrics:
1056
+ - type: map_at_1
1057
+ value: 22.095000000000002
1058
+ - type: map_at_10
1059
+ value: 30.935000000000002
1060
+ - type: map_at_100
1061
+ value: 31.907000000000004
1062
+ - type: map_at_1000
1063
+ value: 32.006
1064
+ - type: map_at_3
1065
+ value: 28.242
1066
+ - type: map_at_5
1067
+ value: 29.963
1068
+ - type: mrr_at_1
1069
+ value: 23.845
1070
+ - type: mrr_at_10
1071
+ value: 32.978
1072
+ - type: mrr_at_100
1073
+ value: 33.802
1074
+ - type: mrr_at_1000
1075
+ value: 33.867000000000004
1076
+ - type: mrr_at_3
1077
+ value: 30.314000000000004
1078
+ - type: mrr_at_5
1079
+ value: 32.089
1080
+ - type: ndcg_at_1
1081
+ value: 23.845
1082
+ - type: ndcg_at_10
1083
+ value: 35.934
1084
+ - type: ndcg_at_100
1085
+ value: 40.598
1086
+ - type: ndcg_at_1000
1087
+ value: 43.089
1088
+ - type: ndcg_at_3
1089
+ value: 30.776999999999997
1090
+ - type: ndcg_at_5
1091
+ value: 33.711999999999996
1092
+ - type: precision_at_1
1093
+ value: 23.845
1094
+ - type: precision_at_10
1095
+ value: 5.656
1096
+ - type: precision_at_100
1097
+ value: 0.861
1098
+ - type: precision_at_1000
1099
+ value: 0.12
1100
+ - type: precision_at_3
1101
+ value: 13.247
1102
+ - type: precision_at_5
1103
+ value: 9.612
1104
+ - type: recall_at_1
1105
+ value: 22.095000000000002
1106
+ - type: recall_at_10
1107
+ value: 49.25
1108
+ - type: recall_at_100
1109
+ value: 70.482
1110
+ - type: recall_at_1000
1111
+ value: 88.98899999999999
1112
+ - type: recall_at_3
1113
+ value: 35.619
1114
+ - type: recall_at_5
1115
+ value: 42.674
1116
+ - task:
1117
+ type: Retrieval
1118
+ dataset:
1119
+ type: climate-fever
1120
+ name: MTEB ClimateFEVER
1121
+ config: default
1122
+ split: test
1123
+ revision: None
1124
+ metrics:
1125
+ - type: map_at_1
1126
+ value: 14.154
1127
+ - type: map_at_10
1128
+ value: 24.654999999999998
1129
+ - type: map_at_100
1130
+ value: 26.723999999999997
1131
+ - type: map_at_1000
1132
+ value: 26.912000000000003
1133
+ - type: map_at_3
1134
+ value: 20.4
1135
+ - type: map_at_5
1136
+ value: 22.477
1137
+ - type: mrr_at_1
1138
+ value: 32.117000000000004
1139
+ - type: mrr_at_10
1140
+ value: 44.590999999999994
1141
+ - type: mrr_at_100
1142
+ value: 45.425
1143
+ - type: mrr_at_1000
1144
+ value: 45.456
1145
+ - type: mrr_at_3
1146
+ value: 41.281
1147
+ - type: mrr_at_5
1148
+ value: 43.219
1149
+ - type: ndcg_at_1
1150
+ value: 32.117000000000004
1151
+ - type: ndcg_at_10
1152
+ value: 33.994
1153
+ - type: ndcg_at_100
1154
+ value: 41.438
1155
+ - type: ndcg_at_1000
1156
+ value: 44.611000000000004
1157
+ - type: ndcg_at_3
1158
+ value: 27.816000000000003
1159
+ - type: ndcg_at_5
1160
+ value: 29.816
1161
+ - type: precision_at_1
1162
+ value: 32.117000000000004
1163
+ - type: precision_at_10
1164
+ value: 10.756
1165
+ - type: precision_at_100
1166
+ value: 1.8679999999999999
1167
+ - type: precision_at_1000
1168
+ value: 0.246
1169
+ - type: precision_at_3
1170
+ value: 20.803
1171
+ - type: precision_at_5
1172
+ value: 15.987000000000002
1173
+ - type: recall_at_1
1174
+ value: 14.154
1175
+ - type: recall_at_10
1176
+ value: 40.489999999999995
1177
+ - type: recall_at_100
1178
+ value: 65.635
1179
+ - type: recall_at_1000
1180
+ value: 83.276
1181
+ - type: recall_at_3
1182
+ value: 25.241000000000003
1183
+ - type: recall_at_5
1184
+ value: 31.211
1185
+ - task:
1186
+ type: Retrieval
1187
+ dataset:
1188
+ type: dbpedia-entity
1189
+ name: MTEB DBPedia
1190
+ config: default
1191
+ split: test
1192
+ revision: None
1193
+ metrics:
1194
+ - type: map_at_1
1195
+ value: 9.332
1196
+ - type: map_at_10
1197
+ value: 20.462
1198
+ - type: map_at_100
1199
+ value: 29.473
1200
+ - type: map_at_1000
1201
+ value: 31.215
1202
+ - type: map_at_3
1203
+ value: 14.466999999999999
1204
+ - type: map_at_5
1205
+ value: 16.922
1206
+ - type: mrr_at_1
1207
+ value: 69.5
1208
+ - type: mrr_at_10
1209
+ value: 77.039
1210
+ - type: mrr_at_100
1211
+ value: 77.265
1212
+ - type: mrr_at_1000
1213
+ value: 77.271
1214
+ - type: mrr_at_3
1215
+ value: 75.5
1216
+ - type: mrr_at_5
1217
+ value: 76.4
1218
+ - type: ndcg_at_1
1219
+ value: 57.125
1220
+ - type: ndcg_at_10
1221
+ value: 42.958
1222
+ - type: ndcg_at_100
1223
+ value: 48.396
1224
+ - type: ndcg_at_1000
1225
+ value: 55.897
1226
+ - type: ndcg_at_3
1227
+ value: 47.188
1228
+ - type: ndcg_at_5
1229
+ value: 44.376
1230
+ - type: precision_at_1
1231
+ value: 69.5
1232
+ - type: precision_at_10
1233
+ value: 34.5
1234
+ - type: precision_at_100
1235
+ value: 11.18
1236
+ - type: precision_at_1000
1237
+ value: 2.13
1238
+ - type: precision_at_3
1239
+ value: 51.083
1240
+ - type: precision_at_5
1241
+ value: 43.1
1242
+ - type: recall_at_1
1243
+ value: 9.332
1244
+ - type: recall_at_10
1245
+ value: 26.422
1246
+ - type: recall_at_100
1247
+ value: 56.098000000000006
1248
+ - type: recall_at_1000
1249
+ value: 79.66
1250
+ - type: recall_at_3
1251
+ value: 15.703
1252
+ - type: recall_at_5
1253
+ value: 19.644000000000002
1254
+ - task:
1255
+ type: Classification
1256
+ dataset:
1257
+ type: mteb/emotion
1258
+ name: MTEB EmotionClassification
1259
+ config: default
1260
+ split: test
1261
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1262
+ metrics:
1263
+ - type: accuracy
1264
+ value: 54.72
1265
+ - type: f1
1266
+ value: 49.67819606587526
1267
+ - task:
1268
+ type: Retrieval
1269
+ dataset:
1270
+ type: fever
1271
+ name: MTEB FEVER
1272
+ config: default
1273
+ split: test
1274
+ revision: None
1275
+ metrics:
1276
+ - type: map_at_1
1277
+ value: 74.97
1278
+ - type: map_at_10
1279
+ value: 82.956
1280
+ - type: map_at_100
1281
+ value: 83.193
1282
+ - type: map_at_1000
1283
+ value: 83.208
1284
+ - type: map_at_3
1285
+ value: 81.837
1286
+ - type: map_at_5
1287
+ value: 82.57
1288
+ - type: mrr_at_1
1289
+ value: 80.783
1290
+ - type: mrr_at_10
1291
+ value: 87.546
1292
+ - type: mrr_at_100
1293
+ value: 87.627
1294
+ - type: mrr_at_1000
1295
+ value: 87.63
1296
+ - type: mrr_at_3
1297
+ value: 86.79400000000001
1298
+ - type: mrr_at_5
1299
+ value: 87.32799999999999
1300
+ - type: ndcg_at_1
1301
+ value: 80.783
1302
+ - type: ndcg_at_10
1303
+ value: 86.54899999999999
1304
+ - type: ndcg_at_100
1305
+ value: 87.355
1306
+ - type: ndcg_at_1000
1307
+ value: 87.629
1308
+ - type: ndcg_at_3
1309
+ value: 84.82
1310
+ - type: ndcg_at_5
1311
+ value: 85.83800000000001
1312
+ - type: precision_at_1
1313
+ value: 80.783
1314
+ - type: precision_at_10
1315
+ value: 10.327
1316
+ - type: precision_at_100
1317
+ value: 1.094
1318
+ - type: precision_at_1000
1319
+ value: 0.11299999999999999
1320
+ - type: precision_at_3
1321
+ value: 32.218
1322
+ - type: precision_at_5
1323
+ value: 20.012
1324
+ - type: recall_at_1
1325
+ value: 74.97
1326
+ - type: recall_at_10
1327
+ value: 93.072
1328
+ - type: recall_at_100
1329
+ value: 96.218
1330
+ - type: recall_at_1000
1331
+ value: 97.991
1332
+ - type: recall_at_3
1333
+ value: 88.357
1334
+ - type: recall_at_5
1335
+ value: 90.983
1336
+ - task:
1337
+ type: Retrieval
1338
+ dataset:
1339
+ type: fiqa
1340
+ name: MTEB FiQA2018
1341
+ config: default
1342
+ split: test
1343
+ revision: None
1344
+ metrics:
1345
+ - type: map_at_1
1346
+ value: 21.12
1347
+ - type: map_at_10
1348
+ value: 35.908
1349
+ - type: map_at_100
1350
+ value: 37.895
1351
+ - type: map_at_1000
1352
+ value: 38.068000000000005
1353
+ - type: map_at_3
1354
+ value: 31.189
1355
+ - type: map_at_5
1356
+ value: 33.908
1357
+ - type: mrr_at_1
1358
+ value: 42.901
1359
+ - type: mrr_at_10
1360
+ value: 52.578
1361
+ - type: mrr_at_100
1362
+ value: 53.308
1363
+ - type: mrr_at_1000
1364
+ value: 53.342
1365
+ - type: mrr_at_3
1366
+ value: 50.385999999999996
1367
+ - type: mrr_at_5
1368
+ value: 51.62799999999999
1369
+ - type: ndcg_at_1
1370
+ value: 42.901
1371
+ - type: ndcg_at_10
1372
+ value: 44.302
1373
+ - type: ndcg_at_100
1374
+ value: 51.132999999999996
1375
+ - type: ndcg_at_1000
1376
+ value: 53.848
1377
+ - type: ndcg_at_3
1378
+ value: 40.464
1379
+ - type: ndcg_at_5
1380
+ value: 41.743
1381
+ - type: precision_at_1
1382
+ value: 42.901
1383
+ - type: precision_at_10
1384
+ value: 12.423
1385
+ - type: precision_at_100
1386
+ value: 1.968
1387
+ - type: precision_at_1000
1388
+ value: 0.246
1389
+ - type: precision_at_3
1390
+ value: 27.622999999999998
1391
+ - type: precision_at_5
1392
+ value: 20.278
1393
+ - type: recall_at_1
1394
+ value: 21.12
1395
+ - type: recall_at_10
1396
+ value: 52.091
1397
+ - type: recall_at_100
1398
+ value: 77.062
1399
+ - type: recall_at_1000
1400
+ value: 93.082
1401
+ - type: recall_at_3
1402
+ value: 37.223
1403
+ - type: recall_at_5
1404
+ value: 43.826
1405
+ - task:
1406
+ type: Retrieval
1407
+ dataset:
1408
+ type: hotpotqa
1409
+ name: MTEB HotpotQA
1410
+ config: default
1411
+ split: test
1412
+ revision: None
1413
+ metrics:
1414
+ - type: map_at_1
1415
+ value: 38.940000000000005
1416
+ - type: map_at_10
1417
+ value: 62.239999999999995
1418
+ - type: map_at_100
1419
+ value: 63.141000000000005
1420
+ - type: map_at_1000
1421
+ value: 63.205999999999996
1422
+ - type: map_at_3
1423
+ value: 58.738
1424
+ - type: map_at_5
1425
+ value: 60.924
1426
+ - type: mrr_at_1
1427
+ value: 77.88000000000001
1428
+ - type: mrr_at_10
1429
+ value: 83.7
1430
+ - type: mrr_at_100
1431
+ value: 83.882
1432
+ - type: mrr_at_1000
1433
+ value: 83.889
1434
+ - type: mrr_at_3
1435
+ value: 82.748
1436
+ - type: mrr_at_5
1437
+ value: 83.381
1438
+ - type: ndcg_at_1
1439
+ value: 77.88000000000001
1440
+ - type: ndcg_at_10
1441
+ value: 70.462
1442
+ - type: ndcg_at_100
1443
+ value: 73.564
1444
+ - type: ndcg_at_1000
1445
+ value: 74.78099999999999
1446
+ - type: ndcg_at_3
1447
+ value: 65.524
1448
+ - type: ndcg_at_5
1449
+ value: 68.282
1450
+ - type: precision_at_1
1451
+ value: 77.88000000000001
1452
+ - type: precision_at_10
1453
+ value: 14.81
1454
+ - type: precision_at_100
1455
+ value: 1.7229999999999999
1456
+ - type: precision_at_1000
1457
+ value: 0.188
1458
+ - type: precision_at_3
1459
+ value: 42.083999999999996
1460
+ - type: precision_at_5
1461
+ value: 27.43
1462
+ - type: recall_at_1
1463
+ value: 38.940000000000005
1464
+ - type: recall_at_10
1465
+ value: 74.051
1466
+ - type: recall_at_100
1467
+ value: 86.158
1468
+ - type: recall_at_1000
1469
+ value: 94.146
1470
+ - type: recall_at_3
1471
+ value: 63.126000000000005
1472
+ - type: recall_at_5
1473
+ value: 68.575
1474
+ - task:
1475
+ type: Classification
1476
+ dataset:
1477
+ type: mteb/imdb
1478
+ name: MTEB ImdbClassification
1479
+ config: default
1480
+ split: test
1481
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1482
+ metrics:
1483
+ - type: accuracy
1484
+ value: 91.23440000000001
1485
+ - type: ap
1486
+ value: 87.33490392265892
1487
+ - type: f1
1488
+ value: 91.21374626021836
1489
+ - task:
1490
+ type: Retrieval
1491
+ dataset:
1492
+ type: msmarco
1493
+ name: MTEB MSMARCO
1494
+ config: default
1495
+ split: dev
1496
+ revision: None
1497
+ metrics:
1498
+ - type: map_at_1
1499
+ value: 22.137999999999998
1500
+ - type: map_at_10
1501
+ value: 34.471000000000004
1502
+ - type: map_at_100
1503
+ value: 35.634
1504
+ - type: map_at_1000
1505
+ value: 35.685
1506
+ - type: map_at_3
1507
+ value: 30.587999999999997
1508
+ - type: map_at_5
1509
+ value: 32.812999999999995
1510
+ - type: mrr_at_1
1511
+ value: 22.736
1512
+ - type: mrr_at_10
1513
+ value: 35.092
1514
+ - type: mrr_at_100
1515
+ value: 36.193999999999996
1516
+ - type: mrr_at_1000
1517
+ value: 36.238
1518
+ - type: mrr_at_3
1519
+ value: 31.28
1520
+ - type: mrr_at_5
1521
+ value: 33.498
1522
+ - type: ndcg_at_1
1523
+ value: 22.736
1524
+ - type: ndcg_at_10
1525
+ value: 41.388999999999996
1526
+ - type: ndcg_at_100
1527
+ value: 46.967999999999996
1528
+ - type: ndcg_at_1000
1529
+ value: 48.178
1530
+ - type: ndcg_at_3
1531
+ value: 33.503
1532
+ - type: ndcg_at_5
1533
+ value: 37.484
1534
+ - type: precision_at_1
1535
+ value: 22.736
1536
+ - type: precision_at_10
1537
+ value: 6.54
1538
+ - type: precision_at_100
1539
+ value: 0.9339999999999999
1540
+ - type: precision_at_1000
1541
+ value: 0.104
1542
+ - type: precision_at_3
1543
+ value: 14.249999999999998
1544
+ - type: precision_at_5
1545
+ value: 10.562000000000001
1546
+ - type: recall_at_1
1547
+ value: 22.137999999999998
1548
+ - type: recall_at_10
1549
+ value: 62.629999999999995
1550
+ - type: recall_at_100
1551
+ value: 88.375
1552
+ - type: recall_at_1000
1553
+ value: 97.529
1554
+ - type: recall_at_3
1555
+ value: 41.245
1556
+ - type: recall_at_5
1557
+ value: 50.808
1558
+ - task:
1559
+ type: Classification
1560
+ dataset:
1561
+ type: mteb/mtop_domain
1562
+ name: MTEB MTOPDomainClassification (en)
1563
+ config: en
1564
+ split: test
1565
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1566
+ metrics:
1567
+ - type: accuracy
1568
+ value: 95.25079799361606
1569
+ - type: f1
1570
+ value: 95.00726023695032
1571
+ - task:
1572
+ type: Classification
1573
+ dataset:
1574
+ type: mteb/mtop_intent
1575
+ name: MTEB MTOPIntentClassification (en)
1576
+ config: en
1577
+ split: test
1578
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1579
+ metrics:
1580
+ - type: accuracy
1581
+ value: 78.23757409940721
1582
+ - type: f1
1583
+ value: 58.534958803195714
1584
+ - task:
1585
+ type: Classification
1586
+ dataset:
1587
+ type: mteb/amazon_massive_intent
1588
+ name: MTEB MassiveIntentClassification (en)
1589
+ config: en
1590
+ split: test
1591
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1592
+ metrics:
1593
+ - type: accuracy
1594
+ value: 76.20040349697378
1595
+ - type: f1
1596
+ value: 74.31261149784696
1597
+ - task:
1598
+ type: Classification
1599
+ dataset:
1600
+ type: mteb/amazon_massive_scenario
1601
+ name: MTEB MassiveScenarioClassification (en)
1602
+ config: en
1603
+ split: test
1604
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
1605
+ metrics:
1606
+ - type: accuracy
1607
+ value: 79.35104236718227
1608
+ - type: f1
1609
+ value: 79.7373049864316
1610
+ - task:
1611
+ type: Clustering
1612
+ dataset:
1613
+ type: mteb/medrxiv-clustering-p2p
1614
+ name: MTEB MedrxivClusteringP2P
1615
+ config: default
1616
+ split: test
1617
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1618
+ metrics:
1619
+ - type: v_measure
1620
+ value: 34.478828180753126
1621
+ - task:
1622
+ type: Clustering
1623
+ dataset:
1624
+ type: mteb/medrxiv-clustering-s2s
1625
+ name: MTEB MedrxivClusteringS2S
1626
+ config: default
1627
+ split: test
1628
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1629
+ metrics:
1630
+ - type: v_measure
1631
+ value: 32.25696147904426
1632
+ - task:
1633
+ type: Reranking
1634
+ dataset:
1635
+ type: mteb/mind_small
1636
+ name: MTEB MindSmallReranking
1637
+ config: default
1638
+ split: test
1639
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1640
+ metrics:
1641
+ - type: map
1642
+ value: 32.82488548405117
1643
+ - type: mrr
1644
+ value: 34.066706809031096
1645
+ - task:
1646
+ type: Retrieval
1647
+ dataset:
1648
+ type: nfcorpus
1649
+ name: MTEB NFCorpus
1650
+ config: default
1651
+ split: test
1652
+ revision: None
1653
+ metrics:
1654
+ - type: map_at_1
1655
+ value: 6.557
1656
+ - type: map_at_10
1657
+ value: 15.055
1658
+ - type: map_at_100
1659
+ value: 19.575
1660
+ - type: map_at_1000
1661
+ value: 21.267
1662
+ - type: map_at_3
1663
+ value: 10.86
1664
+ - type: map_at_5
1665
+ value: 12.83
1666
+ - type: mrr_at_1
1667
+ value: 50.464
1668
+ - type: mrr_at_10
1669
+ value: 59.050999999999995
1670
+ - type: mrr_at_100
1671
+ value: 59.436
1672
+ - type: mrr_at_1000
1673
+ value: 59.476
1674
+ - type: mrr_at_3
1675
+ value: 56.811
1676
+ - type: mrr_at_5
1677
+ value: 58.08
1678
+ - type: ndcg_at_1
1679
+ value: 47.988
1680
+ - type: ndcg_at_10
1681
+ value: 38.645
1682
+ - type: ndcg_at_100
1683
+ value: 36.339
1684
+ - type: ndcg_at_1000
1685
+ value: 45.279
1686
+ - type: ndcg_at_3
1687
+ value: 43.35
1688
+ - type: ndcg_at_5
1689
+ value: 41.564
1690
+ - type: precision_at_1
1691
+ value: 49.845
1692
+ - type: precision_at_10
1693
+ value: 28.544999999999998
1694
+ - type: precision_at_100
1695
+ value: 9.322
1696
+ - type: precision_at_1000
1697
+ value: 2.258
1698
+ - type: precision_at_3
1699
+ value: 40.144000000000005
1700
+ - type: precision_at_5
1701
+ value: 35.913000000000004
1702
+ - type: recall_at_1
1703
+ value: 6.557
1704
+ - type: recall_at_10
1705
+ value: 19.5
1706
+ - type: recall_at_100
1707
+ value: 37.153999999999996
1708
+ - type: recall_at_1000
1709
+ value: 69.581
1710
+ - type: recall_at_3
1711
+ value: 12.133
1712
+ - type: recall_at_5
1713
+ value: 15.43
1714
+ - task:
1715
+ type: Retrieval
1716
+ dataset:
1717
+ type: nq
1718
+ name: MTEB NQ
1719
+ config: default
1720
+ split: test
1721
+ revision: None
1722
+ metrics:
1723
+ - type: map_at_1
1724
+ value: 31.740000000000002
1725
+ - type: map_at_10
1726
+ value: 48.150999999999996
1727
+ - type: map_at_100
1728
+ value: 49.125
1729
+ - type: map_at_1000
1730
+ value: 49.149
1731
+ - type: map_at_3
1732
+ value: 43.645
1733
+ - type: map_at_5
1734
+ value: 46.417
1735
+ - type: mrr_at_1
1736
+ value: 35.892
1737
+ - type: mrr_at_10
1738
+ value: 50.524
1739
+ - type: mrr_at_100
1740
+ value: 51.232
1741
+ - type: mrr_at_1000
1742
+ value: 51.24999999999999
1743
+ - type: mrr_at_3
1744
+ value: 46.852
1745
+ - type: mrr_at_5
1746
+ value: 49.146
1747
+ - type: ndcg_at_1
1748
+ value: 35.892
1749
+ - type: ndcg_at_10
1750
+ value: 56.08800000000001
1751
+ - type: ndcg_at_100
1752
+ value: 60.077000000000005
1753
+ - type: ndcg_at_1000
1754
+ value: 60.632
1755
+ - type: ndcg_at_3
1756
+ value: 47.765
1757
+ - type: ndcg_at_5
1758
+ value: 52.322
1759
+ - type: precision_at_1
1760
+ value: 35.892
1761
+ - type: precision_at_10
1762
+ value: 9.296
1763
+ - type: precision_at_100
1764
+ value: 1.154
1765
+ - type: precision_at_1000
1766
+ value: 0.12
1767
+ - type: precision_at_3
1768
+ value: 21.92
1769
+ - type: precision_at_5
1770
+ value: 15.781999999999998
1771
+ - type: recall_at_1
1772
+ value: 31.740000000000002
1773
+ - type: recall_at_10
1774
+ value: 77.725
1775
+ - type: recall_at_100
1776
+ value: 94.841
1777
+ - type: recall_at_1000
1778
+ value: 99.003
1779
+ - type: recall_at_3
1780
+ value: 56.407
1781
+ - type: recall_at_5
1782
+ value: 66.848
1783
+ - task:
1784
+ type: Retrieval
1785
+ dataset:
1786
+ type: quora
1787
+ name: MTEB QuoraRetrieval
1788
+ config: default
1789
+ split: test
1790
+ revision: None
1791
+ metrics:
1792
+ - type: map_at_1
1793
+ value: 71.429
1794
+ - type: map_at_10
1795
+ value: 85.42699999999999
1796
+ - type: map_at_100
1797
+ value: 86.063
1798
+ - type: map_at_1000
1799
+ value: 86.077
1800
+ - type: map_at_3
1801
+ value: 82.573
1802
+ - type: map_at_5
1803
+ value: 84.371
1804
+ - type: mrr_at_1
1805
+ value: 82.34
1806
+ - type: mrr_at_10
1807
+ value: 88.247
1808
+ - type: mrr_at_100
1809
+ value: 88.357
1810
+ - type: mrr_at_1000
1811
+ value: 88.357
1812
+ - type: mrr_at_3
1813
+ value: 87.38
1814
+ - type: mrr_at_5
1815
+ value: 87.981
1816
+ - type: ndcg_at_1
1817
+ value: 82.34
1818
+ - type: ndcg_at_10
1819
+ value: 88.979
1820
+ - type: ndcg_at_100
1821
+ value: 90.18599999999999
1822
+ - type: ndcg_at_1000
1823
+ value: 90.254
1824
+ - type: ndcg_at_3
1825
+ value: 86.378
1826
+ - type: ndcg_at_5
1827
+ value: 87.821
1828
+ - type: precision_at_1
1829
+ value: 82.34
1830
+ - type: precision_at_10
1831
+ value: 13.482
1832
+ - type: precision_at_100
1833
+ value: 1.537
1834
+ - type: precision_at_1000
1835
+ value: 0.157
1836
+ - type: precision_at_3
1837
+ value: 37.852999999999994
1838
+ - type: precision_at_5
1839
+ value: 24.798000000000002
1840
+ - type: recall_at_1
1841
+ value: 71.429
1842
+ - type: recall_at_10
1843
+ value: 95.64099999999999
1844
+ - type: recall_at_100
1845
+ value: 99.723
1846
+ - type: recall_at_1000
1847
+ value: 99.98
1848
+ - type: recall_at_3
1849
+ value: 88.011
1850
+ - type: recall_at_5
1851
+ value: 92.246
1852
+ - task:
1853
+ type: Clustering
1854
+ dataset:
1855
+ type: mteb/reddit-clustering
1856
+ name: MTEB RedditClustering
1857
+ config: default
1858
+ split: test
1859
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
1860
+ metrics:
1861
+ - type: v_measure
1862
+ value: 60.62148584103299
1863
+ - task:
1864
+ type: Clustering
1865
+ dataset:
1866
+ type: mteb/reddit-clustering-p2p
1867
+ name: MTEB RedditClusteringP2P
1868
+ config: default
1869
+ split: test
1870
+ revision: 282350215ef01743dc01b456c7f5241fa8937f16
1871
+ metrics:
1872
+ - type: v_measure
1873
+ value: 63.2923987272903
1874
+ - task:
1875
+ type: Retrieval
1876
+ dataset:
1877
+ type: scidocs
1878
+ name: MTEB SCIDOCS
1879
+ config: default
1880
+ split: test
1881
+ revision: None
1882
+ metrics:
1883
+ - type: map_at_1
1884
+ value: 5.128
1885
+ - type: map_at_10
1886
+ value: 14.63
1887
+ - type: map_at_100
1888
+ value: 17.285
1889
+ - type: map_at_1000
1890
+ value: 17.676
1891
+ - type: map_at_3
1892
+ value: 9.993
1893
+ - type: map_at_5
1894
+ value: 12.286999999999999
1895
+ - type: mrr_at_1
1896
+ value: 25.4
1897
+ - type: mrr_at_10
1898
+ value: 38.423
1899
+ - type: mrr_at_100
1900
+ value: 39.497
1901
+ - type: mrr_at_1000
1902
+ value: 39.531
1903
+ - type: mrr_at_3
1904
+ value: 34.9
1905
+ - type: mrr_at_5
1906
+ value: 37.01
1907
+ - type: ndcg_at_1
1908
+ value: 25.4
1909
+ - type: ndcg_at_10
1910
+ value: 24.062
1911
+ - type: ndcg_at_100
1912
+ value: 33.823
1913
+ - type: ndcg_at_1000
1914
+ value: 39.663
1915
+ - type: ndcg_at_3
1916
+ value: 22.246
1917
+ - type: ndcg_at_5
1918
+ value: 19.761
1919
+ - type: precision_at_1
1920
+ value: 25.4
1921
+ - type: precision_at_10
1922
+ value: 12.85
1923
+ - type: precision_at_100
1924
+ value: 2.71
1925
+ - type: precision_at_1000
1926
+ value: 0.41000000000000003
1927
+ - type: precision_at_3
1928
+ value: 21.4
1929
+ - type: precision_at_5
1930
+ value: 17.86
1931
+ - type: recall_at_1
1932
+ value: 5.128
1933
+ - type: recall_at_10
1934
+ value: 26.06
1935
+ - type: recall_at_100
1936
+ value: 54.993
1937
+ - type: recall_at_1000
1938
+ value: 83.165
1939
+ - type: recall_at_3
1940
+ value: 13.003
1941
+ - type: recall_at_5
1942
+ value: 18.117
1943
+ - task:
1944
+ type: STS
1945
+ dataset:
1946
+ type: mteb/sickr-sts
1947
+ name: MTEB SICK-R
1948
+ config: default
1949
+ split: test
1950
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
1951
+ metrics:
1952
+ - type: cos_sim_pearson
1953
+ value: 87.5466779326323
1954
+ - type: cos_sim_spearman
1955
+ value: 82.79782085421951
1956
+ - type: euclidean_pearson
1957
+ value: 84.76929982677339
1958
+ - type: euclidean_spearman
1959
+ value: 82.51802536005597
1960
+ - type: manhattan_pearson
1961
+ value: 84.76736312526177
1962
+ - type: manhattan_spearman
1963
+ value: 82.50799656335593
1964
+ - task:
1965
+ type: STS
1966
+ dataset:
1967
+ type: mteb/sts12-sts
1968
+ name: MTEB STS12
1969
+ config: default
1970
+ split: test
1971
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
1972
+ metrics:
1973
+ - type: cos_sim_pearson
1974
+ value: 86.40486308108694
1975
+ - type: cos_sim_spearman
1976
+ value: 77.12670500926937
1977
+ - type: euclidean_pearson
1978
+ value: 85.23836845503847
1979
+ - type: euclidean_spearman
1980
+ value: 78.41475117006176
1981
+ - type: manhattan_pearson
1982
+ value: 85.24302039610805
1983
+ - type: manhattan_spearman
1984
+ value: 78.4053162562707
1985
+ - task:
1986
+ type: STS
1987
+ dataset:
1988
+ type: mteb/sts13-sts
1989
+ name: MTEB STS13
1990
+ config: default
1991
+ split: test
1992
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
1993
+ metrics:
1994
+ - type: cos_sim_pearson
1995
+ value: 88.83570289087565
1996
+ - type: cos_sim_spearman
1997
+ value: 89.28563503553643
1998
+ - type: euclidean_pearson
1999
+ value: 87.77516003996445
2000
+ - type: euclidean_spearman
2001
+ value: 88.8656149534085
2002
+ - type: manhattan_pearson
2003
+ value: 87.75568872417946
2004
+ - type: manhattan_spearman
2005
+ value: 88.80445489340585
2006
+ - task:
2007
+ type: STS
2008
+ dataset:
2009
+ type: mteb/sts14-sts
2010
+ name: MTEB STS14
2011
+ config: default
2012
+ split: test
2013
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
2014
+ metrics:
2015
+ - type: cos_sim_pearson
2016
+ value: 86.776406555485
2017
+ - type: cos_sim_spearman
2018
+ value: 83.8288465070091
2019
+ - type: euclidean_pearson
2020
+ value: 85.37827999808123
2021
+ - type: euclidean_spearman
2022
+ value: 84.11079529992739
2023
+ - type: manhattan_pearson
2024
+ value: 85.35336495689121
2025
+ - type: manhattan_spearman
2026
+ value: 84.08618492649347
2027
+ - task:
2028
+ type: STS
2029
+ dataset:
2030
+ type: mteb/sts15-sts
2031
+ name: MTEB STS15
2032
+ config: default
2033
+ split: test
2034
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
2035
+ metrics:
2036
+ - type: cos_sim_pearson
2037
+ value: 88.57644404820684
2038
+ - type: cos_sim_spearman
2039
+ value: 89.69728364350713
2040
+ - type: euclidean_pearson
2041
+ value: 88.28202320389443
2042
+ - type: euclidean_spearman
2043
+ value: 88.9560567319321
2044
+ - type: manhattan_pearson
2045
+ value: 88.29461100044172
2046
+ - type: manhattan_spearman
2047
+ value: 88.96030920678558
2048
+ - task:
2049
+ type: STS
2050
+ dataset:
2051
+ type: mteb/sts16-sts
2052
+ name: MTEB STS16
2053
+ config: default
2054
+ split: test
2055
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2056
+ metrics:
2057
+ - type: cos_sim_pearson
2058
+ value: 85.05211938460621
2059
+ - type: cos_sim_spearman
2060
+ value: 86.43413865667489
2061
+ - type: euclidean_pearson
2062
+ value: 85.62760689259562
2063
+ - type: euclidean_spearman
2064
+ value: 86.28867831982394
2065
+ - type: manhattan_pearson
2066
+ value: 85.60828879163458
2067
+ - type: manhattan_spearman
2068
+ value: 86.27823731462473
2069
+ - task:
2070
+ type: STS
2071
+ dataset:
2072
+ type: mteb/sts17-crosslingual-sts
2073
+ name: MTEB STS17 (en-en)
2074
+ config: en-en
2075
+ split: test
2076
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2077
+ metrics:
2078
+ - type: cos_sim_pearson
2079
+ value: 90.00254140466377
2080
+ - type: cos_sim_spearman
2081
+ value: 89.66118745178284
2082
+ - type: euclidean_pearson
2083
+ value: 89.46985446236553
2084
+ - type: euclidean_spearman
2085
+ value: 88.92649032371526
2086
+ - type: manhattan_pearson
2087
+ value: 89.49600028180247
2088
+ - type: manhattan_spearman
2089
+ value: 88.86948431519099
2090
+ - task:
2091
+ type: STS
2092
+ dataset:
2093
+ type: mteb/sts22-crosslingual-sts
2094
+ name: MTEB STS22 (en)
2095
+ config: en
2096
+ split: test
2097
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
2098
+ metrics:
2099
+ - type: cos_sim_pearson
2100
+ value: 68.93578321067938
2101
+ - type: cos_sim_spearman
2102
+ value: 69.60639595839257
2103
+ - type: euclidean_pearson
2104
+ value: 70.33485090574897
2105
+ - type: euclidean_spearman
2106
+ value: 69.03380379185452
2107
+ - type: manhattan_pearson
2108
+ value: 70.42097254943839
2109
+ - type: manhattan_spearman
2110
+ value: 69.25296348304255
2111
+ - task:
2112
+ type: STS
2113
+ dataset:
2114
+ type: mteb/stsbenchmark-sts
2115
+ name: MTEB STSBenchmark
2116
+ config: default
2117
+ split: test
2118
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2119
+ metrics:
2120
+ - type: cos_sim_pearson
2121
+ value: 87.29588700755069
2122
+ - type: cos_sim_spearman
2123
+ value: 88.30389489193672
2124
+ - type: euclidean_pearson
2125
+ value: 87.60349838180346
2126
+ - type: euclidean_spearman
2127
+ value: 87.91041868311692
2128
+ - type: manhattan_pearson
2129
+ value: 87.59373630607907
2130
+ - type: manhattan_spearman
2131
+ value: 87.88690174001724
2132
+ - task:
2133
+ type: Reranking
2134
+ dataset:
2135
+ type: mteb/scidocs-reranking
2136
+ name: MTEB SciDocsRR
2137
+ config: default
2138
+ split: test
2139
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2140
+ metrics:
2141
+ - type: map
2142
+ value: 87.8030655700857
2143
+ - type: mrr
2144
+ value: 96.3950637234951
2145
+ - task:
2146
+ type: Retrieval
2147
+ dataset:
2148
+ type: scifact
2149
+ name: MTEB SciFact
2150
+ config: default
2151
+ split: test
2152
+ revision: None
2153
+ metrics:
2154
+ - type: map_at_1
2155
+ value: 60.028000000000006
2156
+ - type: map_at_10
2157
+ value: 69.855
2158
+ - type: map_at_100
2159
+ value: 70.257
2160
+ - type: map_at_1000
2161
+ value: 70.283
2162
+ - type: map_at_3
2163
+ value: 66.769
2164
+ - type: map_at_5
2165
+ value: 68.679
2166
+ - type: mrr_at_1
2167
+ value: 62.666999999999994
2168
+ - type: mrr_at_10
2169
+ value: 70.717
2170
+ - type: mrr_at_100
2171
+ value: 71.00800000000001
2172
+ - type: mrr_at_1000
2173
+ value: 71.033
2174
+ - type: mrr_at_3
2175
+ value: 68.389
2176
+ - type: mrr_at_5
2177
+ value: 69.939
2178
+ - type: ndcg_at_1
2179
+ value: 62.666999999999994
2180
+ - type: ndcg_at_10
2181
+ value: 74.715
2182
+ - type: ndcg_at_100
2183
+ value: 76.364
2184
+ - type: ndcg_at_1000
2185
+ value: 76.89399999999999
2186
+ - type: ndcg_at_3
2187
+ value: 69.383
2188
+ - type: ndcg_at_5
2189
+ value: 72.322
2190
+ - type: precision_at_1
2191
+ value: 62.666999999999994
2192
+ - type: precision_at_10
2193
+ value: 10.067
2194
+ - type: precision_at_100
2195
+ value: 1.09
2196
+ - type: precision_at_1000
2197
+ value: 0.11299999999999999
2198
+ - type: precision_at_3
2199
+ value: 27.111
2200
+ - type: precision_at_5
2201
+ value: 18.267
2202
+ - type: recall_at_1
2203
+ value: 60.028000000000006
2204
+ - type: recall_at_10
2205
+ value: 88.822
2206
+ - type: recall_at_100
2207
+ value: 96.167
2208
+ - type: recall_at_1000
2209
+ value: 100.0
2210
+ - type: recall_at_3
2211
+ value: 74.367
2212
+ - type: recall_at_5
2213
+ value: 81.661
2214
+ - task:
2215
+ type: PairClassification
2216
+ dataset:
2217
+ type: mteb/sprintduplicatequestions-pairclassification
2218
+ name: MTEB SprintDuplicateQuestions
2219
+ config: default
2220
+ split: test
2221
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2222
+ metrics:
2223
+ - type: cos_sim_accuracy
2224
+ value: 99.84554455445544
2225
+ - type: cos_sim_ap
2226
+ value: 96.54482863244152
2227
+ - type: cos_sim_f1
2228
+ value: 92.13709677419355
2229
+ - type: cos_sim_precision
2230
+ value: 92.88617886178862
2231
+ - type: cos_sim_recall
2232
+ value: 91.4
2233
+ - type: dot_accuracy
2234
+ value: 99.76039603960396
2235
+ - type: dot_ap
2236
+ value: 93.20115278887057
2237
+ - type: dot_f1
2238
+ value: 87.92079207920793
2239
+ - type: dot_precision
2240
+ value: 87.05882352941177
2241
+ - type: dot_recall
2242
+ value: 88.8
2243
+ - type: euclidean_accuracy
2244
+ value: 99.84950495049505
2245
+ - type: euclidean_ap
2246
+ value: 96.53268343961348
2247
+ - type: euclidean_f1
2248
+ value: 92.23697650663942
2249
+ - type: euclidean_precision
2250
+ value: 94.258872651357
2251
+ - type: euclidean_recall
2252
+ value: 90.3
2253
+ - type: manhattan_accuracy
2254
+ value: 99.85346534653465
2255
+ - type: manhattan_ap
2256
+ value: 96.54495433438355
2257
+ - type: manhattan_f1
2258
+ value: 92.51012145748987
2259
+ - type: manhattan_precision
2260
+ value: 93.64754098360656
2261
+ - type: manhattan_recall
2262
+ value: 91.4
2263
+ - type: max_accuracy
2264
+ value: 99.85346534653465
2265
+ - type: max_ap
2266
+ value: 96.54495433438355
2267
+ - type: max_f1
2268
+ value: 92.51012145748987
2269
+ - task:
2270
+ type: Clustering
2271
+ dataset:
2272
+ type: mteb/stackexchange-clustering
2273
+ name: MTEB StackExchangeClustering
2274
+ config: default
2275
+ split: test
2276
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2277
+ metrics:
2278
+ - type: v_measure
2279
+ value: 66.46940443952006
2280
+ - task:
2281
+ type: Clustering
2282
+ dataset:
2283
+ type: mteb/stackexchange-clustering-p2p
2284
+ name: MTEB StackExchangeClusteringP2P
2285
+ config: default
2286
+ split: test
2287
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2288
+ metrics:
2289
+ - type: v_measure
2290
+ value: 36.396194493841584
2291
+ - task:
2292
+ type: Reranking
2293
+ dataset:
2294
+ type: mteb/stackoverflowdupquestions-reranking
2295
+ name: MTEB StackOverflowDupQuestions
2296
+ config: default
2297
+ split: test
2298
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2299
+ metrics:
2300
+ - type: map
2301
+ value: 54.881717673695555
2302
+ - type: mrr
2303
+ value: 55.73439224174519
2304
+ - task:
2305
+ type: Summarization
2306
+ dataset:
2307
+ type: mteb/summeval
2308
+ name: MTEB SummEval
2309
+ config: default
2310
+ split: test
2311
+ revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
2312
+ metrics:
2313
+ - type: cos_sim_pearson
2314
+ value: 31.438177268254087
2315
+ - type: cos_sim_spearman
2316
+ value: 30.96177698848688
2317
+ - type: dot_pearson
2318
+ value: 30.513850376431435
2319
+ - type: dot_spearman
2320
+ value: 29.932421046509706
2321
+ - task:
2322
+ type: Retrieval
2323
+ dataset:
2324
+ type: trec-covid
2325
+ name: MTEB TRECCOVID
2326
+ config: default
2327
+ split: test
2328
+ revision: None
2329
+ metrics:
2330
+ - type: map_at_1
2331
+ value: 0.21
2332
+ - type: map_at_10
2333
+ value: 1.727
2334
+ - type: map_at_100
2335
+ value: 9.881
2336
+ - type: map_at_1000
2337
+ value: 24.245
2338
+ - type: map_at_3
2339
+ value: 0.615
2340
+ - type: map_at_5
2341
+ value: 0.966
2342
+ - type: mrr_at_1
2343
+ value: 78.0
2344
+ - type: mrr_at_10
2345
+ value: 87.333
2346
+ - type: mrr_at_100
2347
+ value: 87.333
2348
+ - type: mrr_at_1000
2349
+ value: 87.333
2350
+ - type: mrr_at_3
2351
+ value: 86.333
2352
+ - type: mrr_at_5
2353
+ value: 87.333
2354
+ - type: ndcg_at_1
2355
+ value: 74.0
2356
+ - type: ndcg_at_10
2357
+ value: 69.12700000000001
2358
+ - type: ndcg_at_100
2359
+ value: 53.893
2360
+ - type: ndcg_at_1000
2361
+ value: 49.639
2362
+ - type: ndcg_at_3
2363
+ value: 74.654
2364
+ - type: ndcg_at_5
2365
+ value: 73.232
2366
+ - type: precision_at_1
2367
+ value: 78.0
2368
+ - type: precision_at_10
2369
+ value: 72.8
2370
+ - type: precision_at_100
2371
+ value: 55.42
2372
+ - type: precision_at_1000
2373
+ value: 21.73
2374
+ - type: precision_at_3
2375
+ value: 79.333
2376
+ - type: precision_at_5
2377
+ value: 77.2
2378
+ - type: recall_at_1
2379
+ value: 0.21
2380
+ - type: recall_at_10
2381
+ value: 1.9709999999999999
2382
+ - type: recall_at_100
2383
+ value: 13.555
2384
+ - type: recall_at_1000
2385
+ value: 46.961999999999996
2386
+ - type: recall_at_3
2387
+ value: 0.66
2388
+ - type: recall_at_5
2389
+ value: 1.052
2390
+ - task:
2391
+ type: Retrieval
2392
+ dataset:
2393
+ type: webis-touche2020
2394
+ name: MTEB Touche2020
2395
+ config: default
2396
+ split: test
2397
+ revision: None
2398
+ metrics:
2399
+ - type: map_at_1
2400
+ value: 2.456
2401
+ - type: map_at_10
2402
+ value: 9.426
2403
+ - type: map_at_100
2404
+ value: 16.066
2405
+ - type: map_at_1000
2406
+ value: 17.652
2407
+ - type: map_at_3
2408
+ value: 5.2459999999999996
2409
+ - type: map_at_5
2410
+ value: 6.5360000000000005
2411
+ - type: mrr_at_1
2412
+ value: 34.694
2413
+ - type: mrr_at_10
2414
+ value: 47.666
2415
+ - type: mrr_at_100
2416
+ value: 48.681999999999995
2417
+ - type: mrr_at_1000
2418
+ value: 48.681999999999995
2419
+ - type: mrr_at_3
2420
+ value: 43.878
2421
+ - type: mrr_at_5
2422
+ value: 46.224
2423
+ - type: ndcg_at_1
2424
+ value: 31.633
2425
+ - type: ndcg_at_10
2426
+ value: 23.454
2427
+ - type: ndcg_at_100
2428
+ value: 36.616
2429
+ - type: ndcg_at_1000
2430
+ value: 48.596000000000004
2431
+ - type: ndcg_at_3
2432
+ value: 28.267999999999997
2433
+ - type: ndcg_at_5
2434
+ value: 25.630999999999997
2435
+ - type: precision_at_1
2436
+ value: 34.694
2437
+ - type: precision_at_10
2438
+ value: 20.204
2439
+ - type: precision_at_100
2440
+ value: 7.754999999999999
2441
+ - type: precision_at_1000
2442
+ value: 1.5709999999999997
2443
+ - type: precision_at_3
2444
+ value: 29.252
2445
+ - type: precision_at_5
2446
+ value: 24.898
2447
+ - type: recall_at_1
2448
+ value: 2.456
2449
+ - type: recall_at_10
2450
+ value: 14.951
2451
+ - type: recall_at_100
2452
+ value: 48.399
2453
+ - type: recall_at_1000
2454
+ value: 85.077
2455
+ - type: recall_at_3
2456
+ value: 6.1370000000000005
2457
+ - type: recall_at_5
2458
+ value: 8.671
2459
+ - task:
2460
+ type: Classification
2461
+ dataset:
2462
+ type: mteb/toxic_conversations_50k
2463
+ name: MTEB ToxicConversationsClassification
2464
+ config: default
2465
+ split: test
2466
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2467
+ metrics:
2468
+ - type: accuracy
2469
+ value: 71.86240000000001
2470
+ - type: ap
2471
+ value: 14.678570078747494
2472
+ - type: f1
2473
+ value: 55.295967793934445
2474
+ - task:
2475
+ type: Classification
2476
+ dataset:
2477
+ type: mteb/tweet_sentiment_extraction
2478
+ name: MTEB TweetSentimentExtractionClassification
2479
+ config: default
2480
+ split: test
2481
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2482
+ metrics:
2483
+ - type: accuracy
2484
+ value: 59.17374080362195
2485
+ - type: f1
2486
+ value: 59.54410874861454
2487
+ - task:
2488
+ type: Clustering
2489
+ dataset:
2490
+ type: mteb/twentynewsgroups-clustering
2491
+ name: MTEB TwentyNewsgroupsClustering
2492
+ config: default
2493
+ split: test
2494
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2495
+ metrics:
2496
+ - type: v_measure
2497
+ value: 51.91227822485289
2498
+ - task:
2499
+ type: PairClassification
2500
+ dataset:
2501
+ type: mteb/twittersemeval2015-pairclassification
2502
+ name: MTEB TwitterSemEval2015
2503
+ config: default
2504
+ split: test
2505
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2506
+ metrics:
2507
+ - type: cos_sim_accuracy
2508
+ value: 87.12523097097217
2509
+ - type: cos_sim_ap
2510
+ value: 77.59606075943269
2511
+ - type: cos_sim_f1
2512
+ value: 71.11395646606915
2513
+ - type: cos_sim_precision
2514
+ value: 69.07960199004975
2515
+ - type: cos_sim_recall
2516
+ value: 73.27176781002639
2517
+ - type: dot_accuracy
2518
+ value: 84.68736961316088
2519
+ - type: dot_ap
2520
+ value: 68.47167450741459
2521
+ - type: dot_f1
2522
+ value: 64.42152354914874
2523
+ - type: dot_precision
2524
+ value: 60.887949260042284
2525
+ - type: dot_recall
2526
+ value: 68.3905013192612
2527
+ - type: euclidean_accuracy
2528
+ value: 86.88084878106932
2529
+ - type: euclidean_ap
2530
+ value: 77.27351204978599
2531
+ - type: euclidean_f1
2532
+ value: 70.99179716629381
2533
+ - type: euclidean_precision
2534
+ value: 67.10526315789474
2535
+ - type: euclidean_recall
2536
+ value: 75.35620052770449
2537
+ - type: manhattan_accuracy
2538
+ value: 86.83316445133218
2539
+ - type: manhattan_ap
2540
+ value: 77.21835357308716
2541
+ - type: manhattan_f1
2542
+ value: 71.05587004676349
2543
+ - type: manhattan_precision
2544
+ value: 66.58210332103322
2545
+ - type: manhattan_recall
2546
+ value: 76.17414248021109
2547
+ - type: max_accuracy
2548
+ value: 87.12523097097217
2549
+ - type: max_ap
2550
+ value: 77.59606075943269
2551
+ - type: max_f1
2552
+ value: 71.11395646606915
2553
+ - task:
2554
+ type: PairClassification
2555
+ dataset:
2556
+ type: mteb/twitterurlcorpus-pairclassification
2557
+ name: MTEB TwitterURLCorpus
2558
+ config: default
2559
+ split: test
2560
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2561
+ metrics:
2562
+ - type: cos_sim_accuracy
2563
+ value: 88.97232894787906
2564
+ - type: cos_sim_ap
2565
+ value: 85.9613736469497
2566
+ - type: cos_sim_f1
2567
+ value: 78.40216655382532
2568
+ - type: cos_sim_precision
2569
+ value: 72.97512437810946
2570
+ - type: cos_sim_recall
2571
+ value: 84.70126270403449
2572
+ - type: dot_accuracy
2573
+ value: 88.04866689952264
2574
+ - type: dot_ap
2575
+ value: 83.15465089499936
2576
+ - type: dot_f1
2577
+ value: 76.32698287879329
2578
+ - type: dot_precision
2579
+ value: 71.23223697378077
2580
+ - type: dot_recall
2581
+ value: 82.20665229442562
2582
+ - type: euclidean_accuracy
2583
+ value: 88.67543757519307
2584
+ - type: euclidean_ap
2585
+ value: 85.4524355531532
2586
+ - type: euclidean_f1
2587
+ value: 77.78729106950081
2588
+ - type: euclidean_precision
2589
+ value: 75.3009009009009
2590
+ - type: euclidean_recall
2591
+ value: 80.44348629504158
2592
+ - type: manhattan_accuracy
2593
+ value: 88.65991384328792
2594
+ - type: manhattan_ap
2595
+ value: 85.43109069046837
2596
+ - type: manhattan_f1
2597
+ value: 77.72639551396425
2598
+ - type: manhattan_precision
2599
+ value: 73.73402417962004
2600
+ - type: manhattan_recall
2601
+ value: 82.17585463504774
2602
+ - type: max_accuracy
2603
+ value: 88.97232894787906
2604
+ - type: max_ap
2605
+ value: 85.9613736469497
2606
+ - type: max_f1
2607
+ value: 78.40216655382532
2608
  ---
2609
+ <h1 align="center">GIST Large Embedding v0</h1>
2610
+
2611
+ *GIST Embedding: Guided In-sample Selection of Training Negatives for Text Embedding*
2612
+
2613
+ The model is fine-tuned on top of the [BAAI/bge-large-en-v1.5](https://huggingface.co/BAAI/bge-large-en-v1.5) using the [MEDI dataset](https://github.com/xlang-ai/instructor-embedding.git) augmented with mined triplets from the [MTEB Classification](https://huggingface.co/mteb) training dataset (excluding data from the Amazon Polarity Classification task).
2614
+
2615
+ The model does not require any instruction for generating embeddings. This means that queries for retrieval tasks can be directly encoded without crafting instructions.
2616
+
2617
+ Technical details of the model will be published shortly.
2618
+
2619
+ # Data
2620
+
2621
+ The dataset used is a compilation of the MEDI dataset and the MTEB Classification training dataset. Third-party datasets may be subject to additional terms and conditions under their associated licenses. A HuggingFace Dataset version of the compiled dataset, and the specific revision used to train the model, is available:
2622
+
2623
+ - Dataset: [avsolatorio/medi-data-mteb_avs_triplets](https://huggingface.co/datasets/avsolatorio/medi-data-mteb_avs_triplets)
2624
+ - Revision: 238a0499b6e6b690cc64ea56fde8461daa8341bb
2625
+
2626
+ The dataset contains a `task_type` key which can be used to select only the mteb classification tasks (prefixed with `mteb_`).
2627
+
2628
+ The **MEDI Dataset** is published in the following paper: [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://arxiv.org/abs/2212.09741).
2629
+
2630
+ The MTEB Benchmark results of the GIST embedding model, compared with the base model, suggest that the fine-tuning dataset has perturbed the model considerably, which resulted in significant improvements in certain tasks while adversely degrading performance in some.
2631
+
2632
+ The retrieval performance for the TRECCOVID task is of note. The fine-tuning dataset does not contain significant knowledge about COVID, which could have caused the observed performance degradation. Further work is currently being undertaken to validate this hypothesis.
2633
+
2634
+ # Usage
2635
+
2636
+ The model can be easily loaded using the Sentence Transformers library.
2637
+
2638
+ ```Python
2639
+ import torch.nn.functional as F
2640
+ from sentence_transformers import SentenceTransformer
2641
+
2642
+ revision = None # Replace with the specific revision to ensure reproducibility in case the model is updated.
2643
+
2644
+ model = SentenceTransformer("avsolatorio/GIST-large-Embedding-v0", revision=revision)
2645
+
2646
+ texts = [
2647
+ "Illustration of the REaLTabFormer model. The left block shows the non-relational tabular data model using GPT-2 with a causal LM head. In contrast, the right block shows how a relational dataset's child table is modeled using a sequence-to-sequence (Seq2Seq) model. The Seq2Seq model uses the observations in the parent table to condition the generation of the observations in the child table. The trained GPT-2 model on the parent table, with weights frozen, is also used as the encoder in the Seq2Seq model.",
2648
+ "Predicting human mobility holds significant practical value, with applications ranging from enhancing disaster risk planning to simulating epidemic spread. In this paper, we present the GeoFormer, a decoder-only transformer model adapted from the GPT architecture to forecast human mobility.",
2649
+ "As the economies of Southeast Asia continue adopting digital technologies, policy makers increasingly ask how to prepare the workforce for emerging labor demands. However, little is known about the skills that workers need to adapt to these changes"
2650
+ ]
2651
+
2652
+ # Compute embeddings
2653
+ embeddings = model.encode(texts, convert_to_tensor=True)
2654
+
2655
+ # Compute cosine-similarity for each pair of sentences
2656
+ scores = F.cosine_similarity(embeddings.unsqueeze(1), embeddings.unsqueeze(0), dim=-1)
2657
+
2658
+ print(scores.cpu().numpy())
2659
+ ```
2660
+
2661
+ # Training Parameters
2662
+
2663
+ Below are the training parameters used to fine-tune the model:
2664
+
2665
+ ```
2666
+ Epochs = 40
2667
+ Warmup ratio = 0.1
2668
+ Learning rate = 5e-6
2669
+ Batch size = 16
2670
+ Checkpoint step = 171000
2671
+ Contrastive loss temperature = 0.01
2672
+ ```
2673
+
2674
+ Specific training details and strategies will be published shortly.
2675
+
2676
+ # Evaluation
2677
+
2678
+ The model was evaluated using the [MTEB Evaluation](https://huggingface.co/mteb) suite.
2679
+
2680
+
2681
+ # Acknowledgements
2682
+
2683
+ This work is supported by the "KCP IV - Exploring Data Use in the Development Economics Literature using Large Language Models (AI and LLMs)" project funded by the [Knowledge for Change Program (KCP)](https://www.worldbank.org/en/programs/knowledge-for-change) of the World Bank - RA-P503405-RESE-TF0C3444.
2684
+
2685
+ The findings, interpretations, and conclusions expressed in this material are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.
commit-info.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"repo_id": "avsolatorio/02-100-11-1-2-2-0-0-cls-normed-1024-512_GIST_BAAI_bge-large-en-v1.5-20240212001152-latest", "commit_message": "{\"loss\": 0.1359, \"learning_rate\": 1.880476475349263e-06, \"epoch\": 1.5, \"step\": 171000}"}
config.json ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "avsolatorio/GIST-large-Embedding-v0",
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "gradient_checkpointing": false,
9
+ "hidden_act": "gelu",
10
+ "hidden_dropout_prob": 0.1,
11
+ "hidden_size": 1024,
12
+ "id2label": {
13
+ "0": "LABEL_0"
14
+ },
15
+ "initializer_range": 0.02,
16
+ "intermediate_size": 4096,
17
+ "label2id": {
18
+ "LABEL_0": 0
19
+ },
20
+ "layer_norm_eps": 1e-12,
21
+ "max_position_embeddings": 512,
22
+ "model_type": "bert",
23
+ "num_attention_heads": 16,
24
+ "num_hidden_layers": 24,
25
+ "pad_token_id": 0,
26
+ "position_embedding_type": "absolute",
27
+ "torch_dtype": "float32",
28
+ "transformers_version": "4.37.2",
29
+ "type_vocab_size": 2,
30
+ "use_cache": true,
31
+ "vocab_size": 30522
32
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "__version__": {
3
+ "sentence_transformers": "2.2.2",
4
+ "transformers": "4.28.1",
5
+ "pytorch": "1.13.0+cu117"
6
+ }
7
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c0402c8c231bed413465ccd6d8c09171f78774a540b42e0b381401c2a247b40a
3
+ size 1340612432
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": true
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,57 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_basic_tokenize": true,
47
+ "do_lower_case": true,
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "never_split": null,
51
+ "pad_token": "[PAD]",
52
+ "sep_token": "[SEP]",
53
+ "strip_accents": null,
54
+ "tokenize_chinese_chars": true,
55
+ "tokenizer_class": "BertTokenizer",
56
+ "unk_token": "[UNK]"
57
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff