intfloat commited on
Commit
bdaaca6
1 Parent(s): 276398b

upload model weights

Browse files
Files changed (7) hide show
  1. README.md +2672 -0
  2. config.json +25 -0
  3. pytorch_model.bin +3 -0
  4. special_tokens_map.json +7 -0
  5. tokenizer.json +0 -0
  6. tokenizer_config.json +13 -0
  7. vocab.txt +0 -0
README.md ADDED
@@ -0,0 +1,2672 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - mteb
4
+ model-index:
5
+ - name: e5-large-v2
6
+ results:
7
+ - task:
8
+ type: Classification
9
+ dataset:
10
+ type: mteb/amazon_counterfactual
11
+ name: MTEB AmazonCounterfactualClassification (en)
12
+ config: en
13
+ split: test
14
+ revision: e8379541af4e31359cca9fbcf4b00f2671dba205
15
+ metrics:
16
+ - type: accuracy
17
+ value: 79.22388059701493
18
+ - type: ap
19
+ value: 43.20816505595132
20
+ - type: f1
21
+ value: 73.27811303522058
22
+ - task:
23
+ type: Classification
24
+ dataset:
25
+ type: mteb/amazon_polarity
26
+ name: MTEB AmazonPolarityClassification
27
+ config: default
28
+ split: test
29
+ revision: e2d317d38cd51312af73b3d32a06d1a08b442046
30
+ metrics:
31
+ - type: accuracy
32
+ value: 93.748325
33
+ - type: ap
34
+ value: 90.72534979701297
35
+ - type: f1
36
+ value: 93.73895874282185
37
+ - task:
38
+ type: Classification
39
+ dataset:
40
+ type: mteb/amazon_reviews_multi
41
+ name: MTEB AmazonReviewsClassification (en)
42
+ config: en
43
+ split: test
44
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
45
+ metrics:
46
+ - type: accuracy
47
+ value: 48.612
48
+ - type: f1
49
+ value: 47.61157345898393
50
+ - task:
51
+ type: Retrieval
52
+ dataset:
53
+ type: arguana
54
+ name: MTEB ArguAna
55
+ config: default
56
+ split: test
57
+ revision: None
58
+ metrics:
59
+ - type: map_at_1
60
+ value: 23.541999999999998
61
+ - type: map_at_10
62
+ value: 38.208
63
+ - type: map_at_100
64
+ value: 39.417
65
+ - type: map_at_1000
66
+ value: 39.428999999999995
67
+ - type: map_at_3
68
+ value: 33.95
69
+ - type: map_at_5
70
+ value: 36.329
71
+ - type: mrr_at_1
72
+ value: 23.755000000000003
73
+ - type: mrr_at_10
74
+ value: 38.288
75
+ - type: mrr_at_100
76
+ value: 39.511
77
+ - type: mrr_at_1000
78
+ value: 39.523
79
+ - type: mrr_at_3
80
+ value: 34.009
81
+ - type: mrr_at_5
82
+ value: 36.434
83
+ - type: ndcg_at_1
84
+ value: 23.541999999999998
85
+ - type: ndcg_at_10
86
+ value: 46.417
87
+ - type: ndcg_at_100
88
+ value: 51.812000000000005
89
+ - type: ndcg_at_1000
90
+ value: 52.137
91
+ - type: ndcg_at_3
92
+ value: 37.528
93
+ - type: ndcg_at_5
94
+ value: 41.81
95
+ - type: precision_at_1
96
+ value: 23.541999999999998
97
+ - type: precision_at_10
98
+ value: 7.269
99
+ - type: precision_at_100
100
+ value: 0.9690000000000001
101
+ - type: precision_at_1000
102
+ value: 0.099
103
+ - type: precision_at_3
104
+ value: 15.979
105
+ - type: precision_at_5
106
+ value: 11.664
107
+ - type: recall_at_1
108
+ value: 23.541999999999998
109
+ - type: recall_at_10
110
+ value: 72.688
111
+ - type: recall_at_100
112
+ value: 96.871
113
+ - type: recall_at_1000
114
+ value: 99.431
115
+ - type: recall_at_3
116
+ value: 47.937000000000005
117
+ - type: recall_at_5
118
+ value: 58.321
119
+ - task:
120
+ type: Clustering
121
+ dataset:
122
+ type: mteb/arxiv-clustering-p2p
123
+ name: MTEB ArxivClusteringP2P
124
+ config: default
125
+ split: test
126
+ revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
127
+ metrics:
128
+ - type: v_measure
129
+ value: 45.546499570522094
130
+ - task:
131
+ type: Clustering
132
+ dataset:
133
+ type: mteb/arxiv-clustering-s2s
134
+ name: MTEB ArxivClusteringS2S
135
+ config: default
136
+ split: test
137
+ revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
138
+ metrics:
139
+ - type: v_measure
140
+ value: 41.01607489943561
141
+ - task:
142
+ type: Reranking
143
+ dataset:
144
+ type: mteb/askubuntudupquestions-reranking
145
+ name: MTEB AskUbuntuDupQuestions
146
+ config: default
147
+ split: test
148
+ revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
149
+ metrics:
150
+ - type: map
151
+ value: 59.616107510107774
152
+ - type: mrr
153
+ value: 72.75106626214661
154
+ - task:
155
+ type: STS
156
+ dataset:
157
+ type: mteb/biosses-sts
158
+ name: MTEB BIOSSES
159
+ config: default
160
+ split: test
161
+ revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
162
+ metrics:
163
+ - type: cos_sim_pearson
164
+ value: 84.33018094733868
165
+ - type: cos_sim_spearman
166
+ value: 83.60190492611737
167
+ - type: euclidean_pearson
168
+ value: 82.1492450218961
169
+ - type: euclidean_spearman
170
+ value: 82.70308926526991
171
+ - type: manhattan_pearson
172
+ value: 81.93959600076842
173
+ - type: manhattan_spearman
174
+ value: 82.73260801016369
175
+ - task:
176
+ type: Classification
177
+ dataset:
178
+ type: mteb/banking77
179
+ name: MTEB Banking77Classification
180
+ config: default
181
+ split: test
182
+ revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
183
+ metrics:
184
+ - type: accuracy
185
+ value: 84.54545454545455
186
+ - type: f1
187
+ value: 84.49582530928923
188
+ - task:
189
+ type: Clustering
190
+ dataset:
191
+ type: mteb/biorxiv-clustering-p2p
192
+ name: MTEB BiorxivClusteringP2P
193
+ config: default
194
+ split: test
195
+ revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
196
+ metrics:
197
+ - type: v_measure
198
+ value: 37.362725540120096
199
+ - task:
200
+ type: Clustering
201
+ dataset:
202
+ type: mteb/biorxiv-clustering-s2s
203
+ name: MTEB BiorxivClusteringS2S
204
+ config: default
205
+ split: test
206
+ revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
207
+ metrics:
208
+ - type: v_measure
209
+ value: 34.849509608178145
210
+ - task:
211
+ type: Retrieval
212
+ dataset:
213
+ type: BeIR/cqadupstack
214
+ name: MTEB CQADupstackAndroidRetrieval
215
+ config: default
216
+ split: test
217
+ revision: None
218
+ metrics:
219
+ - type: map_at_1
220
+ value: 31.502999999999997
221
+ - type: map_at_10
222
+ value: 43.323
223
+ - type: map_at_100
224
+ value: 44.708999999999996
225
+ - type: map_at_1000
226
+ value: 44.838
227
+ - type: map_at_3
228
+ value: 38.987
229
+ - type: map_at_5
230
+ value: 41.516999999999996
231
+ - type: mrr_at_1
232
+ value: 38.769999999999996
233
+ - type: mrr_at_10
234
+ value: 49.13
235
+ - type: mrr_at_100
236
+ value: 49.697
237
+ - type: mrr_at_1000
238
+ value: 49.741
239
+ - type: mrr_at_3
240
+ value: 45.804
241
+ - type: mrr_at_5
242
+ value: 47.842
243
+ - type: ndcg_at_1
244
+ value: 38.769999999999996
245
+ - type: ndcg_at_10
246
+ value: 50.266999999999996
247
+ - type: ndcg_at_100
248
+ value: 54.967
249
+ - type: ndcg_at_1000
250
+ value: 56.976000000000006
251
+ - type: ndcg_at_3
252
+ value: 43.823
253
+ - type: ndcg_at_5
254
+ value: 47.12
255
+ - type: precision_at_1
256
+ value: 38.769999999999996
257
+ - type: precision_at_10
258
+ value: 10.057
259
+ - type: precision_at_100
260
+ value: 1.554
261
+ - type: precision_at_1000
262
+ value: 0.202
263
+ - type: precision_at_3
264
+ value: 21.125
265
+ - type: precision_at_5
266
+ value: 15.851
267
+ - type: recall_at_1
268
+ value: 31.502999999999997
269
+ - type: recall_at_10
270
+ value: 63.715999999999994
271
+ - type: recall_at_100
272
+ value: 83.61800000000001
273
+ - type: recall_at_1000
274
+ value: 96.63199999999999
275
+ - type: recall_at_3
276
+ value: 45.403
277
+ - type: recall_at_5
278
+ value: 54.481
279
+ - task:
280
+ type: Retrieval
281
+ dataset:
282
+ type: BeIR/cqadupstack
283
+ name: MTEB CQADupstackEnglishRetrieval
284
+ config: default
285
+ split: test
286
+ revision: None
287
+ metrics:
288
+ - type: map_at_1
289
+ value: 27.833000000000002
290
+ - type: map_at_10
291
+ value: 37.330999999999996
292
+ - type: map_at_100
293
+ value: 38.580999999999996
294
+ - type: map_at_1000
295
+ value: 38.708
296
+ - type: map_at_3
297
+ value: 34.713
298
+ - type: map_at_5
299
+ value: 36.104
300
+ - type: mrr_at_1
301
+ value: 35.223
302
+ - type: mrr_at_10
303
+ value: 43.419000000000004
304
+ - type: mrr_at_100
305
+ value: 44.198
306
+ - type: mrr_at_1000
307
+ value: 44.249
308
+ - type: mrr_at_3
309
+ value: 41.614000000000004
310
+ - type: mrr_at_5
311
+ value: 42.553000000000004
312
+ - type: ndcg_at_1
313
+ value: 35.223
314
+ - type: ndcg_at_10
315
+ value: 42.687999999999995
316
+ - type: ndcg_at_100
317
+ value: 47.447
318
+ - type: ndcg_at_1000
319
+ value: 49.701
320
+ - type: ndcg_at_3
321
+ value: 39.162
322
+ - type: ndcg_at_5
323
+ value: 40.557
324
+ - type: precision_at_1
325
+ value: 35.223
326
+ - type: precision_at_10
327
+ value: 7.962
328
+ - type: precision_at_100
329
+ value: 1.304
330
+ - type: precision_at_1000
331
+ value: 0.18
332
+ - type: precision_at_3
333
+ value: 19.023
334
+ - type: precision_at_5
335
+ value: 13.184999999999999
336
+ - type: recall_at_1
337
+ value: 27.833000000000002
338
+ - type: recall_at_10
339
+ value: 51.881
340
+ - type: recall_at_100
341
+ value: 72.04
342
+ - type: recall_at_1000
343
+ value: 86.644
344
+ - type: recall_at_3
345
+ value: 40.778
346
+ - type: recall_at_5
347
+ value: 45.176
348
+ - task:
349
+ type: Retrieval
350
+ dataset:
351
+ type: BeIR/cqadupstack
352
+ name: MTEB CQADupstackGamingRetrieval
353
+ config: default
354
+ split: test
355
+ revision: None
356
+ metrics:
357
+ - type: map_at_1
358
+ value: 38.175
359
+ - type: map_at_10
360
+ value: 51.174
361
+ - type: map_at_100
362
+ value: 52.26499999999999
363
+ - type: map_at_1000
364
+ value: 52.315999999999995
365
+ - type: map_at_3
366
+ value: 47.897
367
+ - type: map_at_5
368
+ value: 49.703
369
+ - type: mrr_at_1
370
+ value: 43.448
371
+ - type: mrr_at_10
372
+ value: 54.505
373
+ - type: mrr_at_100
374
+ value: 55.216
375
+ - type: mrr_at_1000
376
+ value: 55.242000000000004
377
+ - type: mrr_at_3
378
+ value: 51.98500000000001
379
+ - type: mrr_at_5
380
+ value: 53.434000000000005
381
+ - type: ndcg_at_1
382
+ value: 43.448
383
+ - type: ndcg_at_10
384
+ value: 57.282
385
+ - type: ndcg_at_100
386
+ value: 61.537
387
+ - type: ndcg_at_1000
388
+ value: 62.546
389
+ - type: ndcg_at_3
390
+ value: 51.73799999999999
391
+ - type: ndcg_at_5
392
+ value: 54.324
393
+ - type: precision_at_1
394
+ value: 43.448
395
+ - type: precision_at_10
396
+ value: 9.292
397
+ - type: precision_at_100
398
+ value: 1.233
399
+ - type: precision_at_1000
400
+ value: 0.136
401
+ - type: precision_at_3
402
+ value: 23.218
403
+ - type: precision_at_5
404
+ value: 15.887
405
+ - type: recall_at_1
406
+ value: 38.175
407
+ - type: recall_at_10
408
+ value: 72.00999999999999
409
+ - type: recall_at_100
410
+ value: 90.155
411
+ - type: recall_at_1000
412
+ value: 97.257
413
+ - type: recall_at_3
414
+ value: 57.133
415
+ - type: recall_at_5
416
+ value: 63.424
417
+ - task:
418
+ type: Retrieval
419
+ dataset:
420
+ type: BeIR/cqadupstack
421
+ name: MTEB CQADupstackGisRetrieval
422
+ config: default
423
+ split: test
424
+ revision: None
425
+ metrics:
426
+ - type: map_at_1
427
+ value: 22.405
428
+ - type: map_at_10
429
+ value: 30.043
430
+ - type: map_at_100
431
+ value: 31.191000000000003
432
+ - type: map_at_1000
433
+ value: 31.275
434
+ - type: map_at_3
435
+ value: 27.034000000000002
436
+ - type: map_at_5
437
+ value: 28.688000000000002
438
+ - type: mrr_at_1
439
+ value: 24.068
440
+ - type: mrr_at_10
441
+ value: 31.993
442
+ - type: mrr_at_100
443
+ value: 32.992
444
+ - type: mrr_at_1000
445
+ value: 33.050000000000004
446
+ - type: mrr_at_3
447
+ value: 28.964000000000002
448
+ - type: mrr_at_5
449
+ value: 30.653000000000002
450
+ - type: ndcg_at_1
451
+ value: 24.068
452
+ - type: ndcg_at_10
453
+ value: 35.198
454
+ - type: ndcg_at_100
455
+ value: 40.709
456
+ - type: ndcg_at_1000
457
+ value: 42.855
458
+ - type: ndcg_at_3
459
+ value: 29.139
460
+ - type: ndcg_at_5
461
+ value: 32.045
462
+ - type: precision_at_1
463
+ value: 24.068
464
+ - type: precision_at_10
465
+ value: 5.65
466
+ - type: precision_at_100
467
+ value: 0.885
468
+ - type: precision_at_1000
469
+ value: 0.11199999999999999
470
+ - type: precision_at_3
471
+ value: 12.279
472
+ - type: precision_at_5
473
+ value: 8.994
474
+ - type: recall_at_1
475
+ value: 22.405
476
+ - type: recall_at_10
477
+ value: 49.391
478
+ - type: recall_at_100
479
+ value: 74.53699999999999
480
+ - type: recall_at_1000
481
+ value: 90.605
482
+ - type: recall_at_3
483
+ value: 33.126
484
+ - type: recall_at_5
485
+ value: 40.073
486
+ - task:
487
+ type: Retrieval
488
+ dataset:
489
+ type: BeIR/cqadupstack
490
+ name: MTEB CQADupstackMathematicaRetrieval
491
+ config: default
492
+ split: test
493
+ revision: None
494
+ metrics:
495
+ - type: map_at_1
496
+ value: 13.309999999999999
497
+ - type: map_at_10
498
+ value: 20.688000000000002
499
+ - type: map_at_100
500
+ value: 22.022
501
+ - type: map_at_1000
502
+ value: 22.152
503
+ - type: map_at_3
504
+ value: 17.954
505
+ - type: map_at_5
506
+ value: 19.439
507
+ - type: mrr_at_1
508
+ value: 16.294
509
+ - type: mrr_at_10
510
+ value: 24.479
511
+ - type: mrr_at_100
512
+ value: 25.515
513
+ - type: mrr_at_1000
514
+ value: 25.593
515
+ - type: mrr_at_3
516
+ value: 21.642
517
+ - type: mrr_at_5
518
+ value: 23.189999999999998
519
+ - type: ndcg_at_1
520
+ value: 16.294
521
+ - type: ndcg_at_10
522
+ value: 25.833000000000002
523
+ - type: ndcg_at_100
524
+ value: 32.074999999999996
525
+ - type: ndcg_at_1000
526
+ value: 35.083
527
+ - type: ndcg_at_3
528
+ value: 20.493
529
+ - type: ndcg_at_5
530
+ value: 22.949
531
+ - type: precision_at_1
532
+ value: 16.294
533
+ - type: precision_at_10
534
+ value: 5.112
535
+ - type: precision_at_100
536
+ value: 0.96
537
+ - type: precision_at_1000
538
+ value: 0.134
539
+ - type: precision_at_3
540
+ value: 9.908999999999999
541
+ - type: precision_at_5
542
+ value: 7.587000000000001
543
+ - type: recall_at_1
544
+ value: 13.309999999999999
545
+ - type: recall_at_10
546
+ value: 37.851
547
+ - type: recall_at_100
548
+ value: 64.835
549
+ - type: recall_at_1000
550
+ value: 86.334
551
+ - type: recall_at_3
552
+ value: 23.493
553
+ - type: recall_at_5
554
+ value: 29.528
555
+ - task:
556
+ type: Retrieval
557
+ dataset:
558
+ type: BeIR/cqadupstack
559
+ name: MTEB CQADupstackPhysicsRetrieval
560
+ config: default
561
+ split: test
562
+ revision: None
563
+ metrics:
564
+ - type: map_at_1
565
+ value: 25.857999999999997
566
+ - type: map_at_10
567
+ value: 35.503
568
+ - type: map_at_100
569
+ value: 36.957
570
+ - type: map_at_1000
571
+ value: 37.065
572
+ - type: map_at_3
573
+ value: 32.275999999999996
574
+ - type: map_at_5
575
+ value: 34.119
576
+ - type: mrr_at_1
577
+ value: 31.954
578
+ - type: mrr_at_10
579
+ value: 40.851
580
+ - type: mrr_at_100
581
+ value: 41.863
582
+ - type: mrr_at_1000
583
+ value: 41.900999999999996
584
+ - type: mrr_at_3
585
+ value: 38.129999999999995
586
+ - type: mrr_at_5
587
+ value: 39.737
588
+ - type: ndcg_at_1
589
+ value: 31.954
590
+ - type: ndcg_at_10
591
+ value: 41.343999999999994
592
+ - type: ndcg_at_100
593
+ value: 47.397
594
+ - type: ndcg_at_1000
595
+ value: 49.501
596
+ - type: ndcg_at_3
597
+ value: 36.047000000000004
598
+ - type: ndcg_at_5
599
+ value: 38.639
600
+ - type: precision_at_1
601
+ value: 31.954
602
+ - type: precision_at_10
603
+ value: 7.68
604
+ - type: precision_at_100
605
+ value: 1.247
606
+ - type: precision_at_1000
607
+ value: 0.16199999999999998
608
+ - type: precision_at_3
609
+ value: 17.132
610
+ - type: precision_at_5
611
+ value: 12.589
612
+ - type: recall_at_1
613
+ value: 25.857999999999997
614
+ - type: recall_at_10
615
+ value: 53.43599999999999
616
+ - type: recall_at_100
617
+ value: 78.82400000000001
618
+ - type: recall_at_1000
619
+ value: 92.78999999999999
620
+ - type: recall_at_3
621
+ value: 38.655
622
+ - type: recall_at_5
623
+ value: 45.216
624
+ - task:
625
+ type: Retrieval
626
+ dataset:
627
+ type: BeIR/cqadupstack
628
+ name: MTEB CQADupstackProgrammersRetrieval
629
+ config: default
630
+ split: test
631
+ revision: None
632
+ metrics:
633
+ - type: map_at_1
634
+ value: 24.709
635
+ - type: map_at_10
636
+ value: 34.318
637
+ - type: map_at_100
638
+ value: 35.657
639
+ - type: map_at_1000
640
+ value: 35.783
641
+ - type: map_at_3
642
+ value: 31.326999999999998
643
+ - type: map_at_5
644
+ value: 33.021
645
+ - type: mrr_at_1
646
+ value: 30.137000000000004
647
+ - type: mrr_at_10
648
+ value: 39.093
649
+ - type: mrr_at_100
650
+ value: 39.992
651
+ - type: mrr_at_1000
652
+ value: 40.056999999999995
653
+ - type: mrr_at_3
654
+ value: 36.606
655
+ - type: mrr_at_5
656
+ value: 37.861
657
+ - type: ndcg_at_1
658
+ value: 30.137000000000004
659
+ - type: ndcg_at_10
660
+ value: 39.974
661
+ - type: ndcg_at_100
662
+ value: 45.647999999999996
663
+ - type: ndcg_at_1000
664
+ value: 48.259
665
+ - type: ndcg_at_3
666
+ value: 35.028
667
+ - type: ndcg_at_5
668
+ value: 37.175999999999995
669
+ - type: precision_at_1
670
+ value: 30.137000000000004
671
+ - type: precision_at_10
672
+ value: 7.363
673
+ - type: precision_at_100
674
+ value: 1.184
675
+ - type: precision_at_1000
676
+ value: 0.161
677
+ - type: precision_at_3
678
+ value: 16.857
679
+ - type: precision_at_5
680
+ value: 11.963
681
+ - type: recall_at_1
682
+ value: 24.709
683
+ - type: recall_at_10
684
+ value: 52.087
685
+ - type: recall_at_100
686
+ value: 76.125
687
+ - type: recall_at_1000
688
+ value: 93.82300000000001
689
+ - type: recall_at_3
690
+ value: 38.149
691
+ - type: recall_at_5
692
+ value: 43.984
693
+ - task:
694
+ type: Retrieval
695
+ dataset:
696
+ type: BeIR/cqadupstack
697
+ name: MTEB CQADupstackRetrieval
698
+ config: default
699
+ split: test
700
+ revision: None
701
+ metrics:
702
+ - type: map_at_1
703
+ value: 23.40791666666667
704
+ - type: map_at_10
705
+ value: 32.458083333333335
706
+ - type: map_at_100
707
+ value: 33.691916666666664
708
+ - type: map_at_1000
709
+ value: 33.81191666666666
710
+ - type: map_at_3
711
+ value: 29.51625
712
+ - type: map_at_5
713
+ value: 31.168083333333335
714
+ - type: mrr_at_1
715
+ value: 27.96591666666666
716
+ - type: mrr_at_10
717
+ value: 36.528583333333344
718
+ - type: mrr_at_100
719
+ value: 37.404
720
+ - type: mrr_at_1000
721
+ value: 37.464333333333336
722
+ - type: mrr_at_3
723
+ value: 33.92883333333333
724
+ - type: mrr_at_5
725
+ value: 35.41933333333333
726
+ - type: ndcg_at_1
727
+ value: 27.96591666666666
728
+ - type: ndcg_at_10
729
+ value: 37.89141666666666
730
+ - type: ndcg_at_100
731
+ value: 43.23066666666666
732
+ - type: ndcg_at_1000
733
+ value: 45.63258333333333
734
+ - type: ndcg_at_3
735
+ value: 32.811249999999994
736
+ - type: ndcg_at_5
737
+ value: 35.22566666666667
738
+ - type: precision_at_1
739
+ value: 27.96591666666666
740
+ - type: precision_at_10
741
+ value: 6.834083333333332
742
+ - type: precision_at_100
743
+ value: 1.12225
744
+ - type: precision_at_1000
745
+ value: 0.15241666666666667
746
+ - type: precision_at_3
747
+ value: 15.264333333333335
748
+ - type: precision_at_5
749
+ value: 11.039416666666666
750
+ - type: recall_at_1
751
+ value: 23.40791666666667
752
+ - type: recall_at_10
753
+ value: 49.927083333333336
754
+ - type: recall_at_100
755
+ value: 73.44641666666668
756
+ - type: recall_at_1000
757
+ value: 90.19950000000001
758
+ - type: recall_at_3
759
+ value: 35.88341666666667
760
+ - type: recall_at_5
761
+ value: 42.061249999999994
762
+ - task:
763
+ type: Retrieval
764
+ dataset:
765
+ type: BeIR/cqadupstack
766
+ name: MTEB CQADupstackStatsRetrieval
767
+ config: default
768
+ split: test
769
+ revision: None
770
+ metrics:
771
+ - type: map_at_1
772
+ value: 19.592000000000002
773
+ - type: map_at_10
774
+ value: 26.895999999999997
775
+ - type: map_at_100
776
+ value: 27.921000000000003
777
+ - type: map_at_1000
778
+ value: 28.02
779
+ - type: map_at_3
780
+ value: 24.883
781
+ - type: map_at_5
782
+ value: 25.812
783
+ - type: mrr_at_1
784
+ value: 22.698999999999998
785
+ - type: mrr_at_10
786
+ value: 29.520999999999997
787
+ - type: mrr_at_100
788
+ value: 30.458000000000002
789
+ - type: mrr_at_1000
790
+ value: 30.526999999999997
791
+ - type: mrr_at_3
792
+ value: 27.633000000000003
793
+ - type: mrr_at_5
794
+ value: 28.483999999999998
795
+ - type: ndcg_at_1
796
+ value: 22.698999999999998
797
+ - type: ndcg_at_10
798
+ value: 31.061
799
+ - type: ndcg_at_100
800
+ value: 36.398
801
+ - type: ndcg_at_1000
802
+ value: 38.89
803
+ - type: ndcg_at_3
804
+ value: 27.149
805
+ - type: ndcg_at_5
806
+ value: 28.627000000000002
807
+ - type: precision_at_1
808
+ value: 22.698999999999998
809
+ - type: precision_at_10
810
+ value: 5.106999999999999
811
+ - type: precision_at_100
812
+ value: 0.857
813
+ - type: precision_at_1000
814
+ value: 0.11499999999999999
815
+ - type: precision_at_3
816
+ value: 11.963
817
+ - type: precision_at_5
818
+ value: 8.221
819
+ - type: recall_at_1
820
+ value: 19.592000000000002
821
+ - type: recall_at_10
822
+ value: 41.329
823
+ - type: recall_at_100
824
+ value: 66.094
825
+ - type: recall_at_1000
826
+ value: 84.511
827
+ - type: recall_at_3
828
+ value: 30.61
829
+ - type: recall_at_5
830
+ value: 34.213
831
+ - task:
832
+ type: Retrieval
833
+ dataset:
834
+ type: BeIR/cqadupstack
835
+ name: MTEB CQADupstackTexRetrieval
836
+ config: default
837
+ split: test
838
+ revision: None
839
+ metrics:
840
+ - type: map_at_1
841
+ value: 14.71
842
+ - type: map_at_10
843
+ value: 20.965
844
+ - type: map_at_100
845
+ value: 21.994
846
+ - type: map_at_1000
847
+ value: 22.133
848
+ - type: map_at_3
849
+ value: 18.741
850
+ - type: map_at_5
851
+ value: 19.951
852
+ - type: mrr_at_1
853
+ value: 18.307000000000002
854
+ - type: mrr_at_10
855
+ value: 24.66
856
+ - type: mrr_at_100
857
+ value: 25.540000000000003
858
+ - type: mrr_at_1000
859
+ value: 25.629
860
+ - type: mrr_at_3
861
+ value: 22.511
862
+ - type: mrr_at_5
863
+ value: 23.72
864
+ - type: ndcg_at_1
865
+ value: 18.307000000000002
866
+ - type: ndcg_at_10
867
+ value: 25.153
868
+ - type: ndcg_at_100
869
+ value: 30.229
870
+ - type: ndcg_at_1000
871
+ value: 33.623
872
+ - type: ndcg_at_3
873
+ value: 21.203
874
+ - type: ndcg_at_5
875
+ value: 23.006999999999998
876
+ - type: precision_at_1
877
+ value: 18.307000000000002
878
+ - type: precision_at_10
879
+ value: 4.725
880
+ - type: precision_at_100
881
+ value: 0.8659999999999999
882
+ - type: precision_at_1000
883
+ value: 0.133
884
+ - type: precision_at_3
885
+ value: 10.14
886
+ - type: precision_at_5
887
+ value: 7.481
888
+ - type: recall_at_1
889
+ value: 14.71
890
+ - type: recall_at_10
891
+ value: 34.087
892
+ - type: recall_at_100
893
+ value: 57.147999999999996
894
+ - type: recall_at_1000
895
+ value: 81.777
896
+ - type: recall_at_3
897
+ value: 22.996
898
+ - type: recall_at_5
899
+ value: 27.73
900
+ - task:
901
+ type: Retrieval
902
+ dataset:
903
+ type: BeIR/cqadupstack
904
+ name: MTEB CQADupstackUnixRetrieval
905
+ config: default
906
+ split: test
907
+ revision: None
908
+ metrics:
909
+ - type: map_at_1
910
+ value: 23.472
911
+ - type: map_at_10
912
+ value: 32.699
913
+ - type: map_at_100
914
+ value: 33.867000000000004
915
+ - type: map_at_1000
916
+ value: 33.967000000000006
917
+ - type: map_at_3
918
+ value: 29.718
919
+ - type: map_at_5
920
+ value: 31.345
921
+ - type: mrr_at_1
922
+ value: 28.265
923
+ - type: mrr_at_10
924
+ value: 36.945
925
+ - type: mrr_at_100
926
+ value: 37.794
927
+ - type: mrr_at_1000
928
+ value: 37.857
929
+ - type: mrr_at_3
930
+ value: 34.266000000000005
931
+ - type: mrr_at_5
932
+ value: 35.768
933
+ - type: ndcg_at_1
934
+ value: 28.265
935
+ - type: ndcg_at_10
936
+ value: 38.35
937
+ - type: ndcg_at_100
938
+ value: 43.739
939
+ - type: ndcg_at_1000
940
+ value: 46.087
941
+ - type: ndcg_at_3
942
+ value: 33.004
943
+ - type: ndcg_at_5
944
+ value: 35.411
945
+ - type: precision_at_1
946
+ value: 28.265
947
+ - type: precision_at_10
948
+ value: 6.715999999999999
949
+ - type: precision_at_100
950
+ value: 1.059
951
+ - type: precision_at_1000
952
+ value: 0.13799999999999998
953
+ - type: precision_at_3
954
+ value: 15.299
955
+ - type: precision_at_5
956
+ value: 10.951
957
+ - type: recall_at_1
958
+ value: 23.472
959
+ - type: recall_at_10
960
+ value: 51.413
961
+ - type: recall_at_100
962
+ value: 75.17
963
+ - type: recall_at_1000
964
+ value: 91.577
965
+ - type: recall_at_3
966
+ value: 36.651
967
+ - type: recall_at_5
968
+ value: 42.814
969
+ - task:
970
+ type: Retrieval
971
+ dataset:
972
+ type: BeIR/cqadupstack
973
+ name: MTEB CQADupstackWebmastersRetrieval
974
+ config: default
975
+ split: test
976
+ revision: None
977
+ metrics:
978
+ - type: map_at_1
979
+ value: 23.666
980
+ - type: map_at_10
981
+ value: 32.963
982
+ - type: map_at_100
983
+ value: 34.544999999999995
984
+ - type: map_at_1000
985
+ value: 34.792
986
+ - type: map_at_3
987
+ value: 29.74
988
+ - type: map_at_5
989
+ value: 31.5
990
+ - type: mrr_at_1
991
+ value: 29.051
992
+ - type: mrr_at_10
993
+ value: 38.013000000000005
994
+ - type: mrr_at_100
995
+ value: 38.997
996
+ - type: mrr_at_1000
997
+ value: 39.055
998
+ - type: mrr_at_3
999
+ value: 34.947
1000
+ - type: mrr_at_5
1001
+ value: 36.815
1002
+ - type: ndcg_at_1
1003
+ value: 29.051
1004
+ - type: ndcg_at_10
1005
+ value: 39.361000000000004
1006
+ - type: ndcg_at_100
1007
+ value: 45.186
1008
+ - type: ndcg_at_1000
1009
+ value: 47.867
1010
+ - type: ndcg_at_3
1011
+ value: 33.797
1012
+ - type: ndcg_at_5
1013
+ value: 36.456
1014
+ - type: precision_at_1
1015
+ value: 29.051
1016
+ - type: precision_at_10
1017
+ value: 7.668
1018
+ - type: precision_at_100
1019
+ value: 1.532
1020
+ - type: precision_at_1000
1021
+ value: 0.247
1022
+ - type: precision_at_3
1023
+ value: 15.876000000000001
1024
+ - type: precision_at_5
1025
+ value: 11.779
1026
+ - type: recall_at_1
1027
+ value: 23.666
1028
+ - type: recall_at_10
1029
+ value: 51.858000000000004
1030
+ - type: recall_at_100
1031
+ value: 77.805
1032
+ - type: recall_at_1000
1033
+ value: 94.504
1034
+ - type: recall_at_3
1035
+ value: 36.207
1036
+ - type: recall_at_5
1037
+ value: 43.094
1038
+ - task:
1039
+ type: Retrieval
1040
+ dataset:
1041
+ type: BeIR/cqadupstack
1042
+ name: MTEB CQADupstackWordpressRetrieval
1043
+ config: default
1044
+ split: test
1045
+ revision: None
1046
+ metrics:
1047
+ - type: map_at_1
1048
+ value: 15.662
1049
+ - type: map_at_10
1050
+ value: 23.594
1051
+ - type: map_at_100
1052
+ value: 24.593999999999998
1053
+ - type: map_at_1000
1054
+ value: 24.694
1055
+ - type: map_at_3
1056
+ value: 20.925
1057
+ - type: map_at_5
1058
+ value: 22.817999999999998
1059
+ - type: mrr_at_1
1060
+ value: 17.375
1061
+ - type: mrr_at_10
1062
+ value: 25.734
1063
+ - type: mrr_at_100
1064
+ value: 26.586
1065
+ - type: mrr_at_1000
1066
+ value: 26.671
1067
+ - type: mrr_at_3
1068
+ value: 23.044
1069
+ - type: mrr_at_5
1070
+ value: 24.975
1071
+ - type: ndcg_at_1
1072
+ value: 17.375
1073
+ - type: ndcg_at_10
1074
+ value: 28.186
1075
+ - type: ndcg_at_100
1076
+ value: 33.436
1077
+ - type: ndcg_at_1000
1078
+ value: 36.203
1079
+ - type: ndcg_at_3
1080
+ value: 23.152
1081
+ - type: ndcg_at_5
1082
+ value: 26.397
1083
+ - type: precision_at_1
1084
+ value: 17.375
1085
+ - type: precision_at_10
1086
+ value: 4.677
1087
+ - type: precision_at_100
1088
+ value: 0.786
1089
+ - type: precision_at_1000
1090
+ value: 0.109
1091
+ - type: precision_at_3
1092
+ value: 10.351
1093
+ - type: precision_at_5
1094
+ value: 7.985
1095
+ - type: recall_at_1
1096
+ value: 15.662
1097
+ - type: recall_at_10
1098
+ value: 40.066
1099
+ - type: recall_at_100
1100
+ value: 65.006
1101
+ - type: recall_at_1000
1102
+ value: 85.94000000000001
1103
+ - type: recall_at_3
1104
+ value: 27.400000000000002
1105
+ - type: recall_at_5
1106
+ value: 35.002
1107
+ - task:
1108
+ type: Retrieval
1109
+ dataset:
1110
+ type: climate-fever
1111
+ name: MTEB ClimateFEVER
1112
+ config: default
1113
+ split: test
1114
+ revision: None
1115
+ metrics:
1116
+ - type: map_at_1
1117
+ value: 8.853
1118
+ - type: map_at_10
1119
+ value: 15.568000000000001
1120
+ - type: map_at_100
1121
+ value: 17.383000000000003
1122
+ - type: map_at_1000
1123
+ value: 17.584
1124
+ - type: map_at_3
1125
+ value: 12.561
1126
+ - type: map_at_5
1127
+ value: 14.056
1128
+ - type: mrr_at_1
1129
+ value: 18.958
1130
+ - type: mrr_at_10
1131
+ value: 28.288000000000004
1132
+ - type: mrr_at_100
1133
+ value: 29.432000000000002
1134
+ - type: mrr_at_1000
1135
+ value: 29.498
1136
+ - type: mrr_at_3
1137
+ value: 25.049
1138
+ - type: mrr_at_5
1139
+ value: 26.857
1140
+ - type: ndcg_at_1
1141
+ value: 18.958
1142
+ - type: ndcg_at_10
1143
+ value: 22.21
1144
+ - type: ndcg_at_100
1145
+ value: 29.596
1146
+ - type: ndcg_at_1000
1147
+ value: 33.583
1148
+ - type: ndcg_at_3
1149
+ value: 16.994999999999997
1150
+ - type: ndcg_at_5
1151
+ value: 18.95
1152
+ - type: precision_at_1
1153
+ value: 18.958
1154
+ - type: precision_at_10
1155
+ value: 7.192
1156
+ - type: precision_at_100
1157
+ value: 1.5
1158
+ - type: precision_at_1000
1159
+ value: 0.22399999999999998
1160
+ - type: precision_at_3
1161
+ value: 12.573
1162
+ - type: precision_at_5
1163
+ value: 10.202
1164
+ - type: recall_at_1
1165
+ value: 8.853
1166
+ - type: recall_at_10
1167
+ value: 28.087
1168
+ - type: recall_at_100
1169
+ value: 53.701
1170
+ - type: recall_at_1000
1171
+ value: 76.29899999999999
1172
+ - type: recall_at_3
1173
+ value: 15.913
1174
+ - type: recall_at_5
1175
+ value: 20.658
1176
+ - task:
1177
+ type: Retrieval
1178
+ dataset:
1179
+ type: dbpedia-entity
1180
+ name: MTEB DBPedia
1181
+ config: default
1182
+ split: test
1183
+ revision: None
1184
+ metrics:
1185
+ - type: map_at_1
1186
+ value: 9.077
1187
+ - type: map_at_10
1188
+ value: 20.788999999999998
1189
+ - type: map_at_100
1190
+ value: 30.429000000000002
1191
+ - type: map_at_1000
1192
+ value: 32.143
1193
+ - type: map_at_3
1194
+ value: 14.692
1195
+ - type: map_at_5
1196
+ value: 17.139
1197
+ - type: mrr_at_1
1198
+ value: 70.75
1199
+ - type: mrr_at_10
1200
+ value: 78.036
1201
+ - type: mrr_at_100
1202
+ value: 78.401
1203
+ - type: mrr_at_1000
1204
+ value: 78.404
1205
+ - type: mrr_at_3
1206
+ value: 76.75
1207
+ - type: mrr_at_5
1208
+ value: 77.47500000000001
1209
+ - type: ndcg_at_1
1210
+ value: 58.12500000000001
1211
+ - type: ndcg_at_10
1212
+ value: 44.015
1213
+ - type: ndcg_at_100
1214
+ value: 49.247
1215
+ - type: ndcg_at_1000
1216
+ value: 56.211999999999996
1217
+ - type: ndcg_at_3
1218
+ value: 49.151
1219
+ - type: ndcg_at_5
1220
+ value: 46.195
1221
+ - type: precision_at_1
1222
+ value: 70.75
1223
+ - type: precision_at_10
1224
+ value: 35.5
1225
+ - type: precision_at_100
1226
+ value: 11.355
1227
+ - type: precision_at_1000
1228
+ value: 2.1950000000000003
1229
+ - type: precision_at_3
1230
+ value: 53.083000000000006
1231
+ - type: precision_at_5
1232
+ value: 44.800000000000004
1233
+ - type: recall_at_1
1234
+ value: 9.077
1235
+ - type: recall_at_10
1236
+ value: 26.259
1237
+ - type: recall_at_100
1238
+ value: 56.547000000000004
1239
+ - type: recall_at_1000
1240
+ value: 78.551
1241
+ - type: recall_at_3
1242
+ value: 16.162000000000003
1243
+ - type: recall_at_5
1244
+ value: 19.753999999999998
1245
+ - task:
1246
+ type: Classification
1247
+ dataset:
1248
+ type: mteb/emotion
1249
+ name: MTEB EmotionClassification
1250
+ config: default
1251
+ split: test
1252
+ revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
1253
+ metrics:
1254
+ - type: accuracy
1255
+ value: 49.44500000000001
1256
+ - type: f1
1257
+ value: 44.67067691783401
1258
+ - task:
1259
+ type: Retrieval
1260
+ dataset:
1261
+ type: fever
1262
+ name: MTEB FEVER
1263
+ config: default
1264
+ split: test
1265
+ revision: None
1266
+ metrics:
1267
+ - type: map_at_1
1268
+ value: 68.182
1269
+ - type: map_at_10
1270
+ value: 78.223
1271
+ - type: map_at_100
1272
+ value: 78.498
1273
+ - type: map_at_1000
1274
+ value: 78.512
1275
+ - type: map_at_3
1276
+ value: 76.71
1277
+ - type: map_at_5
1278
+ value: 77.725
1279
+ - type: mrr_at_1
1280
+ value: 73.177
1281
+ - type: mrr_at_10
1282
+ value: 82.513
1283
+ - type: mrr_at_100
1284
+ value: 82.633
1285
+ - type: mrr_at_1000
1286
+ value: 82.635
1287
+ - type: mrr_at_3
1288
+ value: 81.376
1289
+ - type: mrr_at_5
1290
+ value: 82.182
1291
+ - type: ndcg_at_1
1292
+ value: 73.177
1293
+ - type: ndcg_at_10
1294
+ value: 82.829
1295
+ - type: ndcg_at_100
1296
+ value: 83.84
1297
+ - type: ndcg_at_1000
1298
+ value: 84.07900000000001
1299
+ - type: ndcg_at_3
1300
+ value: 80.303
1301
+ - type: ndcg_at_5
1302
+ value: 81.846
1303
+ - type: precision_at_1
1304
+ value: 73.177
1305
+ - type: precision_at_10
1306
+ value: 10.241999999999999
1307
+ - type: precision_at_100
1308
+ value: 1.099
1309
+ - type: precision_at_1000
1310
+ value: 0.11399999999999999
1311
+ - type: precision_at_3
1312
+ value: 31.247999999999998
1313
+ - type: precision_at_5
1314
+ value: 19.697
1315
+ - type: recall_at_1
1316
+ value: 68.182
1317
+ - type: recall_at_10
1318
+ value: 92.657
1319
+ - type: recall_at_100
1320
+ value: 96.709
1321
+ - type: recall_at_1000
1322
+ value: 98.184
1323
+ - type: recall_at_3
1324
+ value: 85.9
1325
+ - type: recall_at_5
1326
+ value: 89.755
1327
+ - task:
1328
+ type: Retrieval
1329
+ dataset:
1330
+ type: fiqa
1331
+ name: MTEB FiQA2018
1332
+ config: default
1333
+ split: test
1334
+ revision: None
1335
+ metrics:
1336
+ - type: map_at_1
1337
+ value: 21.108
1338
+ - type: map_at_10
1339
+ value: 33.342
1340
+ - type: map_at_100
1341
+ value: 35.281
1342
+ - type: map_at_1000
1343
+ value: 35.478
1344
+ - type: map_at_3
1345
+ value: 29.067
1346
+ - type: map_at_5
1347
+ value: 31.563000000000002
1348
+ - type: mrr_at_1
1349
+ value: 41.667
1350
+ - type: mrr_at_10
1351
+ value: 49.913000000000004
1352
+ - type: mrr_at_100
1353
+ value: 50.724000000000004
1354
+ - type: mrr_at_1000
1355
+ value: 50.766
1356
+ - type: mrr_at_3
1357
+ value: 47.504999999999995
1358
+ - type: mrr_at_5
1359
+ value: 49.033
1360
+ - type: ndcg_at_1
1361
+ value: 41.667
1362
+ - type: ndcg_at_10
1363
+ value: 41.144
1364
+ - type: ndcg_at_100
1365
+ value: 48.326
1366
+ - type: ndcg_at_1000
1367
+ value: 51.486
1368
+ - type: ndcg_at_3
1369
+ value: 37.486999999999995
1370
+ - type: ndcg_at_5
1371
+ value: 38.78
1372
+ - type: precision_at_1
1373
+ value: 41.667
1374
+ - type: precision_at_10
1375
+ value: 11.358
1376
+ - type: precision_at_100
1377
+ value: 1.873
1378
+ - type: precision_at_1000
1379
+ value: 0.244
1380
+ - type: precision_at_3
1381
+ value: 25.0
1382
+ - type: precision_at_5
1383
+ value: 18.519
1384
+ - type: recall_at_1
1385
+ value: 21.108
1386
+ - type: recall_at_10
1387
+ value: 47.249
1388
+ - type: recall_at_100
1389
+ value: 74.52
1390
+ - type: recall_at_1000
1391
+ value: 93.31
1392
+ - type: recall_at_3
1393
+ value: 33.271
1394
+ - type: recall_at_5
1395
+ value: 39.723000000000006
1396
+ - task:
1397
+ type: Retrieval
1398
+ dataset:
1399
+ type: hotpotqa
1400
+ name: MTEB HotpotQA
1401
+ config: default
1402
+ split: test
1403
+ revision: None
1404
+ metrics:
1405
+ - type: map_at_1
1406
+ value: 40.317
1407
+ - type: map_at_10
1408
+ value: 64.861
1409
+ - type: map_at_100
1410
+ value: 65.697
1411
+ - type: map_at_1000
1412
+ value: 65.755
1413
+ - type: map_at_3
1414
+ value: 61.258
1415
+ - type: map_at_5
1416
+ value: 63.590999999999994
1417
+ - type: mrr_at_1
1418
+ value: 80.635
1419
+ - type: mrr_at_10
1420
+ value: 86.528
1421
+ - type: mrr_at_100
1422
+ value: 86.66199999999999
1423
+ - type: mrr_at_1000
1424
+ value: 86.666
1425
+ - type: mrr_at_3
1426
+ value: 85.744
1427
+ - type: mrr_at_5
1428
+ value: 86.24300000000001
1429
+ - type: ndcg_at_1
1430
+ value: 80.635
1431
+ - type: ndcg_at_10
1432
+ value: 73.13199999999999
1433
+ - type: ndcg_at_100
1434
+ value: 75.927
1435
+ - type: ndcg_at_1000
1436
+ value: 76.976
1437
+ - type: ndcg_at_3
1438
+ value: 68.241
1439
+ - type: ndcg_at_5
1440
+ value: 71.071
1441
+ - type: precision_at_1
1442
+ value: 80.635
1443
+ - type: precision_at_10
1444
+ value: 15.326
1445
+ - type: precision_at_100
1446
+ value: 1.7500000000000002
1447
+ - type: precision_at_1000
1448
+ value: 0.189
1449
+ - type: precision_at_3
1450
+ value: 43.961
1451
+ - type: precision_at_5
1452
+ value: 28.599999999999998
1453
+ - type: recall_at_1
1454
+ value: 40.317
1455
+ - type: recall_at_10
1456
+ value: 76.631
1457
+ - type: recall_at_100
1458
+ value: 87.495
1459
+ - type: recall_at_1000
1460
+ value: 94.362
1461
+ - type: recall_at_3
1462
+ value: 65.94200000000001
1463
+ - type: recall_at_5
1464
+ value: 71.499
1465
+ - task:
1466
+ type: Classification
1467
+ dataset:
1468
+ type: mteb/imdb
1469
+ name: MTEB ImdbClassification
1470
+ config: default
1471
+ split: test
1472
+ revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
1473
+ metrics:
1474
+ - type: accuracy
1475
+ value: 91.686
1476
+ - type: ap
1477
+ value: 87.5577120393173
1478
+ - type: f1
1479
+ value: 91.6629447355139
1480
+ - task:
1481
+ type: Retrieval
1482
+ dataset:
1483
+ type: msmarco
1484
+ name: MTEB MSMARCO
1485
+ config: default
1486
+ split: dev
1487
+ revision: None
1488
+ metrics:
1489
+ - type: map_at_1
1490
+ value: 23.702
1491
+ - type: map_at_10
1492
+ value: 36.414
1493
+ - type: map_at_100
1494
+ value: 37.561
1495
+ - type: map_at_1000
1496
+ value: 37.605
1497
+ - type: map_at_3
1498
+ value: 32.456
1499
+ - type: map_at_5
1500
+ value: 34.827000000000005
1501
+ - type: mrr_at_1
1502
+ value: 24.355
1503
+ - type: mrr_at_10
1504
+ value: 37.01
1505
+ - type: mrr_at_100
1506
+ value: 38.085
1507
+ - type: mrr_at_1000
1508
+ value: 38.123000000000005
1509
+ - type: mrr_at_3
1510
+ value: 33.117999999999995
1511
+ - type: mrr_at_5
1512
+ value: 35.452
1513
+ - type: ndcg_at_1
1514
+ value: 24.384
1515
+ - type: ndcg_at_10
1516
+ value: 43.456
1517
+ - type: ndcg_at_100
1518
+ value: 48.892
1519
+ - type: ndcg_at_1000
1520
+ value: 49.964
1521
+ - type: ndcg_at_3
1522
+ value: 35.475
1523
+ - type: ndcg_at_5
1524
+ value: 39.711
1525
+ - type: precision_at_1
1526
+ value: 24.384
1527
+ - type: precision_at_10
1528
+ value: 6.7940000000000005
1529
+ - type: precision_at_100
1530
+ value: 0.951
1531
+ - type: precision_at_1000
1532
+ value: 0.104
1533
+ - type: precision_at_3
1534
+ value: 15.052999999999999
1535
+ - type: precision_at_5
1536
+ value: 11.189
1537
+ - type: recall_at_1
1538
+ value: 23.702
1539
+ - type: recall_at_10
1540
+ value: 65.057
1541
+ - type: recall_at_100
1542
+ value: 90.021
1543
+ - type: recall_at_1000
1544
+ value: 98.142
1545
+ - type: recall_at_3
1546
+ value: 43.551
1547
+ - type: recall_at_5
1548
+ value: 53.738
1549
+ - task:
1550
+ type: Classification
1551
+ dataset:
1552
+ type: mteb/mtop_domain
1553
+ name: MTEB MTOPDomainClassification (en)
1554
+ config: en
1555
+ split: test
1556
+ revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
1557
+ metrics:
1558
+ - type: accuracy
1559
+ value: 94.62380300957591
1560
+ - type: f1
1561
+ value: 94.49871222100734
1562
+ - task:
1563
+ type: Classification
1564
+ dataset:
1565
+ type: mteb/mtop_intent
1566
+ name: MTEB MTOPIntentClassification (en)
1567
+ config: en
1568
+ split: test
1569
+ revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
1570
+ metrics:
1571
+ - type: accuracy
1572
+ value: 77.14090287277702
1573
+ - type: f1
1574
+ value: 60.32101258220515
1575
+ - task:
1576
+ type: Classification
1577
+ dataset:
1578
+ type: mteb/amazon_massive_intent
1579
+ name: MTEB MassiveIntentClassification (en)
1580
+ config: en
1581
+ split: test
1582
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
1583
+ metrics:
1584
+ - type: accuracy
1585
+ value: 73.84330867518494
1586
+ - type: f1
1587
+ value: 71.92248688515255
1588
+ - task:
1589
+ type: Classification
1590
+ dataset:
1591
+ type: mteb/amazon_massive_scenario
1592
+ name: MTEB MassiveScenarioClassification (en)
1593
+ config: en
1594
+ split: test
1595
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
1596
+ metrics:
1597
+ - type: accuracy
1598
+ value: 78.10692669804976
1599
+ - type: f1
1600
+ value: 77.9904839122866
1601
+ - task:
1602
+ type: Clustering
1603
+ dataset:
1604
+ type: mteb/medrxiv-clustering-p2p
1605
+ name: MTEB MedrxivClusteringP2P
1606
+ config: default
1607
+ split: test
1608
+ revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
1609
+ metrics:
1610
+ - type: v_measure
1611
+ value: 31.822988923078444
1612
+ - task:
1613
+ type: Clustering
1614
+ dataset:
1615
+ type: mteb/medrxiv-clustering-s2s
1616
+ name: MTEB MedrxivClusteringS2S
1617
+ config: default
1618
+ split: test
1619
+ revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
1620
+ metrics:
1621
+ - type: v_measure
1622
+ value: 30.38394880253403
1623
+ - task:
1624
+ type: Reranking
1625
+ dataset:
1626
+ type: mteb/mind_small
1627
+ name: MTEB MindSmallReranking
1628
+ config: default
1629
+ split: test
1630
+ revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
1631
+ metrics:
1632
+ - type: map
1633
+ value: 31.82504612539082
1634
+ - type: mrr
1635
+ value: 32.84462298174977
1636
+ - task:
1637
+ type: Retrieval
1638
+ dataset:
1639
+ type: nfcorpus
1640
+ name: MTEB NFCorpus
1641
+ config: default
1642
+ split: test
1643
+ revision: None
1644
+ metrics:
1645
+ - type: map_at_1
1646
+ value: 6.029
1647
+ - type: map_at_10
1648
+ value: 14.088999999999999
1649
+ - type: map_at_100
1650
+ value: 17.601
1651
+ - type: map_at_1000
1652
+ value: 19.144
1653
+ - type: map_at_3
1654
+ value: 10.156
1655
+ - type: map_at_5
1656
+ value: 11.892
1657
+ - type: mrr_at_1
1658
+ value: 46.44
1659
+ - type: mrr_at_10
1660
+ value: 56.596999999999994
1661
+ - type: mrr_at_100
1662
+ value: 57.11000000000001
1663
+ - type: mrr_at_1000
1664
+ value: 57.14
1665
+ - type: mrr_at_3
1666
+ value: 54.334
1667
+ - type: mrr_at_5
1668
+ value: 55.774
1669
+ - type: ndcg_at_1
1670
+ value: 44.891999999999996
1671
+ - type: ndcg_at_10
1672
+ value: 37.134
1673
+ - type: ndcg_at_100
1674
+ value: 33.652
1675
+ - type: ndcg_at_1000
1676
+ value: 42.548
1677
+ - type: ndcg_at_3
1678
+ value: 41.851
1679
+ - type: ndcg_at_5
1680
+ value: 39.842
1681
+ - type: precision_at_1
1682
+ value: 46.44
1683
+ - type: precision_at_10
1684
+ value: 27.647
1685
+ - type: precision_at_100
1686
+ value: 8.309999999999999
1687
+ - type: precision_at_1000
1688
+ value: 2.146
1689
+ - type: precision_at_3
1690
+ value: 39.422000000000004
1691
+ - type: precision_at_5
1692
+ value: 34.675
1693
+ - type: recall_at_1
1694
+ value: 6.029
1695
+ - type: recall_at_10
1696
+ value: 18.907
1697
+ - type: recall_at_100
1698
+ value: 33.76
1699
+ - type: recall_at_1000
1700
+ value: 65.14999999999999
1701
+ - type: recall_at_3
1702
+ value: 11.584999999999999
1703
+ - type: recall_at_5
1704
+ value: 14.626
1705
+ - task:
1706
+ type: Retrieval
1707
+ dataset:
1708
+ type: nq
1709
+ name: MTEB NQ
1710
+ config: default
1711
+ split: test
1712
+ revision: None
1713
+ metrics:
1714
+ - type: map_at_1
1715
+ value: 39.373000000000005
1716
+ - type: map_at_10
1717
+ value: 55.836
1718
+ - type: map_at_100
1719
+ value: 56.611999999999995
1720
+ - type: map_at_1000
1721
+ value: 56.63
1722
+ - type: map_at_3
1723
+ value: 51.747
1724
+ - type: map_at_5
1725
+ value: 54.337999999999994
1726
+ - type: mrr_at_1
1727
+ value: 44.147999999999996
1728
+ - type: mrr_at_10
1729
+ value: 58.42699999999999
1730
+ - type: mrr_at_100
1731
+ value: 58.902
1732
+ - type: mrr_at_1000
1733
+ value: 58.914
1734
+ - type: mrr_at_3
1735
+ value: 55.156000000000006
1736
+ - type: mrr_at_5
1737
+ value: 57.291000000000004
1738
+ - type: ndcg_at_1
1739
+ value: 44.119
1740
+ - type: ndcg_at_10
1741
+ value: 63.444
1742
+ - type: ndcg_at_100
1743
+ value: 66.40599999999999
1744
+ - type: ndcg_at_1000
1745
+ value: 66.822
1746
+ - type: ndcg_at_3
1747
+ value: 55.962
1748
+ - type: ndcg_at_5
1749
+ value: 60.228
1750
+ - type: precision_at_1
1751
+ value: 44.119
1752
+ - type: precision_at_10
1753
+ value: 10.006
1754
+ - type: precision_at_100
1755
+ value: 1.17
1756
+ - type: precision_at_1000
1757
+ value: 0.121
1758
+ - type: precision_at_3
1759
+ value: 25.135
1760
+ - type: precision_at_5
1761
+ value: 17.59
1762
+ - type: recall_at_1
1763
+ value: 39.373000000000005
1764
+ - type: recall_at_10
1765
+ value: 83.78999999999999
1766
+ - type: recall_at_100
1767
+ value: 96.246
1768
+ - type: recall_at_1000
1769
+ value: 99.324
1770
+ - type: recall_at_3
1771
+ value: 64.71900000000001
1772
+ - type: recall_at_5
1773
+ value: 74.508
1774
+ - task:
1775
+ type: Retrieval
1776
+ dataset:
1777
+ type: quora
1778
+ name: MTEB QuoraRetrieval
1779
+ config: default
1780
+ split: test
1781
+ revision: None
1782
+ metrics:
1783
+ - type: map_at_1
1784
+ value: 69.199
1785
+ - type: map_at_10
1786
+ value: 82.892
1787
+ - type: map_at_100
1788
+ value: 83.578
1789
+ - type: map_at_1000
1790
+ value: 83.598
1791
+ - type: map_at_3
1792
+ value: 79.948
1793
+ - type: map_at_5
1794
+ value: 81.779
1795
+ - type: mrr_at_1
1796
+ value: 79.67
1797
+ - type: mrr_at_10
1798
+ value: 86.115
1799
+ - type: mrr_at_100
1800
+ value: 86.249
1801
+ - type: mrr_at_1000
1802
+ value: 86.251
1803
+ - type: mrr_at_3
1804
+ value: 85.08200000000001
1805
+ - type: mrr_at_5
1806
+ value: 85.783
1807
+ - type: ndcg_at_1
1808
+ value: 79.67
1809
+ - type: ndcg_at_10
1810
+ value: 86.839
1811
+ - type: ndcg_at_100
1812
+ value: 88.252
1813
+ - type: ndcg_at_1000
1814
+ value: 88.401
1815
+ - type: ndcg_at_3
1816
+ value: 83.86200000000001
1817
+ - type: ndcg_at_5
1818
+ value: 85.473
1819
+ - type: precision_at_1
1820
+ value: 79.67
1821
+ - type: precision_at_10
1822
+ value: 13.19
1823
+ - type: precision_at_100
1824
+ value: 1.521
1825
+ - type: precision_at_1000
1826
+ value: 0.157
1827
+ - type: precision_at_3
1828
+ value: 36.677
1829
+ - type: precision_at_5
1830
+ value: 24.118000000000002
1831
+ - type: recall_at_1
1832
+ value: 69.199
1833
+ - type: recall_at_10
1834
+ value: 94.321
1835
+ - type: recall_at_100
1836
+ value: 99.20400000000001
1837
+ - type: recall_at_1000
1838
+ value: 99.947
1839
+ - type: recall_at_3
1840
+ value: 85.787
1841
+ - type: recall_at_5
1842
+ value: 90.365
1843
+ - task:
1844
+ type: Clustering
1845
+ dataset:
1846
+ type: mteb/reddit-clustering
1847
+ name: MTEB RedditClustering
1848
+ config: default
1849
+ split: test
1850
+ revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
1851
+ metrics:
1852
+ - type: v_measure
1853
+ value: 55.82810046856353
1854
+ - task:
1855
+ type: Clustering
1856
+ dataset:
1857
+ type: mteb/reddit-clustering-p2p
1858
+ name: MTEB RedditClusteringP2P
1859
+ config: default
1860
+ split: test
1861
+ revision: 282350215ef01743dc01b456c7f5241fa8937f16
1862
+ metrics:
1863
+ - type: v_measure
1864
+ value: 63.38132611783628
1865
+ - task:
1866
+ type: Retrieval
1867
+ dataset:
1868
+ type: scidocs
1869
+ name: MTEB SCIDOCS
1870
+ config: default
1871
+ split: test
1872
+ revision: None
1873
+ metrics:
1874
+ - type: map_at_1
1875
+ value: 5.127000000000001
1876
+ - type: map_at_10
1877
+ value: 12.235
1878
+ - type: map_at_100
1879
+ value: 14.417
1880
+ - type: map_at_1000
1881
+ value: 14.75
1882
+ - type: map_at_3
1883
+ value: 8.906
1884
+ - type: map_at_5
1885
+ value: 10.591000000000001
1886
+ - type: mrr_at_1
1887
+ value: 25.2
1888
+ - type: mrr_at_10
1889
+ value: 35.879
1890
+ - type: mrr_at_100
1891
+ value: 36.935
1892
+ - type: mrr_at_1000
1893
+ value: 36.997
1894
+ - type: mrr_at_3
1895
+ value: 32.783
1896
+ - type: mrr_at_5
1897
+ value: 34.367999999999995
1898
+ - type: ndcg_at_1
1899
+ value: 25.2
1900
+ - type: ndcg_at_10
1901
+ value: 20.509
1902
+ - type: ndcg_at_100
1903
+ value: 28.67
1904
+ - type: ndcg_at_1000
1905
+ value: 34.42
1906
+ - type: ndcg_at_3
1907
+ value: 19.948
1908
+ - type: ndcg_at_5
1909
+ value: 17.166
1910
+ - type: precision_at_1
1911
+ value: 25.2
1912
+ - type: precision_at_10
1913
+ value: 10.440000000000001
1914
+ - type: precision_at_100
1915
+ value: 2.214
1916
+ - type: precision_at_1000
1917
+ value: 0.359
1918
+ - type: precision_at_3
1919
+ value: 18.533
1920
+ - type: precision_at_5
1921
+ value: 14.860000000000001
1922
+ - type: recall_at_1
1923
+ value: 5.127000000000001
1924
+ - type: recall_at_10
1925
+ value: 21.147
1926
+ - type: recall_at_100
1927
+ value: 44.946999999999996
1928
+ - type: recall_at_1000
1929
+ value: 72.89
1930
+ - type: recall_at_3
1931
+ value: 11.277
1932
+ - type: recall_at_5
1933
+ value: 15.042
1934
+ - task:
1935
+ type: STS
1936
+ dataset:
1937
+ type: mteb/sickr-sts
1938
+ name: MTEB SICK-R
1939
+ config: default
1940
+ split: test
1941
+ revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
1942
+ metrics:
1943
+ - type: cos_sim_pearson
1944
+ value: 83.0373011786213
1945
+ - type: cos_sim_spearman
1946
+ value: 79.27889560856613
1947
+ - type: euclidean_pearson
1948
+ value: 80.31186315495655
1949
+ - type: euclidean_spearman
1950
+ value: 79.41630415280811
1951
+ - type: manhattan_pearson
1952
+ value: 80.31755140442013
1953
+ - type: manhattan_spearman
1954
+ value: 79.43069870027611
1955
+ - task:
1956
+ type: STS
1957
+ dataset:
1958
+ type: mteb/sts12-sts
1959
+ name: MTEB STS12
1960
+ config: default
1961
+ split: test
1962
+ revision: a0d554a64d88156834ff5ae9920b964011b16384
1963
+ metrics:
1964
+ - type: cos_sim_pearson
1965
+ value: 84.8659751342045
1966
+ - type: cos_sim_spearman
1967
+ value: 76.95377612997667
1968
+ - type: euclidean_pearson
1969
+ value: 81.24552945497848
1970
+ - type: euclidean_spearman
1971
+ value: 77.18236963555253
1972
+ - type: manhattan_pearson
1973
+ value: 81.26477607759037
1974
+ - type: manhattan_spearman
1975
+ value: 77.13821753062756
1976
+ - task:
1977
+ type: STS
1978
+ dataset:
1979
+ type: mteb/sts13-sts
1980
+ name: MTEB STS13
1981
+ config: default
1982
+ split: test
1983
+ revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
1984
+ metrics:
1985
+ - type: cos_sim_pearson
1986
+ value: 83.34597139044875
1987
+ - type: cos_sim_spearman
1988
+ value: 84.124169425592
1989
+ - type: euclidean_pearson
1990
+ value: 83.68590721511401
1991
+ - type: euclidean_spearman
1992
+ value: 84.18846190846398
1993
+ - type: manhattan_pearson
1994
+ value: 83.57630235061498
1995
+ - type: manhattan_spearman
1996
+ value: 84.10244043726902
1997
+ - task:
1998
+ type: STS
1999
+ dataset:
2000
+ type: mteb/sts14-sts
2001
+ name: MTEB STS14
2002
+ config: default
2003
+ split: test
2004
+ revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
2005
+ metrics:
2006
+ - type: cos_sim_pearson
2007
+ value: 82.67641885599572
2008
+ - type: cos_sim_spearman
2009
+ value: 80.46450725650428
2010
+ - type: euclidean_pearson
2011
+ value: 81.61645042715865
2012
+ - type: euclidean_spearman
2013
+ value: 80.61418394236874
2014
+ - type: manhattan_pearson
2015
+ value: 81.55712034928871
2016
+ - type: manhattan_spearman
2017
+ value: 80.57905670523951
2018
+ - task:
2019
+ type: STS
2020
+ dataset:
2021
+ type: mteb/sts15-sts
2022
+ name: MTEB STS15
2023
+ config: default
2024
+ split: test
2025
+ revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
2026
+ metrics:
2027
+ - type: cos_sim_pearson
2028
+ value: 88.86650310886782
2029
+ - type: cos_sim_spearman
2030
+ value: 89.76081629222328
2031
+ - type: euclidean_pearson
2032
+ value: 89.1530747029954
2033
+ - type: euclidean_spearman
2034
+ value: 89.80990657280248
2035
+ - type: manhattan_pearson
2036
+ value: 89.10640563278132
2037
+ - type: manhattan_spearman
2038
+ value: 89.76282108434047
2039
+ - task:
2040
+ type: STS
2041
+ dataset:
2042
+ type: mteb/sts16-sts
2043
+ name: MTEB STS16
2044
+ config: default
2045
+ split: test
2046
+ revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
2047
+ metrics:
2048
+ - type: cos_sim_pearson
2049
+ value: 83.93864027911118
2050
+ - type: cos_sim_spearman
2051
+ value: 85.47096193999023
2052
+ - type: euclidean_pearson
2053
+ value: 85.03141840870533
2054
+ - type: euclidean_spearman
2055
+ value: 85.43124029598181
2056
+ - type: manhattan_pearson
2057
+ value: 84.99002664393512
2058
+ - type: manhattan_spearman
2059
+ value: 85.39169195120834
2060
+ - task:
2061
+ type: STS
2062
+ dataset:
2063
+ type: mteb/sts17-crosslingual-sts
2064
+ name: MTEB STS17 (en-en)
2065
+ config: en-en
2066
+ split: test
2067
+ revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
2068
+ metrics:
2069
+ - type: cos_sim_pearson
2070
+ value: 88.7045343749832
2071
+ - type: cos_sim_spearman
2072
+ value: 89.03262221146677
2073
+ - type: euclidean_pearson
2074
+ value: 89.56078218264365
2075
+ - type: euclidean_spearman
2076
+ value: 89.17827006466868
2077
+ - type: manhattan_pearson
2078
+ value: 89.52717595468582
2079
+ - type: manhattan_spearman
2080
+ value: 89.15878115952923
2081
+ - task:
2082
+ type: STS
2083
+ dataset:
2084
+ type: mteb/sts22-crosslingual-sts
2085
+ name: MTEB STS22 (en)
2086
+ config: en
2087
+ split: test
2088
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
2089
+ metrics:
2090
+ - type: cos_sim_pearson
2091
+ value: 64.20191302875551
2092
+ - type: cos_sim_spearman
2093
+ value: 64.11446552557646
2094
+ - type: euclidean_pearson
2095
+ value: 64.6918197393619
2096
+ - type: euclidean_spearman
2097
+ value: 63.440182631197764
2098
+ - type: manhattan_pearson
2099
+ value: 64.55692904121835
2100
+ - type: manhattan_spearman
2101
+ value: 63.424877742756266
2102
+ - task:
2103
+ type: STS
2104
+ dataset:
2105
+ type: mteb/stsbenchmark-sts
2106
+ name: MTEB STSBenchmark
2107
+ config: default
2108
+ split: test
2109
+ revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
2110
+ metrics:
2111
+ - type: cos_sim_pearson
2112
+ value: 86.37793104662344
2113
+ - type: cos_sim_spearman
2114
+ value: 87.7357802629067
2115
+ - type: euclidean_pearson
2116
+ value: 87.4286301545109
2117
+ - type: euclidean_spearman
2118
+ value: 87.78452920777421
2119
+ - type: manhattan_pearson
2120
+ value: 87.42445169331255
2121
+ - type: manhattan_spearman
2122
+ value: 87.78537677249598
2123
+ - task:
2124
+ type: Reranking
2125
+ dataset:
2126
+ type: mteb/scidocs-reranking
2127
+ name: MTEB SciDocsRR
2128
+ config: default
2129
+ split: test
2130
+ revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
2131
+ metrics:
2132
+ - type: map
2133
+ value: 84.31465405081792
2134
+ - type: mrr
2135
+ value: 95.7173781193389
2136
+ - task:
2137
+ type: Retrieval
2138
+ dataset:
2139
+ type: scifact
2140
+ name: MTEB SciFact
2141
+ config: default
2142
+ split: test
2143
+ revision: None
2144
+ metrics:
2145
+ - type: map_at_1
2146
+ value: 57.760999999999996
2147
+ - type: map_at_10
2148
+ value: 67.904
2149
+ - type: map_at_100
2150
+ value: 68.539
2151
+ - type: map_at_1000
2152
+ value: 68.562
2153
+ - type: map_at_3
2154
+ value: 65.415
2155
+ - type: map_at_5
2156
+ value: 66.788
2157
+ - type: mrr_at_1
2158
+ value: 60.333000000000006
2159
+ - type: mrr_at_10
2160
+ value: 68.797
2161
+ - type: mrr_at_100
2162
+ value: 69.236
2163
+ - type: mrr_at_1000
2164
+ value: 69.257
2165
+ - type: mrr_at_3
2166
+ value: 66.667
2167
+ - type: mrr_at_5
2168
+ value: 67.967
2169
+ - type: ndcg_at_1
2170
+ value: 60.333000000000006
2171
+ - type: ndcg_at_10
2172
+ value: 72.24199999999999
2173
+ - type: ndcg_at_100
2174
+ value: 74.86
2175
+ - type: ndcg_at_1000
2176
+ value: 75.354
2177
+ - type: ndcg_at_3
2178
+ value: 67.93400000000001
2179
+ - type: ndcg_at_5
2180
+ value: 70.02199999999999
2181
+ - type: precision_at_1
2182
+ value: 60.333000000000006
2183
+ - type: precision_at_10
2184
+ value: 9.533
2185
+ - type: precision_at_100
2186
+ value: 1.09
2187
+ - type: precision_at_1000
2188
+ value: 0.11299999999999999
2189
+ - type: precision_at_3
2190
+ value: 26.778000000000002
2191
+ - type: precision_at_5
2192
+ value: 17.467
2193
+ - type: recall_at_1
2194
+ value: 57.760999999999996
2195
+ - type: recall_at_10
2196
+ value: 84.383
2197
+ - type: recall_at_100
2198
+ value: 96.267
2199
+ - type: recall_at_1000
2200
+ value: 100.0
2201
+ - type: recall_at_3
2202
+ value: 72.628
2203
+ - type: recall_at_5
2204
+ value: 78.094
2205
+ - task:
2206
+ type: PairClassification
2207
+ dataset:
2208
+ type: mteb/sprintduplicatequestions-pairclassification
2209
+ name: MTEB SprintDuplicateQuestions
2210
+ config: default
2211
+ split: test
2212
+ revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
2213
+ metrics:
2214
+ - type: cos_sim_accuracy
2215
+ value: 99.8029702970297
2216
+ - type: cos_sim_ap
2217
+ value: 94.9210324173411
2218
+ - type: cos_sim_f1
2219
+ value: 89.8521162672106
2220
+ - type: cos_sim_precision
2221
+ value: 91.67533818938605
2222
+ - type: cos_sim_recall
2223
+ value: 88.1
2224
+ - type: dot_accuracy
2225
+ value: 99.69504950495049
2226
+ - type: dot_ap
2227
+ value: 90.4919719146181
2228
+ - type: dot_f1
2229
+ value: 84.72289156626506
2230
+ - type: dot_precision
2231
+ value: 81.76744186046511
2232
+ - type: dot_recall
2233
+ value: 87.9
2234
+ - type: euclidean_accuracy
2235
+ value: 99.79702970297029
2236
+ - type: euclidean_ap
2237
+ value: 94.87827463795753
2238
+ - type: euclidean_f1
2239
+ value: 89.55680081507896
2240
+ - type: euclidean_precision
2241
+ value: 91.27725856697819
2242
+ - type: euclidean_recall
2243
+ value: 87.9
2244
+ - type: manhattan_accuracy
2245
+ value: 99.7990099009901
2246
+ - type: manhattan_ap
2247
+ value: 94.87587025149682
2248
+ - type: manhattan_f1
2249
+ value: 89.76298537569339
2250
+ - type: manhattan_precision
2251
+ value: 90.53916581892166
2252
+ - type: manhattan_recall
2253
+ value: 89.0
2254
+ - type: max_accuracy
2255
+ value: 99.8029702970297
2256
+ - type: max_ap
2257
+ value: 94.9210324173411
2258
+ - type: max_f1
2259
+ value: 89.8521162672106
2260
+ - task:
2261
+ type: Clustering
2262
+ dataset:
2263
+ type: mteb/stackexchange-clustering
2264
+ name: MTEB StackExchangeClustering
2265
+ config: default
2266
+ split: test
2267
+ revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
2268
+ metrics:
2269
+ - type: v_measure
2270
+ value: 65.92385753948724
2271
+ - task:
2272
+ type: Clustering
2273
+ dataset:
2274
+ type: mteb/stackexchange-clustering-p2p
2275
+ name: MTEB StackExchangeClusteringP2P
2276
+ config: default
2277
+ split: test
2278
+ revision: 815ca46b2622cec33ccafc3735d572c266efdb44
2279
+ metrics:
2280
+ - type: v_measure
2281
+ value: 33.671756975431144
2282
+ - task:
2283
+ type: Reranking
2284
+ dataset:
2285
+ type: mteb/stackoverflowdupquestions-reranking
2286
+ name: MTEB StackOverflowDupQuestions
2287
+ config: default
2288
+ split: test
2289
+ revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
2290
+ metrics:
2291
+ - type: map
2292
+ value: 50.677928036739004
2293
+ - type: mrr
2294
+ value: 51.56413133435193
2295
+ - task:
2296
+ type: Summarization
2297
+ dataset:
2298
+ type: mteb/summeval
2299
+ name: MTEB SummEval
2300
+ config: default
2301
+ split: test
2302
+ revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
2303
+ metrics:
2304
+ - type: cos_sim_pearson
2305
+ value: 30.523589340819683
2306
+ - type: cos_sim_spearman
2307
+ value: 30.187407518823235
2308
+ - type: dot_pearson
2309
+ value: 29.039713969699015
2310
+ - type: dot_spearman
2311
+ value: 29.114740651155508
2312
+ - task:
2313
+ type: Retrieval
2314
+ dataset:
2315
+ type: trec-covid
2316
+ name: MTEB TRECCOVID
2317
+ config: default
2318
+ split: test
2319
+ revision: None
2320
+ metrics:
2321
+ - type: map_at_1
2322
+ value: 0.211
2323
+ - type: map_at_10
2324
+ value: 1.6199999999999999
2325
+ - type: map_at_100
2326
+ value: 8.658000000000001
2327
+ - type: map_at_1000
2328
+ value: 21.538
2329
+ - type: map_at_3
2330
+ value: 0.575
2331
+ - type: map_at_5
2332
+ value: 0.919
2333
+ - type: mrr_at_1
2334
+ value: 78.0
2335
+ - type: mrr_at_10
2336
+ value: 86.18599999999999
2337
+ - type: mrr_at_100
2338
+ value: 86.18599999999999
2339
+ - type: mrr_at_1000
2340
+ value: 86.18599999999999
2341
+ - type: mrr_at_3
2342
+ value: 85.0
2343
+ - type: mrr_at_5
2344
+ value: 85.9
2345
+ - type: ndcg_at_1
2346
+ value: 74.0
2347
+ - type: ndcg_at_10
2348
+ value: 66.542
2349
+ - type: ndcg_at_100
2350
+ value: 50.163999999999994
2351
+ - type: ndcg_at_1000
2352
+ value: 45.696999999999996
2353
+ - type: ndcg_at_3
2354
+ value: 71.531
2355
+ - type: ndcg_at_5
2356
+ value: 70.45
2357
+ - type: precision_at_1
2358
+ value: 78.0
2359
+ - type: precision_at_10
2360
+ value: 69.39999999999999
2361
+ - type: precision_at_100
2362
+ value: 51.06
2363
+ - type: precision_at_1000
2364
+ value: 20.022000000000002
2365
+ - type: precision_at_3
2366
+ value: 76.0
2367
+ - type: precision_at_5
2368
+ value: 74.8
2369
+ - type: recall_at_1
2370
+ value: 0.211
2371
+ - type: recall_at_10
2372
+ value: 1.813
2373
+ - type: recall_at_100
2374
+ value: 12.098
2375
+ - type: recall_at_1000
2376
+ value: 42.618
2377
+ - type: recall_at_3
2378
+ value: 0.603
2379
+ - type: recall_at_5
2380
+ value: 0.987
2381
+ - task:
2382
+ type: Retrieval
2383
+ dataset:
2384
+ type: webis-touche2020
2385
+ name: MTEB Touche2020
2386
+ config: default
2387
+ split: test
2388
+ revision: None
2389
+ metrics:
2390
+ - type: map_at_1
2391
+ value: 2.2079999999999997
2392
+ - type: map_at_10
2393
+ value: 7.777000000000001
2394
+ - type: map_at_100
2395
+ value: 12.825000000000001
2396
+ - type: map_at_1000
2397
+ value: 14.196
2398
+ - type: map_at_3
2399
+ value: 4.285
2400
+ - type: map_at_5
2401
+ value: 6.177
2402
+ - type: mrr_at_1
2403
+ value: 30.612000000000002
2404
+ - type: mrr_at_10
2405
+ value: 42.635
2406
+ - type: mrr_at_100
2407
+ value: 43.955
2408
+ - type: mrr_at_1000
2409
+ value: 43.955
2410
+ - type: mrr_at_3
2411
+ value: 38.435
2412
+ - type: mrr_at_5
2413
+ value: 41.088
2414
+ - type: ndcg_at_1
2415
+ value: 28.571
2416
+ - type: ndcg_at_10
2417
+ value: 20.666999999999998
2418
+ - type: ndcg_at_100
2419
+ value: 31.840000000000003
2420
+ - type: ndcg_at_1000
2421
+ value: 43.191
2422
+ - type: ndcg_at_3
2423
+ value: 23.45
2424
+ - type: ndcg_at_5
2425
+ value: 22.994
2426
+ - type: precision_at_1
2427
+ value: 30.612000000000002
2428
+ - type: precision_at_10
2429
+ value: 17.959
2430
+ - type: precision_at_100
2431
+ value: 6.755
2432
+ - type: precision_at_1000
2433
+ value: 1.4200000000000002
2434
+ - type: precision_at_3
2435
+ value: 23.810000000000002
2436
+ - type: precision_at_5
2437
+ value: 23.673
2438
+ - type: recall_at_1
2439
+ value: 2.2079999999999997
2440
+ - type: recall_at_10
2441
+ value: 13.144
2442
+ - type: recall_at_100
2443
+ value: 42.491
2444
+ - type: recall_at_1000
2445
+ value: 77.04299999999999
2446
+ - type: recall_at_3
2447
+ value: 5.3469999999999995
2448
+ - type: recall_at_5
2449
+ value: 9.139
2450
+ - task:
2451
+ type: Classification
2452
+ dataset:
2453
+ type: mteb/toxic_conversations_50k
2454
+ name: MTEB ToxicConversationsClassification
2455
+ config: default
2456
+ split: test
2457
+ revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
2458
+ metrics:
2459
+ - type: accuracy
2460
+ value: 70.9044
2461
+ - type: ap
2462
+ value: 14.625783489340755
2463
+ - type: f1
2464
+ value: 54.814936562590546
2465
+ - task:
2466
+ type: Classification
2467
+ dataset:
2468
+ type: mteb/tweet_sentiment_extraction
2469
+ name: MTEB TweetSentimentExtractionClassification
2470
+ config: default
2471
+ split: test
2472
+ revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
2473
+ metrics:
2474
+ - type: accuracy
2475
+ value: 60.94227504244483
2476
+ - type: f1
2477
+ value: 61.22516038508854
2478
+ - task:
2479
+ type: Clustering
2480
+ dataset:
2481
+ type: mteb/twentynewsgroups-clustering
2482
+ name: MTEB TwentyNewsgroupsClustering
2483
+ config: default
2484
+ split: test
2485
+ revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
2486
+ metrics:
2487
+ - type: v_measure
2488
+ value: 49.602409155145864
2489
+ - task:
2490
+ type: PairClassification
2491
+ dataset:
2492
+ type: mteb/twittersemeval2015-pairclassification
2493
+ name: MTEB TwitterSemEval2015
2494
+ config: default
2495
+ split: test
2496
+ revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
2497
+ metrics:
2498
+ - type: cos_sim_accuracy
2499
+ value: 86.94641473445789
2500
+ - type: cos_sim_ap
2501
+ value: 76.91572747061197
2502
+ - type: cos_sim_f1
2503
+ value: 70.14348097317529
2504
+ - type: cos_sim_precision
2505
+ value: 66.53254437869822
2506
+ - type: cos_sim_recall
2507
+ value: 74.1688654353562
2508
+ - type: dot_accuracy
2509
+ value: 84.80061989628658
2510
+ - type: dot_ap
2511
+ value: 70.7952548895177
2512
+ - type: dot_f1
2513
+ value: 65.44780728844965
2514
+ - type: dot_precision
2515
+ value: 61.53310104529617
2516
+ - type: dot_recall
2517
+ value: 69.89445910290237
2518
+ - type: euclidean_accuracy
2519
+ value: 86.94641473445789
2520
+ - type: euclidean_ap
2521
+ value: 76.80774009393652
2522
+ - type: euclidean_f1
2523
+ value: 70.30522503879979
2524
+ - type: euclidean_precision
2525
+ value: 68.94977168949772
2526
+ - type: euclidean_recall
2527
+ value: 71.71503957783642
2528
+ - type: manhattan_accuracy
2529
+ value: 86.8629671574179
2530
+ - type: manhattan_ap
2531
+ value: 76.76518632600317
2532
+ - type: manhattan_f1
2533
+ value: 70.16056518946692
2534
+ - type: manhattan_precision
2535
+ value: 68.360450563204
2536
+ - type: manhattan_recall
2537
+ value: 72.0580474934037
2538
+ - type: max_accuracy
2539
+ value: 86.94641473445789
2540
+ - type: max_ap
2541
+ value: 76.91572747061197
2542
+ - type: max_f1
2543
+ value: 70.30522503879979
2544
+ - task:
2545
+ type: PairClassification
2546
+ dataset:
2547
+ type: mteb/twitterurlcorpus-pairclassification
2548
+ name: MTEB TwitterURLCorpus
2549
+ config: default
2550
+ split: test
2551
+ revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
2552
+ metrics:
2553
+ - type: cos_sim_accuracy
2554
+ value: 89.10428066907285
2555
+ - type: cos_sim_ap
2556
+ value: 86.25114759921435
2557
+ - type: cos_sim_f1
2558
+ value: 78.37857884586856
2559
+ - type: cos_sim_precision
2560
+ value: 75.60818546078993
2561
+ - type: cos_sim_recall
2562
+ value: 81.35971666153372
2563
+ - type: dot_accuracy
2564
+ value: 87.41995575736406
2565
+ - type: dot_ap
2566
+ value: 81.51838010086782
2567
+ - type: dot_f1
2568
+ value: 74.77398015435503
2569
+ - type: dot_precision
2570
+ value: 71.53002390662354
2571
+ - type: dot_recall
2572
+ value: 78.32614721281182
2573
+ - type: euclidean_accuracy
2574
+ value: 89.12368533395428
2575
+ - type: euclidean_ap
2576
+ value: 86.33456799874504
2577
+ - type: euclidean_f1
2578
+ value: 78.45496750232127
2579
+ - type: euclidean_precision
2580
+ value: 75.78388462366364
2581
+ - type: euclidean_recall
2582
+ value: 81.32121958731136
2583
+ - type: manhattan_accuracy
2584
+ value: 89.10622113556099
2585
+ - type: manhattan_ap
2586
+ value: 86.31215061745333
2587
+ - type: manhattan_f1
2588
+ value: 78.40684906011539
2589
+ - type: manhattan_precision
2590
+ value: 75.89536643366722
2591
+ - type: manhattan_recall
2592
+ value: 81.09023714197721
2593
+ - type: max_accuracy
2594
+ value: 89.12368533395428
2595
+ - type: max_ap
2596
+ value: 86.33456799874504
2597
+ - type: max_f1
2598
+ value: 78.45496750232127
2599
+ ---
2600
+
2601
+ # E5-large-v2
2602
+
2603
+ [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://arxiv.org/pdf/2212.03533.pdf).
2604
+ Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, Furu Wei, arXiv 2022
2605
+
2606
+ This model has 24 layers and the embedding size is 1024.
2607
+
2608
+ ## Usage
2609
+
2610
+ Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset.
2611
+
2612
+ ```python
2613
+ import torch.nn.functional as F
2614
+
2615
+ from torch import Tensor
2616
+ from transformers import AutoTokenizer, AutoModel
2617
+
2618
+
2619
+ def average_pool(last_hidden_states: Tensor,
2620
+ attention_mask: Tensor) -> Tensor:
2621
+ last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
2622
+ return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]
2623
+
2624
+
2625
+ # Each input text should start with "query: " or "passage: ".
2626
+ # For tasks other than retrieval, you can simply use the "query: " prefix.
2627
+ input_texts = ['query: how much protein should a female eat',
2628
+ 'query: summit define',
2629
+ "passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
2630
+ "passage: Definition of summit for English Language Learners. : 1 the highest point of a mountain : the top of a mountain. : 2 the highest level. : 3 a meeting or series of meetings between the leaders of two or more governments."]
2631
+
2632
+ tokenizer = AutoTokenizer.from_pretrained('intfloat/e5-large-v2')
2633
+ model = AutoModel.from_pretrained('intfloat/e5-large-v2')
2634
+
2635
+ # Tokenize the input texts
2636
+ batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')
2637
+
2638
+ outputs = model(**batch_dict)
2639
+ embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
2640
+
2641
+ # (Optionally) normalize embeddings
2642
+ embeddings = F.normalize(embeddings, p=2, dim=1)
2643
+ scores = (embeddings[:2] @ embeddings[2:].T) * 100
2644
+ print(scores.tolist())
2645
+ ```
2646
+
2647
+ ## Training Details
2648
+
2649
+ Please refer to our paper at [https://arxiv.org/pdf/2212.03533.pdf](https://arxiv.org/pdf/2212.03533.pdf).
2650
+
2651
+ ## Benchmark Evaluation
2652
+
2653
+ Check out [unilm/e5](https://github.com/microsoft/unilm/tree/master/e5) to reproduce evaluation results
2654
+ on the [BEIR](https://arxiv.org/abs/2104.08663) and [MTEB benchmark](https://arxiv.org/abs/2210.07316).
2655
+
2656
+ ## Citation
2657
+
2658
+ If you find our paper or models helpful, please consider cite as follows:
2659
+
2660
+ ```
2661
+ @article{wang2022text,
2662
+ title={Text Embeddings by Weakly-Supervised Contrastive Pre-training},
2663
+ author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Jiao, Binxing and Yang, Linjun and Jiang, Daxin and Majumder, Rangan and Wei, Furu},
2664
+ journal={arXiv preprint arXiv:2212.03533},
2665
+ year={2022}
2666
+ }
2667
+ ```
2668
+
2669
+ ## Limitations
2670
+
2671
+ This model only works for English texts. Long texts will be truncated to at most 512 tokens.
2672
+
config.json ADDED
@@ -0,0 +1,25 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "tmp/",
3
+ "architectures": [
4
+ "BertModel"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 1024,
11
+ "initializer_range": 0.02,
12
+ "intermediate_size": 4096,
13
+ "layer_norm_eps": 1e-12,
14
+ "max_position_embeddings": 512,
15
+ "model_type": "bert",
16
+ "num_attention_heads": 16,
17
+ "num_hidden_layers": 24,
18
+ "pad_token_id": 0,
19
+ "position_embedding_type": "absolute",
20
+ "torch_dtype": "float32",
21
+ "transformers_version": "4.29.0.dev0",
22
+ "type_vocab_size": 2,
23
+ "use_cache": true,
24
+ "vocab_size": 30522
25
+ }
pytorch_model.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7d3fda35853349a026a61027d93bbfd65d7658287a2043f95af3e4397a4e9c5e
3
+ size 1340698349
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,13 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "clean_up_tokenization_spaces": true,
3
+ "cls_token": "[CLS]",
4
+ "do_lower_case": true,
5
+ "mask_token": "[MASK]",
6
+ "model_max_length": 512,
7
+ "pad_token": "[PAD]",
8
+ "sep_token": "[SEP]",
9
+ "strip_accents": null,
10
+ "tokenize_chinese_chars": true,
11
+ "tokenizer_class": "BertTokenizer",
12
+ "unk_token": "[UNK]"
13
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff