Update README.md
Browse files
README.md
CHANGED
|
@@ -5,9 +5,9 @@ license: apache-2.0
|
|
| 5 |
This model is a merged model, using [bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5), [GIST-Embedding-v0](https://huggingface.co/avsolatorio/GIST-Embedding-v0) and [gte-base](https://huggingface.co/thenlper/gte-base).
|
| 6 |
This model focuses on retrieval tasks while also performing well on various tasks (See experiment details below).
|
| 7 |
|
| 8 |
-
##Usage
|
| 9 |
|
| 10 |
-
##For retrieval tasks
|
| 11 |
```python
|
| 12 |
from transformers import AutoTokenizer, AutoModel
|
| 13 |
import torch
|
|
@@ -37,5 +37,13 @@ with torch.no_grad():
|
|
| 37 |
sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
|
| 38 |
print("Sentence embeddings:", sentence_embeddings)
|
| 39 |
```
|
| 40 |
-
|
| 41 |
-
<img src="slerp.png" alt="109M models retrieval benchmarks" width="650" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
This model is a merged model, using [bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5), [GIST-Embedding-v0](https://huggingface.co/avsolatorio/GIST-Embedding-v0) and [gte-base](https://huggingface.co/thenlper/gte-base).
|
| 6 |
This model focuses on retrieval tasks while also performing well on various tasks (See experiment details below).
|
| 7 |
|
| 8 |
+
## Usage
|
| 9 |
|
| 10 |
+
## For retrieval tasks
|
| 11 |
```python
|
| 12 |
from transformers import AutoTokenizer, AutoModel
|
| 13 |
import torch
|
|
|
|
| 37 |
sentence_embeddings = torch.nn.functional.normalize(sentence_embeddings, p=2, dim=1)
|
| 38 |
print("Sentence embeddings:", sentence_embeddings)
|
| 39 |
```
|
| 40 |
+
## Evaluation
|
| 41 |
+
<img src="slerp.png" alt="109M models retrieval benchmarks" width="650" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
| 42 |
+
|
| 43 |
+
| Models | Average | ArguAna | ClimateFEVER | CQADupstackRetrieval | DBPedia | FEVER | FiQA2018 | HotpotQA | MSMARCO | NFCorpus | NQ | QuoraRetrieval | SCIDOCS | SciFact | Touche2020 | TRECCOVID |
|
| 44 |
+
| ------ | ------- | ------- | ------------ | -------------------- | ------- | ----- | -------- | -------- | ------- | -------- | -- | -------------- | ------- | ------- | ---------- | --------- |
|
| 45 |
+
| Slerp (average prefixing) | 53.37 | 63.45 | 31.66 | 42.51 | 42.15 | 86.53 | 43.28 | 73.72 | 39.16 | 37.66 | 54.39 | 88.96 | 22.7 | 76.66 | 21.46 | 76.23 |
|
| 46 |
+
| Slerp (prefixing) | 53.18 | 63.58 | 30.67 | 43.23 | 41.52 | 86.54 | 41.28 | 71.43 | 41.16 | 38.01 | 55.28 | 88.72 | 22.75 | 75.05 | 22.15 | 76.27 |
|
| 47 |
+
| BGE | 53.25 | 63.61 | 31.17 | 42.35 | 40.77 | 86.29 | 40.65 | 72.6 | 41.35 | 37.39 | 54.15 | 88.9 | 21.73 | 74.04 | 25.7 | 78.07 |
|
| 48 |
+
| GIST | 52.31 | 62.62 | 31.49 | 43.2 | 41.7 | 86.65 | 40.64 | 68.92 | 40.64 | 37.64 | 53.43 | 88.81 | 23.47 | 75.29 | 20.58 | 69.6 |
|
| 49 |
+
| GTE | 51.14 | 57.12 | 28.1 | 42.91 | 41.19 | 81.52 | 40.76 | 65.75 | 40.21 | 37.9 | 52.84 | 88.15 | 23.13 | 76.18 | 22.55 | 68.78 |
|