Sentence Similarity
sentence-transformers
PyTorch
Transformers
English
t5
text-embedding
embeddings
information-retrieval
beir
text-classification
language-model
text-clustering
text-semantic-similarity
text-evaluation
prompt-retrieval
text-reranking
feature-extraction
English
Sentence Similarity
natural_questions
ms_marco
fever
hotpot_qa
mteb
Eval Results
multi-train
commited on
Commit
•
31a5899
1
Parent(s):
e9d20e0
Update README.md
Browse files
README.md
CHANGED
@@ -37,11 +37,27 @@ You can further use the model to compute similarities between two groups of sent
|
|
37 |
```python
|
38 |
from sklearn.metrics.pairwise import cosine_similarity
|
39 |
sentences_a = [['Represent the Science sentence; Input: ','Parton energy loss in QCD matter',0],
|
40 |
-
['Represent the Financial statement; Input: ','The Federal Reserve on Wednesday raised its benchmark interest rate.',0]
|
41 |
sentences_b = [['Represent the Science sentence; Input: ','The Chiral Phase Transition in Dissipative Dynamics', 0],
|
42 |
-
['Represent the Financial statement; Input: ','The funds rose less than 0.5 per cent on Friday',0]
|
43 |
embeddings_a = model.encode(sentences_a)
|
44 |
embeddings_b = model.encode(sentences_b)
|
45 |
similarities = cosine_similarity(embeddings_a,embeddings_b)
|
46 |
print(similarities)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
```
|
|
|
37 |
```python
|
38 |
from sklearn.metrics.pairwise import cosine_similarity
|
39 |
sentences_a = [['Represent the Science sentence; Input: ','Parton energy loss in QCD matter',0],
|
40 |
+
['Represent the Financial statement; Input: ','The Federal Reserve on Wednesday raised its benchmark interest rate.',0]]
|
41 |
sentences_b = [['Represent the Science sentence; Input: ','The Chiral Phase Transition in Dissipative Dynamics', 0],
|
42 |
+
['Represent the Financial statement; Input: ','The funds rose less than 0.5 per cent on Friday',0]]
|
43 |
embeddings_a = model.encode(sentences_a)
|
44 |
embeddings_b = model.encode(sentences_b)
|
45 |
similarities = cosine_similarity(embeddings_a,embeddings_b)
|
46 |
print(similarities)
|
47 |
+
```
|
48 |
+
|
49 |
+
## Information Retrieval
|
50 |
+
You can also use **customized embeddings** for information retrieval.
|
51 |
+
```python
|
52 |
+
import numpy as np
|
53 |
+
from sklearn.metrics.pairwise import cosine_similarity
|
54 |
+
query = [['Represent the Wikipedia question for retrieving supporting documents; Input: ','where is the food stored in a yam plant',0]]
|
55 |
+
corpus = [['Represent the Wikipedia document for retrieval; Input: ','Capitalism has been dominant in the Western world since the end of feudalism, but most feel[who?] that the term "mixed economies" more precisely describes most contemporary economies, due to their containing both private-owned and state-owned enterprises. In capitalism, prices determine the demand-supply scale. For example, higher demand for certain goods and services lead to higher prices and lower demand for certain goods lead to lower prices.', 0],
|
56 |
+
['Represent the Wikipedia document for retrieval; Input: ','The disparate impact theory is especially controversial under the Fair Housing Act because the Act regulates many activities relating to housing, insurance, and mortgage loans—and some scholars have argued that the theory's use under the Fair Housing Act, combined with extensions of the Community Reinvestment Act, contributed to rise of sub-prime lending and the crash of the U.S. housing market and ensuing global economic recession',0],
|
57 |
+
['Represent the Wikipedia document for retrieval; Input: ','Disparate impact in United States labor law refers to practices in employment, housing, and other areas that adversely affect one group of people of a protected characteristic more than another, even though rules applied by employers or landlords are formally neutral. Although the protected classes vary by statute, most federal civil rights laws protect based on race, color, religion, national origin, and sex as protected traits, and some laws include disability status and other traits as well.',0]]
|
58 |
+
query_embeddings = model.encode(query)
|
59 |
+
corpus_embeddings = model.encode(corpus)
|
60 |
+
similarities = cosine_similarity(query_embeddings,corpus_embeddings)
|
61 |
+
retrieved_doc_id = np.argmax(similarities)
|
62 |
+
print(retrieved_doc_id)
|
63 |
```
|