danf0 commited on
Commit
e094451
1 Parent(s): b2dcfc6

Update README

Browse files
Files changed (1) hide show
  1. README.md +52 -60
README.md CHANGED
@@ -80,75 +80,67 @@ Given n samples, the value of the Vendi Score ranges between 1 and n, with highe
80
 
81
  ### Examples
82
 
83
- ```python
84
- import numpy as np
85
- vendiscore = evaluate.load("danf0/vendiscore")
86
-
87
- samples = [0, 0, 10, 10, 20, 20]
88
- k = lambda a, b: np.exp(-np.abs(a - b))
89
-
90
- vendiscore.compute(samples, k)
91
-
92
- # 2.9999
93
  ```
94
 
95
  If you already have precomputed a similarity matrix:
96
- ```python
97
- K = np.array([[1.0, 0.9, 0.0],
98
- [0.9, 1.0, 0.0],
99
- [0.0, 0.0, 1.0]])
100
- vendiscore.compute(K, score_K=True)
 
 
101
 
102
- # 2.1573
 
 
 
 
 
 
 
103
  ```
104
 
105
- If your similarity function is a dot product between normalized
106
- embeddings $X\in\mathbb{R}^{n\times d}$, and $d < n$, it is faster
107
- to compute the Vendi Score using the covariance matrix,
108
- $\frac{1}{n} \sum_i x_i x_i^{\top}$:
109
- ```python
110
- vendiscore.compute(X, score_dual=True)
111
  ```
112
- If the rows of $X$ are not normalized, set `normalize = True`.
113
-
114
- Images:
115
- ```python
116
- from torchvision import datasets
117
-
118
- mnist = datasets.MNIST("data/mnist", train=False, download=True)
119
- digits = [[x for x, y in mnist if y == c] for c in range(10)]
120
- pixel_vs = [vendiscore.compute(imgs, k="pixels") for imgs in digits]
121
- # The default embeddings are from the pool-2048 layer of the torchvision
122
- # Inception v3 model.
123
- inception_vs = [vendiscore.compute(imgs, k="image_embeddings", batch_size=64, device="cuda") for imgs in digits]
124
- for y, (pvs, ivs) in enumerate(zip(pixel_vs, inception_vs)): print(f"{y}\t{pvs:.02f}\t{ivs:02f}")
125
-
126
- # Output:
127
- # 0 7.68 3.45
128
- # 1 5.31 3.50
129
- # 2 12.18 3.62
130
- # 3 9.97 2.97
131
- # 4 11.10 3.75
132
- # 5 13.51 3.16
133
- # 6 9.06 3.63
134
- # 7 9.58 4.07
135
- # 8 9.69 3.74
136
- # 9 8.56 3.43
137
  ```
138
 
139
- Text:
140
- ```python
141
- sents = ["Look, Jane.",
142
- "See Spot.",
143
- "See Spot run.",
144
- "Run, Spot, run.",
145
- "Jane sees Spot run."]
146
- ngram_vs = vendiscore.compute(sents, k="ngram_overlap", ns=[1, 2])
147
- bert_vs = vendiscore.compute(sents, k="text_embeddings", model_path="bert-base-uncased")
148
- simcse_vs = vendiscore.compute(sents, k="text_embeddings", model_path="princeton-nlp/unsup-simcse-bert-base-uncased")
149
- print(f"N-grams: {ngram_vs:.02f}, BERT: {bert_vs:.02f}, SimCSE: {simcse_vs:.02f})
150
-
151
- # N-grams: 3.91, BERT: 1.21, SimCSE: 2.81
152
  ```
153
 
154
  ## Limitations and Bias
 
80
 
81
  ### Examples
82
 
83
+ ```
84
+ >>> import numpy as np
85
+ >>> vendiscore = evaluate.load("danf0/vendiscore")
86
+ >>> samples = [0, 0, 10, 10, 20, 20]
87
+ >>> k = lambda a, b: np.exp(-np.abs(a - b))
88
+ >>> vendiscore.compute(samples, k)
89
+ 2.9999
 
 
 
90
  ```
91
 
92
  If you already have precomputed a similarity matrix:
93
+ ```
94
+ >>> K = np.array([[1.0, 0.9, 0.0],
95
+ [0.9, 1.0, 0.0],
96
+ [0.0, 0.0, 1.0]])
97
+ >>> vendiscore.compute(K, score_K=True)
98
+ 2.1573
99
+ ```
100
 
101
+ If your similarity function is a dot product between `n` normalized
102
+ `d`-dimensional embeddings `X`, and `d` < `n`, it is faster
103
+ to compute the Vendi Score using the covariance matrix, `X @ X.T`.
104
+ (If the rows of `X` are not normalized, set `normalize = True`.)
105
+ ```
106
+ >>> X = np.array([[100, 0], [99, 1], [1, 99], [0, 100])
107
+ >>> vendiscore.compute(X, score_dual=True, normalize=True)
108
+ 1.9989...
109
  ```
110
 
111
+ Image similarity can be calculated using inner products between pixel vectors or between embeddings from a neural network.
112
+ The default embeddings are from the pool-2048 layer of the torchvision version of the Inception v3 model; other embedding functions can be passed to the `model` argument.
 
 
 
 
113
  ```
114
+ >>> from torchvision import datasets
115
+ >>> mnist = datasets.MNIST("data/mnist", train=False, download=True)
116
+ >>> digits = [[x for x, y in mnist if y == c] for c in range(10)]
117
+ >>> pixel_vs = [vendiscore.compute(imgs, k="pixels") for imgs in digits]
118
+ >>> inception_vs = [vendiscore.compute(imgs, k="image_embeddings", batch_size=64, device="cuda") for imgs in digits]
119
+ >>> for y, (pvs, ivs) in enumerate(zip(pixel_vs, inception_vs)): print(f"{y}\t{pvs:.02f}\t{ivs:02f}")
120
+ 0 7.68 3.45
121
+ 1 5.31 3.50
122
+ 2 12.18 3.62
123
+ 3 9.97 2.97
124
+ 4 11.10 3.75
125
+ 5 13.51 3.16
126
+ 6 9.06 3.63
127
+ 7 9.58 4.07
128
+ 8 9.69 3.74
129
+ 9 8.56 3.43
 
 
 
 
 
 
 
 
 
130
  ```
131
 
132
+ Text similarity can be calculated using n-gram overlap or using inner products between embeddings from a neural network.
133
+ ```
134
+ >>> sents = ["Look, Jane.",
135
+ "See Spot.",
136
+ "See Spot run.",
137
+ "Run, Spot, run.",
138
+ "Jane sees Spot run."]
139
+ >>> ngram_vs = vendiscore.compute(sents, k="ngram_overlap", ns=[1, 2])
140
+ >>> bert_vs = vendiscore.compute(sents, k="text_embeddings", model_path="bert-base-uncased")
141
+ >>> simcse_vs = vendiscore.compute(sents, k="text_embeddings", model_path="princeton-nlp/unsup-simcse-bert-base-uncased")
142
+ >>> print(f"N-grams: {ngram_vs:.02f}, BERT: {bert_vs:.02f}, SimCSE: {simcse_vs:.02f})
143
+ N-grams: 3.91, BERT: 1.21, SimCSE: 2.81
 
144
  ```
145
 
146
  ## Limitations and Bias