Xenova HF staff commited on
Commit
2bf70a2
1 Parent(s): 770c52e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +70 -0
README.md CHANGED
@@ -3,5 +3,75 @@ library_name: transformers.js
3
  ---
4
 
5
  https://huggingface.co/google/siglip-large-patch16-384 with ONNX weights to be compatible with Transformers.js.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6
 
7
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).
 
3
  ---
4
 
5
  https://huggingface.co/google/siglip-large-patch16-384 with ONNX weights to be compatible with Transformers.js.
6
+ ## Usage (Transformers.js)
7
+
8
+ If you haven't already, you can install the [Transformers.js](https://huggingface.co/docs/transformers.js) JavaScript library from [NPM](https://www.npmjs.com/package/@xenova/transformers) using:
9
+ ```bash
10
+ npm i @xenova/transformers
11
+ ```
12
+
13
+ **Example:** Zero-shot image classification w/ `Xenova/siglip-large-patch16-384`:
14
+ ```js
15
+ import { pipeline } from '@xenova/transformers';
16
+
17
+ const classifier = await pipeline('zero-shot-image-classification', 'Xenova/siglip-large-patch16-384');
18
+ const url = 'http://images.cocodataset.org/val2017/000000039769.jpg';
19
+ const output = await classifier(url, ['2 cats', '2 dogs'], {
20
+ hypothesis_template: 'a photo of {}',
21
+ });
22
+ console.log(output);
23
+ // [
24
+ // { score: 0.4783420264720917, label: '2 cats' },
25
+ // { score: 0.00022271279885899276, label: '2 dogs' }
26
+ // ]
27
+ ```
28
+
29
+ **Example:** Compute text embeddings with `SiglipTextModel`.
30
+
31
+ ```javascript
32
+ import { AutoTokenizer, SiglipTextModel } from '@xenova/transformers';
33
+
34
+ // Load tokenizer and text model
35
+ const tokenizer = await AutoTokenizer.from_pretrained('Xenova/siglip-large-patch16-384');
36
+ const text_model = await SiglipTextModel.from_pretrained('Xenova/siglip-large-patch16-384');
37
+
38
+ // Run tokenization
39
+ const texts = ['a photo of 2 cats', 'a photo of 2 dogs'];
40
+ const text_inputs = tokenizer(texts, { padding: 'max_length', truncation: true });
41
+
42
+ // Compute embeddings
43
+ const { pooler_output } = await text_model(text_inputs);
44
+ // Tensor {
45
+ // dims: [ 2, 768 ],
46
+ // type: 'float32',
47
+ // data: Float32Array(1536) [ ... ],
48
+ // size: 1536
49
+ // }
50
+ ```
51
+
52
+ **Example:** Compute vision embeddings with `SiglipVisionModel`.
53
+
54
+ ```javascript
55
+ import { AutoProcessor, SiglipVisionModel, RawImage} from '@xenova/transformers';
56
+
57
+ // Load processor and vision model
58
+ const processor = await AutoProcessor.from_pretrained('Xenova/siglip-large-patch16-384');
59
+ const vision_model = await SiglipVisionModel.from_pretrained('Xenova/siglip-large-patch16-384');
60
+
61
+ // Read image and run processor
62
+ const image = await RawImage.read('https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/football-match.jpg');
63
+ const image_inputs = await processor(image);
64
+
65
+ // Compute embeddings
66
+ const { pooler_output } = await vision_model(image_inputs);
67
+ // Tensor {
68
+ // dims: [ 1, 768 ],
69
+ // type: 'float32',
70
+ // data: Float32Array(768) [ ... ],
71
+ // size: 768
72
+ // }
73
+ ```
74
+
75
+ ---
76
 
77
  Note: Having a separate repo for ONNX weights is intended to be a temporary solution until WebML gains more traction. If you would like to make your models web-ready, we recommend converting to ONNX using [🤗 Optimum](https://huggingface.co/docs/optimum/index) and structuring your repo like this one (with ONNX weights located in a subfolder named `onnx`).