Commit
·
8687cf1
1
Parent(s):
f6179b9
Update README.md
Browse files
README.md
CHANGED
@@ -1,16 +1,70 @@
|
|
1 |
-
|
2 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
3 |
|
4 |
### Convert a model to ONNX format
|
5 |
|
6 |
#### Converting a Hugging Face Transformers Model
|
7 |
To convert any model from Hugging Face to ONNX format, you can follow the instructions in [this link](https://huggingface.co/docs/transformers/serialization#export-to-onnx) using the ```optimum-cli```.
|
|
|
8 |
#### Converting a PyTorch Model
|
9 |
If you have a PyTorch model, you can use the ```torch.onnx``` APIs to convert it to the ONNX format. More information on the conversion process can be found [here](https://pytorch.org/docs/stable/onnx.html).
|
|
|
10 |
#### Converting a Tensorflow Model
|
11 |
For Tensorflow models, you can utilize the tf2onnx tool to convert them to the ONNX format. Detailed guidance on this conversion can be found [here](https://onnxruntime.ai/docs/tutorials/tf-get-started.html#getting-started-converting-tensorflow-to-onnx).
|
12 |
|
13 |
-
|
|
|
14 |
Before submitting your ONNX model through a PR, you need to organize the necessary files under a folder with the model's name. Ensure that your model configuration adheres to the following structure:
|
15 |
|
16 |
- **Model File**: The ONNX model file.
|
@@ -28,4 +82,4 @@ Before submitting your ONNX model through a PR, you need to organize the necessa
|
|
28 |
|
29 |
Please make sure that the information in the configuration file is accurate and complete before submitting your PR.
|
30 |
|
31 |
-
We appreciate your contributions to expand our collection of supported embedding models!
|
|
|
1 |
+
---
|
2 |
+
license: gpl-3.0
|
3 |
+
tags:
|
4 |
+
- typesense
|
5 |
+
- semantic search
|
6 |
+
- vector search
|
7 |
+
---
|
8 |
+
|
9 |
+
# Typesense Built-in Embedding Models
|
10 |
+
|
11 |
+
This repository holds all the built-in ML models supported by [Typesense](https://typesense.org) for semantic search currently.
|
12 |
+
|
13 |
+
If you have a model that you would like to add to our supported list, you can convert it to the ONNX format and create a Pull Request (PR) to include it. (See below for instructions).
|
14 |
+
|
15 |
+
## Usage
|
16 |
+
|
17 |
+
Here's an example of how to specify the model to use for auto-embedding generation when creating a collection in Typesense:
|
18 |
+
|
19 |
+
```bash
|
20 |
+
curl -X POST \
|
21 |
+
'http://localhost:8108/collections' \
|
22 |
+
-H 'Content-Type: application/json' \
|
23 |
+
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
|
24 |
+
-d '{
|
25 |
+
"name": "products",
|
26 |
+
"fields": [
|
27 |
+
{
|
28 |
+
"name": "product_name",
|
29 |
+
"type": "string"
|
30 |
+
},
|
31 |
+
{
|
32 |
+
"name": "embedding",
|
33 |
+
"type": "float[]",
|
34 |
+
"embed": {
|
35 |
+
"from": [
|
36 |
+
"product_name"
|
37 |
+
],
|
38 |
+
"model_config": {
|
39 |
+
"model_name": "ts/all-MiniLM-L12-v2"
|
40 |
+
}
|
41 |
+
}
|
42 |
+
}
|
43 |
+
]
|
44 |
+
}'
|
45 |
+
```
|
46 |
+
|
47 |
+
Replace `all-MiniLM-L12-v2` with any model name from this repository.
|
48 |
+
|
49 |
+
Here's a detailed step-by-step article with more information: https://typesense.org/docs/guide/semantic-search.html
|
50 |
+
|
51 |
+
## Contributing
|
52 |
+
|
53 |
+
If you have a model that you would like to add to our supported list, you can convert it to the ONNX format and create a Pull Request (PR) to include it. (See below for instructions).
|
54 |
|
55 |
### Convert a model to ONNX format
|
56 |
|
57 |
#### Converting a Hugging Face Transformers Model
|
58 |
To convert any model from Hugging Face to ONNX format, you can follow the instructions in [this link](https://huggingface.co/docs/transformers/serialization#export-to-onnx) using the ```optimum-cli```.
|
59 |
+
|
60 |
#### Converting a PyTorch Model
|
61 |
If you have a PyTorch model, you can use the ```torch.onnx``` APIs to convert it to the ONNX format. More information on the conversion process can be found [here](https://pytorch.org/docs/stable/onnx.html).
|
62 |
+
|
63 |
#### Converting a Tensorflow Model
|
64 |
For Tensorflow models, you can utilize the tf2onnx tool to convert them to the ONNX format. Detailed guidance on this conversion can be found [here](https://onnxruntime.ai/docs/tutorials/tf-get-started.html#getting-started-converting-tensorflow-to-onnx).
|
65 |
|
66 |
+
#### Creating model config
|
67 |
+
|
68 |
Before submitting your ONNX model through a PR, you need to organize the necessary files under a folder with the model's name. Ensure that your model configuration adheres to the following structure:
|
69 |
|
70 |
- **Model File**: The ONNX model file.
|
|
|
82 |
|
83 |
Please make sure that the information in the configuration file is accurate and complete before submitting your PR.
|
84 |
|
85 |
+
We appreciate your contributions to expand our collection of supported embedding models!
|