michaelfeil commited on
Commit
12f5e10
·
verified ·
1 Parent(s): c9e87c7

Update instruction with infinity

Browse files

Command tested:

```
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" michaelf34/infinity:0.0.68 v2 --model-id intfloat/multilingual-e5-large-instruct --revision "main" --dtype float16 --batch-size 32 --device cuda --engine torch --port 7997

INFO: Started server process [1]
INFO: Waiting for application startup.
INFO 2024-11-13 00:46:42,260 infinity_emb INFO: infinity_server.py:89
Creating 1engines:
engines=['intfloat/multilingual-e5-large-instruct
']
INFO 2024-11-13 00:46:42,264 infinity_emb INFO: Anonymized telemetry.py:30
telemetry can be disabled via environment variable
`DO_NOT_TRACK=1`.
INFO 2024-11-13 00:46:42,272 infinity_emb INFO: select_model.py:64
model=`intfloat/multilingual-e5-large-instruct`
selected, using engine=`torch` and device=`cuda`
INFO 2024-11-13 00:46:42,367 SentenceTransformer.py:216
sentence_transformers.SentenceTransformer
INFO: Load pretrained SentenceTransformer:
intfloat/multilingual-e5-large-instruct
INFO 2024-11-13 00:46:46,145 infinity_emb INFO: Adding acceleration.py:56
optimizations via Huggingface optimum.
```

Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -5466,6 +5466,16 @@ print(scores.tolist())
5466
  # [[91.92853546142578, 67.5802993774414], [70.38143157958984, 92.13307189941406]]
5467
  ```
5468
 
 
 
 
 
 
 
 
 
 
 
5469
  ## Supported Languages
5470
 
5471
  This model is initialized from [xlm-roberta-large](https://huggingface.co/xlm-roberta-large)
 
5466
  # [[91.92853546142578, 67.5802993774414], [70.38143157958984, 92.13307189941406]]
5467
  ```
5468
 
5469
+ ### Infinity
5470
+
5471
+ Usage with [Infinity](https://github.com/michaelfeil/infinity):
5472
+
5473
+ ```bash
5474
+ docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
5475
+ michaelf34/infinity:0.0.68 \
5476
+ v2 --model-id intfloat/multilingual-e5-large-instruct --revision "main" --dtype float16 --batch-size 32 -engine torch --port 7997
5477
+ ```
5478
+
5479
  ## Supported Languages
5480
 
5481
  This model is initialized from [xlm-roberta-large](https://huggingface.co/xlm-roberta-large)