Hugging Face's logo

FriendliAI
/
Mixtral-8x7B-Instruct-v0.1-fp8

Text Generation
Transformers
Safetensors
mixtral
pretrained
conversational
text-generation-inference
8-bit precision
Model card Files Community
yunmorning commited on
Commit
77f8af9
•
1 Parent(s): 4eb2929

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -85,7 +85,7 @@ export POLICY_DIR=$PWD/policy
85
  mkdir -p $POLICY_DIR
86
 
87
  docker run \
88
- --gpus $GPU_ENUMERATION \
89
  -p 8000:8000 \
90
  -v ~/.cache/huggingface:/root/.cache/huggingface \
91
  -v $POLICY_DIR:/policy \
@@ -102,7 +102,7 @@ Now you can create an inference endpoint with this optimal policy as follows:
102
 
103
  ```sh
104
  docker run \
105
- --gpus $GPU_ENUMERATION \
106
  -p 8000:8000 \
107
  -v ~/.cache/huggingface:/root/.cache/huggingface \
108
  -v $POLICY_DIR:/policy \
 
85
  mkdir -p $POLICY_DIR
86
 
87
  docker run \
88
+ --gpus '"device=0"' \
89
  -p 8000:8000 \
90
  -v ~/.cache/huggingface:/root/.cache/huggingface \
91
  -v $POLICY_DIR:/policy \
 
102
 
103
  ```sh
104
  docker run \
105
+ --gpus '"device=0"' \
106
  -p 8000:8000 \
107
  -v ~/.cache/huggingface:/root/.cache/huggingface \
108
  -v $POLICY_DIR:/policy \