Copycats
/

EEVE-Korean-Instruct-10.8B-v1.0-AWQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Copycats commited on Apr 3

Commit

17719a0

•

1 Parent(s): 3ecfcab

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -44,7 +44,7 @@ Documentation on installing and using vLLM [can be found here](https://vllm.read
 - vLLM can be deployed as a server that implements the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API
 ```shell
-python3 -m vllm.entrypoints.openai.api_server --model Copycats/EEVE-Korean-Instruct-10.8B-v1.0-AWQ --quantization awq --dtype auto
 ```
 #### Querying the model using OpenAI Chat API:

 - vLLM can be deployed as a server that implements the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API
 ```shell
+python3 -m vllm.entrypoints.openai.api_server --model Copycats/EEVE-Korean-Instruct-10.8B-v1.0-AWQ --quantization awq --dtype float16
 ```
 #### Querying the model using OpenAI Chat API: