Free and ready to use zephyr-7B-beta-GGUF model as OpenAI API compatible endpoint
Hi there,
I deployed the model as OpenAI API compatible endpoint at https://huggingface.co./spaces/limcheekin/zephyr-7B-beta-GGUF.
If you find this resource valuable, your support in the form of starring the space would be greatly appreciated.
Thank you.
Hi, are you still maintaining this? I tried using it but it seems to time out/not return a response. Perhaps it's overloaded or some other issue. The only endpoint I can get to successfully return a response is the GET /v1/models. Thanks in advance,
I just tested with the API doc at https://limcheekin-zephyr-7b-beta-gguf.hf.space/docs. It is working for a simple prompt.
If you use it for longer generation/response, you need to set stream=True.
Please refer to the following notebook at https://huggingface.co./spaces/limcheekin/Mistral-7B-Instruct-v0.1-GGUF/blob/main/mistral-7b-instruct.ipynb.
Hope this help.
Hi, are you still maintaining this? I tried using it but it seems to time out/not return a response. Perhaps it's overloaded or some other issue. The only endpoint I can get to successfully return a response is the
GET /v1/models. Thanks in advance,
Added a jupyter notebook at https://huggingface.co./spaces/limcheekin/zephyr-7B-beta-GGUF/blob/main/zephyr-7b.ipynb
I hope this is helpful to you. :)
How can I create my own o'penai API with zephyr-7b-beta and share your code? Thank you
You just need to select "Duplicate this Space" from the drop down menu.
All codes are located at https://huggingface.co./spaces/limcheekin/zephyr-7B-beta-GGUF/tree/main.
By the way, you can use git to clone the repo of the space.
I hope this helps and clarify.
Thank you very much. I have just learned about Zephyr, and I have seen your way of using CPP. I would like to directly use Zephyr-7b-beta, while also implementing stream output.
I find ohter coder create api by using pipeline in the github,i want to use AutoModelForCausalLM.from_pretrained.
I didn't do that, can't help you. Appreciate your sharing if you managed to publish the api.
Thanks.
i got an error:
AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = ', or you can set the environment variable OPENAI_API_KEY=). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path =
i got an error:
AuthenticationError: No API key provided. You can set your API key in code using 'openai.api_key = ', or you can set the environment variable OPENAI_API_KEY=). If your API key is stored in a file, you can point the openai module at it with 'openai.api_key_path =
Set your HF token as the API key.