Minimal example
#1
by
hermanda
- opened
Thanks for making the temporary working version of the model. Could you please also prepare minimal working example in google colab and link it in Model card? I have hard time configuring it correctly. Or at least please tell me what I am doing incorrectly. https://colab.research.google.com/drive/1d3pF9MBOCFpQ_KqWnWdfuG1w_6CBWjrK?usp=sharing
This version of the model isn't compatible with transformers. It's designed to work with SGLang. With sglang you should be able to do pip install "sglang[all]" and then
python -m sglang.launch_server --model-path SurfaceData/llava-v1.6-mistral-7b-sglang --chat-template vicuna_v1.1 --port 30000
fozziethebeat
changed discussion status to
closed