bartowski

/

QVQ-72B-Preview-GGUF

like 52

I was going to say you can run directly in ollama from HF but I actually don't know how that works with vision models.. will look into it !

thank you!

I was going to say you can run directly in ollama from HF but I actually don't know how that works with vision models.. will look into it !

well i imported the gguff into openwebui and....

this happens every time and here is the translation:

, Hello little fairies~
Your editor is online again
Recently I'm following "The Next Stop Is Happiness"
It's really too sweet
Song Weilong and Song Qian have a full sense of CP

wait... this is diffrent a while ago it was yapping about a cook and giving me recipies

Hi, seems like we have a problem parsing GGUF file on hugging face backend. I'll report it to the team

So, this is a real bug? I thought this was a problem on my side, thanks a lot! Now I don't have to debug my Ollama or OWUI.

Yes it's a bug on HF side. I don't know yet when we can deploy a fix, given that many of us are on vacation.

In the meantime, you can try adding the chat template yourself via a Modelfile, ref ollama docs

FROM hf.co/bartowski/QVQ-72B-Preview-GGUF
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""
PARAMETER stop "<|im_end|>"

thank you and also, is the problem with all the ggufs or just the q4km's?

it's for all quants

any updates?

I'll try to take a look tomorrow at the process

❤️ thanks a lot! you are a legend!

Haven't been able to figure it out :( was able to upload a file, but not sure how vision is meant to be uploaded/work, if anyone has insight feel free to let me know but the documentation seems quite lacking

Hi @bartowski , sorry for the late response. It seems like the fix can be applied from HF server side. I'll discuss this with our team this week.

It should work correctly now. Tested using https://huggingface.co./spaces/ngxson/debug_ollama_manifest

======================

Template:
{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>



======================

Parameters:
{
  "stop": [
    "<|im_start|>",
    "<|im_end|>",
    "<|im_start|>",
    "<|im_end|>",
    "<|im_start|>",
    "<|im_end|>",
    "<|im_start|>user"
  ]
}

so i just have redownload and import it and it should work?