TheBloke

/

sqlcoder2-GGUF

like 25

I'm seeing the same error loading sqlcoder2.Q4_K_M.gguf in text-generation-webui via llama.cpp model loader.

ERROR: byte not found in vocab: '
'
Segmentation fault (core dumped)

Exactly!!! Getting the same error on SqlCoder2.Q5_K_M.gguf and also Q5_0. I think we should just keep using the SQLCoder for now :)
Any hope for this @TheBloke ?

Thanks!

@AayushShah What models have you been using for SQLGen? Do you know any benchmarks/blog/discussions on the efficiency of LLMs for SQLGen. I've been trying code llama to a moderate level of success.

what is the reason??? Failed to create LLM 'starcoder' from '/root/.cache/huggingface/hub/models--TheBloke--sqlcoder2-GGUF/blobs/b5e26875dc981af3ef803aef36a7f6da08d75e9ea5484a95d1bf2aa622ac3cb0'.

@mvalente
Yeah actually I had very high hopes for SQLCoder-2 and since it was not working I tried running it on A5000 GPU but still it wasn't good as I expected it.
As you have found, me too.
CodeLlama is literally understanding the instructions and giving good results with almost all times proper grammar (valid SQL).

So for now, I think codellama-7b is promising model for me.
Other models I have tried:

• Zephyer: This is amazing model. It can handle amazing queries but it is not commercially usable and is general purpose so can't beat codellama as of now.
• Wizard-Coder: It is good for small and simple queries but not as efficient as code llama
• NumbersStation's 2B model for SQL: It seem great in the start, but it doesn't have the GGUF support. Need to test more for my usecase, still it is 2B model at most. But they have Llama-7B version too. You may check that out as well (probably the model isn't capable of understanding the instructions... but worth checking out)

I am expecting to test more models like:

• Mistral
• Llama-instruct (by together)

Let me know if you get any success with any model or other model, I am still figuring out.
Thanks.