Hugging Face's logo

TheBloke
/
CodeLlama-34B-Instruct-GGUF

Text Generation
Transformers
GGUF
code
llama
llama-2
Model card Files Community
5

Apple M1

#1
by md2 - opened
Discussion
md2

@TheBloke thanks as usual for uploading so quickly these new models!

Anyone succeeding with an Apple M1?

If so, any recommendation on the command line to use with llama.cpp or equivalent?

Any help or suggestions highly appreciated.

Danielbrdz

I have a MacBook Air M1, I can tell you that the 7b version works perfectly on a M1 in LM Studio, if you want to go to the 13b and 34b versions with M1 at once I tell you it will not work well, it will go super slow, I recommend you use the 7b versions.

wezzmeister

I get about 3 tokens/s on the Q5_K_M variant with my M1 Max (64gb ram).
I am using text-generation-webui with the recommended settings for Metal when installing.

Sign up or log in to comment