Draft model as accelerator for DeepSeek-R1?

#174
by inputout - opened

Is there a compatible draft model for use with llama.cpp for speculative decoding as an accelerator for Deepseek R1?
I have tested some but llama.cpp does not accept them. Is a Draft model even possible in principle with Deepseek R1?

Sign up or log in to comment