llama-moe

/

LLaMA-MoE-v1-3_5B-4_16

like 15

Follow

LLaMA-MoE 10

Hi there. The model is constructed on LLaMA-2 7B, so it'll take (almost) the same VRAM consumption as the 7B model. ~14GB for the model only, and some extra space for the context cache depending on the context length.

Thanks for ur reply.♥