meta-llama

/

Meta-Llama-3.1-70B-Instruct

like 492

Hi there,

I was trying to build agent agent-based application by using llama3.1 models and it is on AWS EC2. I need suggestions of which instance should I opt for which will be capable of running the models cost-effectively.

I explored the GPU requirement of the model from the hugging face blog, here
https://huggingface.co./blog/llama31#whats-new-with-llama-31

But I'm still sceptical about choosing which instance type should I go for.

Thanks for your idea and for taking the time to reply to this topic.