Cannot download the model
#1
by
JimmyC
- opened
This comment has been hidden
I have followed the commands given in https://huggingface.co./blog/text-generation-inference-on-inferentia2 but I am still getting the message
WARNING:sagemaker:Your model is not compiled. Please compile your model before using Inferentia.
when attempting to run the deploy line:
llm = llm_model.deploy(
initial_instance_count=1,
instance_type=instance_type,
container_startup_health_check_timeout=health_check_timeout,
)
Hey @charlescearl ,
Leaving this here for anyone else that comes across it. The warning from sagemaker is something that you will (currently) see for every Inferentia model you try to load regardless.