Cannot download the model

#1
by JimmyC - opened
This comment has been hidden

I have followed the commands given in https://huggingface.co./blog/text-generation-inference-on-inferentia2 but I am still getting the message

WARNING:sagemaker:Your model is not compiled. Please compile your model before using Inferentia.

when attempting to run the deploy line:

llm = llm_model.deploy(
        initial_instance_count=1,
        instance_type=instance_type,
        container_startup_health_check_timeout=health_check_timeout,
    )
AWS Inferentia and Trainium org

Hey @charlescearl ,

Leaving this here for anyone else that comes across it. The warning from sagemaker is something that you will (currently) see for every Inferentia model you try to load regardless.

Sign up or log in to comment