I Used the docker image to use this model.
When I hit the api it return the error that socket hung up. I have 16 gb Nvidia 4060TI.

Call the server using curl:

curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "meta-llama/Llama-3.2-1B"
"messages": [
{"role": "user", "content": "Hello!"}
]
}'