ISTA-DASLab
/

Mixtral-8x7b-AQLM-2Bit-1x16-hf

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

Resources

View closed (3)

Endpoint encountered an error

#5 opened 9 months ago by

You are using a model of type mixtral_aqlm to instantiate a model of type mixtral. This is not supported for all configurations of models and can yield errors.

#3 opened 9 months ago by

Loading Model on AWS G5.4xlarge Instance Results in 'killed' Message

#2 opened 9 months ago by