OpenAI compatible endpoint using VLLM

Runs well on 4090

python -m vllm.entrypoints.openai.api_server --model macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ  --max-model-len 25000

Downloads last month: 6

Safetensors

Model size

3.46B params

Tensor type

I32

FP16

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported third-party Inference Providers, and the model is not deployed on the HF Inference API.

Model tree for macadeliccc/laser-dolphin-mixtral-4x7b-dpo-AWQ

Base model

macadeliccc/laser-dolphin-mixtral-4x7b-dpo

Quantized

(4)

this model