NeoChen1024
/

dolphin-2.7-mixtral-8x7b-GGUF-IQ3_M

Inference Endpoints

Model card Files Files and versions Community

dolphin-2.7-mixtral-8x7b-GGUF-IQ3_M / README.md

NeoChen1024's picture

Update README.md

054a565 verified about 2 months ago

|

233 Bytes

metadata

license: apache-2.0
base_model:
  - cognitivecomputations/dolphin-2.7-mixtral-8x7b

GGUF IQ3_M quant of cognitivecomputations/dolphin-2.7-mixtral-8x7b
It fits into 24GiB VRAM with 32768 context (@ 8bit KV cache quantization).