heyholetsgo/Nous-Hermes-2-Mistral-7B-DPO-AWQ

Nous Hermes 2 - Mistral 7B - DPO - AWQ

Model Description

This repo contains the AWQ quantized version of the Nous Hermes 2 - Mistral 7B - DPO model. It was quantized with AutoAWQ using the following settings:

{"zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM"}