OPT-13B-Nerybus-Mix-4bit-128g
Model description
Warning: THIS model is NOT suitable for use by minors. The model will output X-rated content.
This is a 4-bit GPTQ quantization of OPT-13B-Nerybus-Mix, original model: https://huggingface.co./KoboldAI/OPT-13B-Nerybus-Mix
Quantization Information
Quantized with: https://github.com/0cc4m/GPTQ-for-LLaMa
python repos/gptq/opt.py --wbits 4 models/KoboldAI_OPT-13B-Nerybus-Mix c4 --groupsize 128 --save models/KoboldAI_OPT-13B-Nerybus-Mix/OPT-13B-Nerybus-Mix-4bit-128g.pt
python repos/gptq/opt.py --wbits 4 models/KoboldAI_OPT-13B-Nerybus-Mix c4 --groupsize 128 --save_safetensors models/KoboldAI_OPT-13B-Nerybus-Mix/OPT-13B-Nerybus-Mix-4bit-128g.safetensors
License
OPT-13B is licensed under the OPT-175B license, Copyright (c) Meta Platforms, Inc. All Rights Reserved.
- Downloads last month
- 1,564
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model authors have turned it off explicitly.