I have converted the thing in GPTQ-v2

Group size 128 for one

Act order for another

and neither for a third

Let's see how many I upload.

Also converted to GGML in the GGML folder. Works on CPU.

Downloads last month: 37

Inference Examples

Text Generation

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.