K024/chatglm2-6b-int8

ChatGLM2 6b int8 量化模型

详情参考 K024/chatglm-q。

See K024/chatglm-q for more details.

import torch
from chatglm_q.decoder import ChatGLMDecoder, chat_template

device = torch.device("cuda")
decoder = ChatGLMDecoder.from_pretrained("K024/chatglm2-6b-int8", device=device)

prompt = chat_template([], "我是谁？")
for text in decoder.generate(prompt):
    print(text)

模型权重按 ChatGLM2-6b 许可发布，见 MODEL LICENSE。

Model weights are released under the same license as ChatGLM2-6b, see MODEL LICENSE.