QuantFactory/codegeex4-all-9b-GGUF

This is quantized version of THUDM/codegeex4-all-9b created using llama.cpp

Model Description

CodeGeeX4: Open Multilingual Code Generation Model

中文

We introduce CodeGeeX4-ALL-9B, the open-source version of the latest CodeGeeX4 model series. It is a multilingual code generation model continually trained on the GLM-4-9B, significantly enhancing its code generation capabilities. Using a single CodeGeeX4-ALL-9B model, it can support comprehensive functions such as code completion and generation, code interpreter, web search, function call, repository-level code Q&A, covering various scenarios of software development. CodeGeeX4-ALL-9B has achieved highly competitive performance on public benchmarks, such as BigCodeBench and NaturalCodeBench. It is currently the most powerful code generation model with less than 10B parameters, even surpassing much larger general-purpose models, achieving the best balance in terms of inference speed and model performance.

Get Started

Use 4.39.0<=transformers<=4.40.2 to quickly launch codegeex4-all-9b

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

device = "cuda" if torch.cuda.is_available() else "cpu"
tokenizer = AutoTokenizer.from_pretrained("THUDM/codegeex4-all-9b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    "THUDM/codegeex4-all-9b",
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True
).to(device).eval()
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "write a quick sort"}], add_generation_prompt=True, tokenize=True, return_tensors="pt", return_dict=True ).to(device)
with torch.no_grad():
    outputs = model.generate(**inputs)
    outputs = outputs[:, inputs['input_ids'].shape[1]:]
    print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Evaluation

Model Seq Length HumanEval MBPP NCB LCB HumanEvalFIM CRUXEval-O
Llama3-70B-intruct 8K 77.4 82.3 37.0 27.4 - -
DeepSeek Coder 33B Instruct 16K 81.1 80.4 39.3 29.3 78.2 49.9
Codestral-22B 32K 81.1 78.2 46.0 35.3 91.6 51.3
CodeGeeX4-All-9B 128K 82.3 75.7 40.4 28.5 85.0 47.1

Model License

The model weights are licensed under the following License.

Model Citation

If you find our work helpful, please feel free to cite the following paper:

@inproceedings{zheng2023codegeex,
  title={CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-X},
  author={Qinkai Zheng and Xiao Xia and Xu Zou and Yuxiao Dong and Shan Wang and Yufei Xue and Zihan Wang and Lei Shen and Andi Wang and Yang Li and Teng Su and Zhilin Yang and Jie Tang},
  booktitle={Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
  pages={5673--5684},
  year={2023}
}
Downloads last month
288
GGUF
Model size
9.4B params
Architecture
chatglm

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Inference Examples
Inference API (serverless) has been turned off for this model.

Model tree for QuantFactory/codegeex4-all-9b-GGUF

Quantized
(5)
this model