Triangle104/ADELIE-SFT-3B-Q8_0-GGUF

This model was converted to GGUF format from THU-KEG/ADELIE-SFT-3B using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

Model details:

We introduce ADELIE (Aligning large language moDELs on Information Extraction), an aligned LLM that effectively solves various IE tasks, including closed IE, open IE, and on-demand IE. We first collect and construct a high-quality alignment corpus IEInstruct for IE. Then we train ADELIESFT using instruction tuning on IEInstruct. We further train ADELIESFT with direct preference optimization (DPO) objective, resulting in ADELIEDPO. Extensive experiments on various held-out IE datasets demonstrate that our models (ADELIESFT and ADELIEDPO) achieve state-of-the-art (SoTA) performance among open-source models. We further explore the general capabilities of ADELIE, and experimental results reveal that their general capabilities do not exhibit a noticeable decline.

📖 Paper: ADELIE: Aligning Large Language Models on Information Extraction

🐧 Github: THU/ADELIE

Model Description

Developed by: Yunjia Qi, Hao Peng, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
Model type: Text Generation
Language(s) (NLP): English
License: LLaMA2 License for the base model.
Finetuned from model [optional]: LLaMA3.2-3B

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/ADELIE-SFT-3B-Q8_0-GGUF --hf-file adelie-sft-3b-q8_0.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/ADELIE-SFT-3B-Q8_0-GGUF --hf-file adelie-sft-3b-q8_0.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/ADELIE-SFT-3B-Q8_0-GGUF --hf-file adelie-sft-3b-q8_0.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/ADELIE-SFT-3B-Q8_0-GGUF --hf-file adelie-sft-3b-q8_0.gguf -c 2048