image/webp

Model Description

Model Details

  • Name: Carrot Llama-3.2 Rabbit Ko 2412
  • Version: 3B Instruct
  • Base Model: CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct
  • Languages: Korean, English
  • Model Type: Large Language Model (Instruction-tuned)

Training Process

๋ณธ ๋ชจ๋ธ์€ ๋‹ค์Œ๊ณผ ๊ฐ™์€ ์ฃผ์š” ํ›ˆ๋ จ ๋‹จ๊ณ„๋ฅผ ๊ฑฐ์ณค์Šต๋‹ˆ๋‹ค:

  1. SFT (Supervised Fine-Tuning)

    • ๊ณ ํ’ˆ์งˆ ํ•œ๊ตญ์–ด ๋ฐ ์˜์–ด ๋ฐ์ดํ„ฐ์…‹์„ ์‚ฌ์šฉํ•˜์—ฌ ๊ธฐ๋ณธ ๋ชจ๋ธ์„ ์„ธ๋ถ€ ์กฐ์ •
  2. DPO (Direct Preference Optimization)

    • ์ธ๊ฐ„์˜ ์„ ํ˜ธ๋„๋ฅผ ์ง์ ‘์ ์œผ๋กœ ๋ฐ˜์˜ํ•˜์—ฌ ๋ชจ๋ธ์˜ ์‘๋‹ต ํ’ˆ์งˆ ๊ฐœ์„ 

Limitations

  • 3B ํŒŒ๋ผ๋ฏธํ„ฐ ๊ทœ๋ชจ๋กœ ์ธํ•œ ๋ณต์žกํ•œ ์ž‘์—…์—์„œ์˜ ์ œํ•œ์  ์„ฑ๋Šฅ
  • ํŠน์ • ๋„๋ฉ”์ธ์— ๋Œ€ํ•œ ๊นŠ์ด ์žˆ๋Š” ์ „๋ฌธ์„ฑ ๋ถ€์กฑ
  • ํŽธํ–ฅ์„ฑ ๋ฐ ํ™˜๊ฐ ๊ฐ€๋Šฅ์„ฑ

Ethics Statement

๋ชจ๋ธ ๊ฐœ๋ฐœ ๊ณผ์ •์—์„œ ์œค๋ฆฌ์  ๊ณ ๋ ค์‚ฌํ•ญ์„ ์ตœ๋Œ€ํ•œ ๋ฐ˜์˜ํ•˜์˜€์œผ๋‚˜, ์‚ฌ์šฉ์ž๋Š” ํ•ญ์ƒ ๊ฒฐ๊ณผ๋ฅผ ๋น„ํŒ์ ์œผ๋กœ ๊ฒ€ํ† ํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค.

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412")
tokenizer = AutoTokenizer.from_pretrained("CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412")

Score

Performance Metrics

LogicKor

Category Single turn Multi turn
์ˆ˜ํ•™(Math) 5.86 5.14
๋ฌธ๋ฒ•(Grammar) 4.71 1.29
์ดํ•ด(Understanding) 4.00 4.43
์ถ”๋ก (Reasoning) 5.14 6.71
์ฝ”๋”ฉ(Coding) 7.43 7.57
๊ธ€์“ฐ๊ธฐ(Writing) 8.43 8.00
Total 5.93 5.52
Overall 5.73
Tasks Version Filter n-shot Metric Value Stderr
gsm8k 3 flexible-extract 5 exact_match โ†‘ 0.7013 ยฑ 0.0126
strict-match 5 exact_match โ†‘ 0.2418 ยฑ 0.0118
gsm8k-ko 1 flexible-extract 5 exact_match โ†‘ 0.4466 ยฑ 0.0137
strict-match 5 exact_match โ†‘ 0.4420 ยฑ 0.0137
ifeval 4 none 0 inst_level_loose_acc โ†‘ 0.8549 ยฑ N/A
none 0 inst_level_strict_acc โ†‘ 0.8225 ยฑ N/A
none 0 prompt_level_loose_acc โ†‘ 0.7874 ยฑ 0.0176
none 0 prompt_level_strict_acc โ†‘ 0.7468 ยฑ 0.0187
Task Score shot
haerae 43.26 5
@article{Llama3.2RabbitKo3BInstruct,
  title={CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412 Card},
  author={CarrotAI (L, GEUN)},
  year={2024},
  url = {https://huggingface.co./CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412}
}
Downloads last month
28
Safetensors
Model size
3.21B params
Tensor type
FP16
ยท
Inference Examples
Unable to determine this model's library. Check the docs .

Model tree for CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412

Collection including CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412