---
license: llama3.2
datasets:
- CarrotAI/Magpie-Ko-Pro-AIR
- CarrotAI/Carrot
- CarrotAI/ko-instruction-dataset
language:
- ko
- en
base_model:
- meta-llama/Llama-3.2-3B-Instruct
pipeline_tag: text-generation
new_version: CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct-2412
---

![image/webp](https://cdn-uploads.huggingface.co/production/uploads/64633ebb39359568c63b52ad/VMYmYpoc0pQVSgZI0kme0.webp)

## Model Description

### Model Details
- **Name**: Carrot Llama-3.2 Rabbit Ko
- **Version**: 3B Instruct
- **Base Model**: CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct
- **Languages**: Korean, English
- **Model Type**: Large Language Model (Instruction-tuned)

### Training Process
본 모델은 다음과 같은 주요 훈련 단계를 거쳤습니다:

1. **SFT (Supervised Fine-Tuning)**
   - 고품질 한국어 및 영어 데이터셋을 사용하여 기본 모델을 세부 조정

### Limitations
- 3B 파라미터 규모로 인한 복잡한 작업에서의 제한적 성능
- 특정 도메인에 대한 깊이 있는 전문성 부족
- 편향성 및 환각 가능성

### Ethics Statement
모델 개발 과정에서 윤리적 고려사항을 최대한 반영하였으나, 사용자는 항상 결과를 비판적으로 검토해야 합니다.

### How to Use
```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct")
tokenizer = AutoTokenizer.from_pretrained("CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct")
```

## Score

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.6490|±  |0.0131|
|     |       |strict-match    |     5|exact_match|↑  |0.0023|±  |0.0013|
|gsm8k-ko|      3|flexible-extract|     5|exact_match|↑  |0.3275|±  |0.0134|
|     |       |strict-match    |     5|exact_match|↑  |0.2737|±  |0.0134|
|ifeval|      4|none           |     5|inst_level_loose_acc   |↑  |0.8058|±  |  N/A|
|     |       |none            |     5|inst_level_strict_acc  |↑  |0.7686|±  |  N/A|
|     |       |none            |     5|prompt_level_loose_acc |↑  |0.7320|±  |0.0191|
|     |       |none            |     5|prompt_level_strict_acc|↑  |0.6858|±  |0.0200|

|             Tasks             |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|-------------------------------|------:|------|-----:|--------|---|-----:|---|-----:|
|haerae                         |      1|none  |      |acc     |↑  |0.4180|±  |0.0148|
|                               |       |none  |      |acc_norm|↑  |0.4180|±  |0.0148|
| - haerae_general_knowledge    |      1|none  |     5|acc     |↑  |0.3125|±  |0.0350|
|                               |       |none  |     5|acc_norm|↑  |0.3125|±  |0.0350|
| - haerae_history              |      1|none  |     5|acc     |↑  |0.3404|±  |0.0347|
|                               |       |none  |     5|acc_norm|↑  |0.3404|±  |0.0347|
| - haerae_loan_word            |      1|none  |     5|acc     |↑  |0.4083|±  |0.0379|
|                               |       |none  |     5|acc_norm|↑  |0.4083|±  |0.0379|
| - haerae_rare_word            |      1|none  |     5|acc     |↑  |0.4815|±  |0.0249|
|                               |       |none  |     5|acc_norm|↑  |0.4815|±  |0.0249|
| - haerae_standard_nomenclature|      1|none  |     5|acc     |↑  |0.4771|±  |0.0405|
|                               |       |none  |     5|acc_norm|↑  |0.4771|±  |0.0405|

|     Tasks      |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|----------------|------:|------|-----:|--------|---|-----:|---|------|
|kobest_boolq    |      1|none  |     5|acc     |↑  |0.7664|±  |0.0113|
|                |       |none  |     5|f1      |↑  |0.7662|±  |   N/A|
|kobest_copa     |      1|none  |     5|acc     |↑  |0.5620|±  |0.0157|
|                |       |none  |     5|f1      |↑  |0.5612|±  |   N/A|
|kobest_hellaswag|      1|none  |     5|acc     |↑  |0.3840|±  |0.0218|
|                |       |none  |     5|acc_norm|↑  |0.4900|±  |0.0224|
|                |       |none  |     5|f1      |↑  |0.3807|±  |   N/A|
|kobest_sentineg |      1|none  |     5|acc     |↑  |0.5869|±  |0.0247|
|                |       |none  |     5|f1      |↑  |0.5545|±  |   N/A|
|kobest_wic      |      1|none  |     5|acc     |↑  |0.4952|±  |0.0141|
|                |       |none  |     5|f1      |↑  |0.4000|±  |   N/A|


```
@article{Llama3.2RabbitKo3BInstruct,
  title={CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct Card},
  author={CarrotAI (L, GEUN)},
  year={2024},
  url = {https://huggingface.co./CarrotAI/Llama-3.2-Rabbit-Ko-3B-Instruct}
}
```