Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,56 @@
|
|
1 |
---
|
2 |
license: cc-by-nc-4.0
|
3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
license: cc-by-nc-4.0
|
3 |
+
datasets:
|
4 |
+
- kyujinpy/KOR-OpenOrca-Platypus-v3
|
5 |
+
- beomi/KoAlpaca-v1.1a
|
6 |
+
- maywell/ko_wikidata_QA
|
7 |
+
language:
|
8 |
+
- ko
|
9 |
+
base_model: beomi/Yi-Ko-6B
|
10 |
+
---
|
11 |
+
|
12 |
+
# Yi-Ko-6B-Instruct-v1.0
|
13 |
+
|
14 |
+
## Model Details
|
15 |
+
|
16 |
+
### Base Model
|
17 |
+
[beomi/Yi-Ko-6B](https://huggingface.co/beomi/Yi-Ko-6B)
|
18 |
+
|
19 |
+
### Training Dataset
|
20 |
+
1. [kyujinpy/KOR-OpenOrca-Platypus-v3](https://huggingface.co/datasets/kyujinpy/KOR-OpenOrca-Platypus-v3) π
|
21 |
+
2. [beomi/KoAlpaca-v1.1a](https://huggingface.co/datasets/beomi/KoAlpaca-v1.1a) π
|
22 |
+
3. [maywell/ko_wikidata_QA](https://huggingface.co/datasets/maywell/ko_wikidata_QA) π
|
23 |
+
4. AIHub MRC λ°μ΄ν° μ λ³ ν Instruction Format λ§κ² λ³κ²½ ν μ¬μ©
|
24 |
+
|
25 |
+
## Benchmark Results
|
26 |
+
|
27 |
+
### AI-Harness Evaluation
|
28 |
+
https://github.com/Beomi/ko-lm-evaluation-harness
|
29 |
+
|
30 |
+
| Model | BoolQ | Copa | HellaSwag | Sentineg |
|
31 |
+
| --- | --- | --- | --- | --- |
|
32 |
+
| | *Zero-shot* ||||
|
33 |
+
| Yi-Ko-6B-Instruct-v1.0 | 0.6619 | 0.7794 | 0.4858 | 0.4589 |
|
34 |
+
| Yi-Ko-6B | 0.7070 | 0.7696 | 0.5009 | 0.4044 |
|
35 |
+
|
36 |
+
## Instruction Format
|
37 |
+
```python
|
38 |
+
### User:
|
39 |
+
{instruction}
|
40 |
+
|
41 |
+
### Assistant:
|
42 |
+
{response}
|
43 |
+
```
|
44 |
+
|
45 |
+
## Loading the Model
|
46 |
+
```python
|
47 |
+
import torch
|
48 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
49 |
+
|
50 |
+
tokenizer = AutoTokenizer.from_pretrained("wkshin89/Yi-Ko-6B-Instruct-v1.0")
|
51 |
+
model = AutoModelForCausalLM.from_pretrained(
|
52 |
+
"wkshin89/Yi-Ko-6B-Instruct-v1.0",
|
53 |
+
device_map="auto",
|
54 |
+
torch_dtype=torch.bfloat16,
|
55 |
+
)
|
56 |
+
```
|