Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: yanolja/EEVE-Korean-Instruct-10.8B-v1.0
|
| 3 |
+
license: apache-2.0
|
| 4 |
+
language:
|
| 5 |
+
- ko
|
| 6 |
+
- en
|
| 7 |
+
---
|
| 8 |
+
|
| 9 |
+
# **Jolteon-Instruct-13B-alpha**
|
| 10 |
+
|
| 11 |
+
The model was trained based on the [EEVE-Korean-Instruct-10.8B-v1.0](https://huggingface.co/yanolja/EEVE-Korean-Instruct-10.8B-v1.0) model from [yanolja](https://www.yanolja.com), extended to 13.4b (12 layer pass-through) utilizing [mergekit](https://github.com/cg123/mergekit).
|
| 12 |
+
|
| 13 |
+
## Methodology
|
| 14 |
+
|
| 15 |
+
TBD
|
| 16 |
+
|
| 17 |
+
## Training Details
|
| 18 |
+
| |Training Data|Parameters|Content Length|Samples Seen|Learning Rate|
|
| 19 |
+
|---|---|---|---|---|---|
|
| 20 |
+
|Jolteon-Instruct-13B-alpha|*A curated mix of English + Korean Instruction set*|13.4B|4k|>400k|1e<sup>-5</sup>|
|
| 21 |
+
|
| 22 |
+
## Example
|
| 23 |
+
|
| 24 |
+
## License
|
| 25 |
+
|
| 26 |
+
본 모델은 apache-2.0 라이센스를 따릅니다. 모델을 사용하여 생성된 데이터셋을 배포할 경우 모델 사용을 명시해 주시기를 권고드립니다.
|
| 27 |
+
|
| 28 |
+
## Thanks to
|
| 29 |
+
|
| 30 |
+
- A100 클러스터를 제공해주신, [Sionic AI](https://sionic.ai/)
|
| 31 |
+
|
| 32 |
+
## Contact
|
| 33 |
+
- [Discord Server Link](https://discord.gg/MrBt3PXdXc)
|