Taishi-N324
commited on
Commit
•
f053604
1
Parent(s):
b6e75aa
Upload README.md
Browse files
README.md
CHANGED
@@ -153,12 +153,21 @@ This format must be adhered to strictly, as deviations may result in less optima
|
|
153 |
The template used to construct a prompt for the Instruct model is specified as follows:
|
154 |
|
155 |
```
|
156 |
-
<s>[INST] <<SYS>>\n{
|
157 |
```
|
158 |
|
|
|
159 |
Please be aware that ``<s>`` and ``</s>`` are special tokens used for the beginning of string (BOS) and end of string (EOS), respectively, while [INST] and [/INST] are considered regular strings.
|
160 |
|
161 |
-
For the "{
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
162 |
|
163 |
|
164 |
### Use the instruct model Ver0.1
|
@@ -228,3 +237,15 @@ Here are the team members:
|
|
228 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
229 |
- [Takumi Okamoto](https://www.linkedin.com/in/takumi-okamoto)
|
230 |
- [Ishida Shigeki](https://www.wantedly.com/id/reborn27)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
153 |
The template used to construct a prompt for the Instruct model is specified as follows:
|
154 |
|
155 |
```
|
156 |
+
<s>[INST] <<SYS>>\n{SYSTEM_PROMPT}\n<</SYS>>\n\n{USER_MESSAGE_1} [/INST] {BOT_MESSAGE_1} </s>[INST] {USER_MESSAGE_2}[/INST]
|
157 |
```
|
158 |
|
159 |
+
|
160 |
Please be aware that ``<s>`` and ``</s>`` are special tokens used for the beginning of string (BOS) and end of string (EOS), respectively, while [INST] and [/INST] are considered regular strings.
|
161 |
|
162 |
+
For the "{SYSTEM_PROMPT}" part, We recommend using "あなたは誠実で優秀な日本人のアシスタントです。"
|
163 |
+
|
164 |
+
For the "{USER_MESSAGE_1}" part, We recommend using {instruction}\n{input}
|
165 |
+
|
166 |
+
In other words, We recommend the following:
|
167 |
+
|
168 |
+
```
|
169 |
+
<s>[INST] <<SYS>>\nあなたは誠実で優秀な日本人のアシスタントです。\n<</SYS>>\n\n{instruction1}\n{input1} [/INST] {BOT_MESSAGE_1}</s>[INST] \n\n{instruction2}\n{input2} [/INST]
|
170 |
+
```
|
171 |
|
172 |
|
173 |
### Use the instruct model Ver0.1
|
|
|
237 |
- [Taishi Nakamura](https://twitter.com/Setuna7777_2)
|
238 |
- [Takumi Okamoto](https://www.linkedin.com/in/takumi-okamoto)
|
239 |
- [Ishida Shigeki](https://www.wantedly.com/id/reborn27)
|
240 |
+
|
241 |
+
## How to cite
|
242 |
+
```
|
243 |
+
@misc{fujii2024continual,
|
244 |
+
title={Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities},
|
245 |
+
author={Kazuki Fujii and Taishi Nakamura and Mengsay Loem and Hiroki Iida and Masanari Ohi and Kakeru Hattori and Hirai Shota and Sakae Mizuki and Rio Yokota and Naoaki Okazaki},
|
246 |
+
year={2024},
|
247 |
+
eprint={2404.17790},
|
248 |
+
archivePrefix={arXiv},
|
249 |
+
primaryClass={cs.CL}
|
250 |
+
}
|
251 |
+
```
|