Update README.md

README.md

@@ -6,9 +6,12 @@ library_name: transformers
 pipeline_tag: text-generation
 license: llama3.1
 model_type: llama
+datasets:
+  - lmsys/lmsys-chat-1m
+  - argilla/magpie-ultra-v0.1
 ---
 
-# Llama3.1 Swallow
+# Llama 3.1 Swallow
 
 Llama 3.1 Swallow is a series of large language models (8B, 70B) that were built by continual pre-training on the [Meta Llama 3.1](https://huggingface.co/collections/meta-llama/llama-31-669fc079a0c406a149a5738f) models.
 Llama 3.1 Swallow enhanced the Japanese language capabilities of the original Llama 3.1 while retaining the English language capabilities.
@@ -197,20 +200,23 @@ print(output[0].outputs[0].text)
 
 ### Instruction Tuning
 
-The following datasets were used for the instruction tuning. 
+The following instruction datasets were used for the instruction tuning. 
+- Japanese
+  - `lmsys-chat-1m-synth-ja-wo-pii-and-template-instructions`
+    - Single-turn Japanese synthetic instruction dataset derived from [lmsys-chat-1m](https://huggingface.co/datasets/lmsys/lmsys-chat-1m) dataset [\[Zhang+, ICLR24\]](https://openreview.net/forum?id=BOfDKxfwt0)).
+      The first-turn user instructions were translated into Japanese via DeepL machine translation, and the assistant responses were generated using the Llama 3.1 405B Instruct model. Rejection sampling (n=6) was applied, with Llama 3.1 70B Instruct serving as a judge.
+    - As implied by the dataset name, conversations that contain personally identifiable information (PII) or template-based user instructions have been removed. Duplicate instuctions have also been removed.
+  - `filtered-magpie-ultra-ja`
+    - A Japanese variant of the `filtered-magpie-ultra-en` dataset, machine-translated into Japanese using the Gemma 2 27B IT.
+  - `gemma-magpie`
+    - Japanese Q&A dataset on diverse topics, generated using prompts with specific category words, with answers by Gemma 2 27B IT, heuristically filtered for quality and length.
+- English
+  - `lmsys-chat-1m-synth-en-wo-pii-and-template-instructions`
+    - Similar to the `lmsys-chat-1m-synth-ja-wo-pii-and-template-instructions`, but this version uses the original English user instructions. The assistant responses were generated in English as well. Rejection sampling was not applied in this version.
+  - `filtered-magpie-ultra-en`
+    - A subset of the [magpie-ultra](https://huggingface.co/datasets/argilla/magpie-ultra-v0.1) dataset, developed following the MAGPIE recipe [\[Xu+, arXiv24\]](https://arxiv.org/abs/2406.08464) using Llama 3.1 405B Instruct. This subset includes only samples rated as 'average,' 'good,' or 'excellent.'
 
-- lmsys-chat-1m-synth-ja-wo-pii
-    
-    - Japanese translation of the lmsys-chat-1m dataset using DeepL, with synthetic instruction data created using the Llama-3.1-405B model.
-    - 'wo-pii' indicates removal of personally identifiable information.
 
-- filtered magpie-ultra
-    - Subset of the [magpie-ultra](https://huggingface.co/datasets/argilla/magpie-ultra-v0.1) dataset, containing samples rated as 'average,' 'good,' or 'excellent.'.
-
-- gemma-magpie
-    - Japanese dataset.
-    - Generated using prompts for specific category words.
- 
 ## Risks and Limitations
 
 The models released here are still in the early stages of our research and development and have not been tuned to ensure outputs align with human intent and safety considerations.