Update README.md
Browse files
README.md
CHANGED
@@ -19,7 +19,7 @@ language:
|
|
19 |
|
20 |
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
|
21 |
The OLMo base models are trained on the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset.
|
22 |
-
The adapted versions are trained on the [Tulu SFT mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) and, for the Instruct version, a [cleaned version of the UltraFeedback dataset](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned).
|
23 |
|
24 |
OLMo 7B Instruct SFT are two adapted versions of these models trained for better question answering.
|
25 |
These are updated OLMo models corresponding to our July 2024 release.
|
@@ -30,8 +30,8 @@ They show the performance gain that OLMo base models can achieve with existing f
|
|
30 |
We release two adapted model versions:
|
31 |
| Model | Training Method(s) | Datasets | Context Length |
|
32 |
|------|--------|---------|--|
|
33 |
-
| [OLMo 7B July 2024 SFT](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-SFT-hf) | SFT | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) |
|
34 |
-
| [OLMo 7B July 2024 Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-Instruct-hf) | SFT + DPO | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) + [Ultrafeedback Cleaned](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned) |
|
35 |
|
36 |
These models are both trained on top of OLMo 7b July 2024:
|
37 |
| Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
|
@@ -104,7 +104,7 @@ Core model results for the 7B adapted models are found below.
|
|
104 |
## Model Details
|
105 |
|
106 |
### Data
|
107 |
-
For training data details, please see the [Dolma](https://huggingface.co/datasets/allenai/dolma), [Tulu 2](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture), and [UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) documentation.
|
108 |
|
109 |
### Architecture
|
110 |
|
|
|
19 |
|
20 |
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models.
|
21 |
The OLMo base models are trained on the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset.
|
22 |
+
The adapted versions are trained on the [Tulu SFT mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture-olmo-4096) and, for the Instruct version, a [cleaned version of the UltraFeedback dataset](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned).
|
23 |
|
24 |
OLMo 7B Instruct SFT are two adapted versions of these models trained for better question answering.
|
25 |
These are updated OLMo models corresponding to our July 2024 release.
|
|
|
30 |
We release two adapted model versions:
|
31 |
| Model | Training Method(s) | Datasets | Context Length |
|
32 |
|------|--------|---------|--|
|
33 |
+
| [OLMo 7B July 2024 SFT](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-SFT-hf) | SFT | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/allenai/tulu-v2-sft-mixture-olmo-4096) | 4096 |
|
34 |
+
| [OLMo 7B July 2024 Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Nitro-Instruct-hf) | SFT + DPO | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture-olmo-4096) + [Ultrafeedback Cleaned](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned) | 4096 |
|
35 |
|
36 |
These models are both trained on top of OLMo 7b July 2024:
|
37 |
| Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
|
|
|
104 |
## Model Details
|
105 |
|
106 |
### Data
|
107 |
+
For training data details, please see the [Dolma](https://huggingface.co/datasets/allenai/dolma), [Tulu 2](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture-olmo-4096), and [UltraFeedback](https://huggingface.co/datasets/openbmb/UltraFeedback) documentation.
|
108 |
|
109 |
### Architecture
|
110 |
|