vaishaal commited on
Commit
6ad4265
1 Parent(s): e9e0a6b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -0
README.md CHANGED
@@ -17,6 +17,14 @@ DCLM-Baseline-7B is a 7 billion parameter language model trained on the DCLM-Bas
17
  |------|-----------------|--------|-------------|-----------------|----------------|
18
  | 7B | 2.5T | 32 | 4096 | 32 | 2048 |
19
 
 
 
 
 
 
 
 
 
20
  ### Model Description
21
 
22
  - **Developed by:** DataComp for Language Models (DCLM) Team
 
17
  |------|-----------------|--------|-------------|-----------------|----------------|
18
  | 7B | 2.5T | 32 | 4096 | 32 | 2048 |
19
 
20
+ | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
21
+ |------|--------|---------|-------------|-----------------|----------------|
22
+ | [OLMo 1B](https://huggingface.co/allenai/OLMo-1B) | 3 Trillion |16 | 2048 | 16 | 2048 |
23
+ | [OLMo 7B](https://huggingface.co/allenai/OLMo-7B) | 2.5 Trillion | 32 | 4096 | 32 | 2048 |
24
+ | [OLMo 7B Twin 2T](https://huggingface.co/allenai/OLMo-7B-Twin-2T) | 2 Trillion | 32 | 4096 | 32 | 2048 |
25
+
26
+
27
+
28
  ### Model Description
29
 
30
  - **Developed by:** DataComp for Language Models (DCLM) Team