allenai
/

Llama-3.1-Tulu-3-405B-DPO

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

vwxyzjn commited on Jan 29

Commit

b7bfe11

·

verified ·

1 Parent(s): e146e04

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -151,7 +151,7 @@ See the Falcon 180B model card for an example of this.
 ## Hyperparamters
 DPO:
-- **Learning Rate**: 5 × 10⁻⁷ (8B), 2.0e-7 (70B), 2.0e-7 (405B)
 - **Learning Rate Schedule**: Linear
 - **Batch Size (effective)**: 32 (8B), 128 (70B), 256(405B)
 - **KL Penalty Coefficient**: 5

 ## Hyperparamters
 DPO:
+- **Learning Rate**: 5 × 10⁻⁷ (8B), 2.0e-7 (70B, 405B)
 - **Learning Rate Schedule**: Linear
 - **Batch Size (effective)**: 32 (8B), 128 (70B), 256(405B)
 - **KL Penalty Coefficient**: 5