contrain instruct datasets

TRAIN.md

@@ -42,6 +42,20 @@ litgpt convert_pretrained_checkpoint ../out/pretrain-4/final/ ../out/pretrain-4-
 #   litgpt convert_pretrained_checkpoint ../out/pretrain-5/final/ ../out/pretrain-5-final-checkpoint
 ```
 
+### Continued Pretraining
+
+```bash
+python -B prepare_contrain_datasets.py
+```
+
+```bash
+CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True litgpt pretrain --config contrain-model-0.yaml
+litgpt convert_pretrained_checkpoint ../out/contrain-0/final/ ../out/contrain-0-final-checkpoint
+
+CUDA_VISIBLE_DEVICES=0 CUDA_LAUNCH_BLOCKING=0 PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True litgpt pretrain --config contrain-model-1.yaml
+litgpt convert_pretrained_checkpoint ../out/contrain-1/final/ ../out/contrain-1-final-checkpoint
+```
+
 ## Chat with Pretrained model
 
 ```bash