slurm submission log: 2024-05-13 22:25:16.975668
created following sbatch script: 

###############################

#!/bin/bash

#SBATCH --account=nlp
#SBATCH --cpus-per-task=16
#SBATCH --dependency=afterok:7608406
#SBATCH --gres=gpu:2
#SBATCH --job-name=tthrush-job-4542903
#SBATCH --mem=400G
#SBATCH --nodelist=sphinx2
#SBATCH --open-mode=append
#SBATCH --output=/juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/llms_basic/pythia-70m_lambada/train_job_output.txt
#SBATCH --partition=sphinx
#SBATCH --time=14-0

# activate your desired anaconda environment
. /nlp/scr/tthrush/miniconda3/etc/profile.d/conda.sh ; conda activate pretraining-coreset-selection

# cd to working directory
cd .

# launch commands
srun --unbuffered run_as_child_processes 'torchrun --master_port 29506 --nproc_per_node=2 train_llm.py --dataset_id /juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/train_data_basic/lambada --output_dir /juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/llms_basic/pythia-70m_lambada --output_hub_id pythia-70m_lambada --model_id EleutherAI/pythia-70m --num_train_epochs 14 --learning_rate 1e-3 --warmup_ratio=0.1 --gradient_accumulation_steps 2'

###############################

submission to slurm complete!


###############################
slurm submission output

Submitted batch job 7608407



###############################

###############################
start time: 2024-05-14 03:07:22.154087
machine: sphinx2
conda env: pretraining-coreset-selection
###############################
running following processes

	torchrun --master_port 29506 --nproc_per_node=2 train_llm.py --dataset_id /juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/train_data_basic/lambada --output_dir /juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/llms_basic/pythia-70m_lambada --output_hub_id pythia-70m_lambada --model_id EleutherAI/pythia-70m --num_train_epochs 14 --learning_rate 1e-3 --warmup_ratio=0.1 --gradient_accumulation_steps 2


###############################
command outputs: 


[2024-05-14 03:07:29,791] torch.distributed.run: [WARNING] 
[2024-05-14 03:07:29,791] torch.distributed.run: [WARNING] *****************************************
[2024-05-14 03:07:29,791] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
[2024-05-14 03:07:29,791] torch.distributed.run: [WARNING] *****************************************
05/14/2024 03:07:53 - INFO - __main__ - Script parameters ScriptArguments(dataset_id='/juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/train_data_basic/lambada', output_dir='/juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/llms_basic/pythia-70m_lambada', output_hub_id='pythia-70m_lambada', hf_hub_token=True, model_id='EleutherAI/pythia-70m', per_device_train_batch_size=256, num_train_epochs=14, learning_rate=0.001, gradient_accumulation_steps=2, from_scratch=True, warmup_ratio=0.1, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, weight_decay=0.01, lr_scheduler_type='cosine', local_rank=0, resume_from_checkpoint=False, deepspeed=None, peft=False)
05/14/2024 03:07:53 - INFO - __main__ - Script parameters ScriptArguments(dataset_id='/juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/train_data_basic/lambada', output_dir='/juice5/scr5/tthrush/pretraining-coreset-selection/llm_pretraining/llms_basic/pythia-70m_lambada', output_hub_id='pythia-70m_lambada', hf_hub_token=True, model_id='EleutherAI/pythia-70m', per_device_train_batch_size=256, num_train_epochs=14, learning_rate=0.001, gradient_accumulation_steps=2, from_scratch=True, warmup_ratio=0.1, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon=1e-08, weight_decay=0.01, lr_scheduler_type='cosine', local_rank=0, resume_from_checkpoint=False, deepspeed=None, peft=False)

  0%|          | 0/10682 [00:00<?, ?it/s][rank1]:[W reducer.cpp:1360] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())
[rank0]:[W reducer.cpp:1360] Warning: find_unused_parameters=True was specified in DDP constructor, but did not find any unused parameters in the forward pass. This flag results in an extra traversal of the autograd graph every iteration,  which can adversely affect performance. If your model indeed never has any unused parameters in the forward pass, consider turning this flag off. Note that this warning may be a false positive if your model has flow control causing later iterations to have unused parameters. (function operator())

  0%|          | 1/10682 [00:51<151:39:39, 51.12s/it]
  0%|          | 2/10682 [01:08<92:56:51, 31.33s/it] 
  0%|          | 3/10682 [01:20<67:06:16, 22.62s/it]
  0%|          | 4/10682 [01:30<51:33:18, 17.38s/it]
  0%|          | 5/10682 [01:36<39:52:10, 13.44s/it]
  0%|          | 6/10682 [01:41<31:13:33, 10.53s/it]
  0%|          | 7/10682 [01:44<23:50:36,  8.04s/it]
  0%|          | 8/10682 [01:46<18:20:47,  6.19s/it]
  0%|          | 9/10682 [01:48<14:12:01,  4.79s/it]
  0%|          | 10/10682 [01:50<11:25:53,  3.86s/it]
  0%|          | 11/10682 [01:51<8:55:53,  3.01s/it] 
  0%|          | 12/10682 [01:52<6:56:15,  2.34s/it]
  0%|          | 13/10682 [01:52<5:31:14,  1.86s/it]
  0%|          | 14/10682 [01:53<4:34:44,  1.55s/it]
  0%|          | 15/10682 [01:54<4:10:12,  1.41s/it]
  0%|          | 16/10682 [01:55<3:31:03,  1.19s/it]
  0%|          | 17/10682 [01:56<3:04:54,  1.04s/it]
  0%|          | 18/10682 [01:56<2:43:03,  1.09it/s]
  0%|          | 19/10682 [01:57<2:24:44,  1.23it/s]
  0%|          | 20/10682 [01:57<2:14:08,  1.32it/s]
  0%|          | 21/10682 [01:58<2:04:15,  1.43it/s]
  0%|          | 22/10682 [01:59<1:55:36,  1.54it/s]
  0%|          | 23/10682 [01:59<1:50:05,  1.61it/s]
  0%|          | 24/10682 [02:00<1:49:22,  1.62it/s]
  0%|          | 25/10682 [02:00<1:47:19,  1.65it/s]
                                                    
{'loss': 10.6608, 'grad_norm': 1.3869956731796265, 'learning_rate': 2.3386342376052384e-05, 'epoch': 0.03}

  0%|          | 25/10682 [02:00<1:47:19,  1.65it/s]
  0%|          | 26/10682 [02:01<1:44:27,  1.70it/s]
  0%|          | 27/10682 [02:01<1:40:32,  1.77it/s]
  0%|          | 28/10682 [02:02<1:38:27,  1.80it/s]
  0%|          | 29/10682 [02:02<1:36:12,  1.85it/s]
  0%|          | 30/10682 [02:03<1:34:44,  1.87it/s]
  0%|          | 31/10682 [02:04<1:51:21,  1.59it/s]
  0%|          | 32/10682 [02:04<1:45:09,  1.69it/s]
  0%|          | 33/10682 [02:05<1:40:48,  1.76it/s]
  0%|          | 34/10682 [02:05<1:37:53,  1.81it/s]
  0%|          | 35/10682 [02:06<1:38:58,  1.79it/s]
  0%|          | 36/10682 [02:06<1:36:00,  1.85it/s]
  0%|          | 37/10682 [02:07<1:34:03,  1.89it/s]
  0%|          | 38/10682 [02:07<1:32:41,  1.91it/s]
  0%|          | 39/10682 [02:08<1:32:27,  1.92it/s]
  0%|          | 40/10682 [02:08<1:31:17,  1.94it/s]
  0%|          | 41/10682 [02:09<1:30:36,  1.96it/s]
  0%|          | 42/10682 [02:09<1:30:20,  1.96it/s]
  0%|          | 43/10682 [02:10<1:29:50,  1.97it/s]
  0%|          | 44/10682 [02:10<1:29:37,  1.98it/s]
  0%|          | 45/10682 [02:11<1:29:27,  1.98it/s]
  0%|          | 46/10682 [02:11<1:29:10,  1.99it/s]
  0%|          | 47/10682 [02:12<1:29:04,  1.99it/s]
  0%|          | 48/10682 [02:12<1:28:49,  2.00it/s]
  0%|          | 49/10682 [02:13<1:28:39,  2.00it/s]
  0%|          | 50/10682 [02:13<1:28:40,  2.00it/s]{'loss': 9.9515, 'grad_norm': 1.2442067861557007, 'learning_rate': 4.677268475210477e-05, 'epoch': 0.07}

                                                    

  0%|          | 50/10682 [02:13<1:28:40,  2.00it/s]
  0%|          | 51/10682 [02:14<1:28:31,  2.00it/s]
  0%|          | 52/10682 [02:14<1:28:26,  2.00it/s]
  0%|          | 53/10682 [02:15<1:28:25,  2.00it/s]
  1%|          | 54/10682 [02:15<1:28:16,  2.01it/s]
  1%|          | 55/10682 [02:16<1:28:16,  2.01it/s]
  1%|          | 56/10682 [02:16<1:28:17,  2.01it/s]
  1%|          | 57/10682 [02:17<1:28:19,  2.00it/s]
  1%|          | 58/10682 [02:17<1:28:17,  2.01it/s]
  1%|          | 59/10682 [02:18<1:29:31,  1.98it/s]
  1%|          | 60/10682 [02:18<1:29:18,  1.98it/s]
  1%|          | 61/10682 [02:19<1:34:28,  1.87it/s]
  1%|          | 62/10682 [02:19<1:32:38,  1.91it/s]
  1%|          | 63/10682 [02:20<1:32:05,  1.92it/s]
  1%|          | 64/10682 [02:20<1:31:06,  1.94it/s]
  1%|          | 65/10682 [02:21<1:30:12,  1.96it/s]
  1%|          | 66/10682 [02:21<1:29:43,  1.97it/s]
  1%|          | 67/10682 [02:22<1:29:59,  1.97it/s]
  1%|          | 68/10682 [02:23<1:29:59,  1.97it/s]
  1%|          | 69/10682 [02:23<1:30:49,  1.95it/s]
  1%|          | 70/10682 [02:24<1:30:12,  1.96it/s]
  1%|          | 71/10682 [02:24<1:29:53,  1.97it/s]
  1%|          | 72/10682 [02:25<1:29:22,  1.98it/s]
  1%|          | 73/10682 [02:25<1:29:32,  1.97it/s]
  1%|          | 74/10682 [02:26<1:29:10,  1.98it/s]
  1%|          | 75/10682 [02:26<1:28:43,  1.99it/s]
                                                    
{'loss': 9.2331, 'grad_norm': 1.1144205331802368, 'learning_rate': 7.015902712815715e-05, 'epoch': 0.1}

  1%|          | 75/10682 [02:26<1:28:43,  1.99it/s]
  1%|          | 76/10682 [02:27<1:29:02,  1.99it/s]
  1%|          | 77/10682 [02:27<1:29:17,  1.98it/s]
  1%|          | 78/10682 [02:28<1:28:55,  1.99it/s]
  1%|          | 79/10682 [02:28<1:28:39,  1.99it/s]
  1%|          | 80/10682 [02:29<1:28:34,  1.99it/s]
  1%|          | 81/10682 [02:29<1:28:38,  1.99it/s]
  1%|          | 82/10682 [02:30<1:28:27,  2.00it/s]
  1%|          | 83/10682 [02:30<1:28:18,  2.00it/s]
  1%|          | 84/10682 [02:31<1:28:22,  2.00it/s]
  1%|          | 85/10682 [02:31<1:28:38,  1.99it/s]
  1%|          | 86/10682 [02:32<1:29:19,  1.98it/s]
  1%|          | 87/10682 [02:32<1:29:15,  1.98it/s]
  1%|          | 88/10682 [02:33<1:28:40,  1.99it/s]
  1%|          | 89/10682 [02:33<1:28:52,  1.99it/s]
  1%|          | 90/10682 [02:34<1:28:37,  1.99it/s]
  1%|          | 91/10682 [02:34<1:28:11,  2.00it/s]
  1%|          | 92/10682 [02:35<1:28:07,  2.00it/s]
  1%|          | 93/10682 [02:35<1:28:07,  2.00it/s]
  1%|          | 94/10682 [02:36<1:27:53,  2.01it/s]
  1%|          | 95/10682 [02:36<1:27:53,  2.01it/s]
  1%|          | 96/10682 [02:37<1:28:40,  1.99it/s]
  1%|          | 97/10682 [02:37<1:28:30,  1.99it/s]
  1%|          | 98/10682 [02:38<1:28:40,  1.99it/s]
  1%|          | 99/10682 [02:38<1:28:38,  1.99it/s]
  1%|          | 100/10682 [02:39<1:29:25,  1.97it/s]
                                                     
{'loss': 8.4427, 'grad_norm': 0.8195424675941467, 'learning_rate': 9.354536950420954e-05, 'epoch': 0.13}

  1%|          | 100/10682 [02:39<1:29:25,  1.97it/s]
  1%|          | 101/10682 [02:39<1:29:18,  1.97it/s]
  1%|          | 102/10682 [02:40<1:29:15,  1.98it/s]
  1%|          | 103/10682 [02:40<1:28:38,  1.99it/s]
  1%|          | 104/10682 [02:41<1:28:17,  2.00it/s]
  1%|          | 105/10682 [02:41<1:27:57,  2.00it/s]
  1%|          | 106/10682 [02:42<1:27:48,  2.01it/s]
  1%|          | 107/10682 [02:42<1:27:44,  2.01it/s]
  1%|          | 108/10682 [02:43<1:27:41,  2.01it/s]
  1%|          | 109/10682 [02:43<1:27:32,  2.01it/s]
  1%|          | 110/10682 [02:44<1:27:33,  2.01it/s]
  1%|          | 111/10682 [02:44<1:27:26,  2.01it/s]
  1%|          | 112/10682 [02:45<1:27:30,  2.01it/s]
  1%|          | 113/10682 [02:45<1:27:30,  2.01it/s]
  1%|          | 114/10682 [02:46<1:27:31,  2.01it/s]
  1%|          | 115/10682 [02:46<1:27:35,  2.01it/s]
  1%|          | 116/10682 [02:47<1:27:27,  2.01it/s]
  1%|          | 117/10682 [02:47<1:27:30,  2.01it/s]
  1%|          | 118/10682 [02:48<1:27:23,  2.01it/s]
  1%|          | 119/10682 [02:48<1:27:27,  2.01it/s]
  1%|          | 120/10682 [02:49<1:27:26,  2.01it/s]
  1%|          | 121/10682 [02:49<1:27:21,  2.01it/s]
  1%|          | 122/10682 [02:50<1:27:33,  2.01it/s]
  1%|          | 123/10682 [02:50<1:27:25,  2.01it/s]
  1%|          | 124/10682 [02:51<1:27:22,  2.01it/s]
  1%|          | 125/10682 [02:51<1:27:21,  2.01it/s]
                                                     {'loss': 7.7906, 'grad_norm': 0.4837629795074463, 'learning_rate': 0.00011693171188026193, 'epoch': 0.16}


  1%|          | 125/10682 [02:51<1:27:21,  2.01it/s]
  1%|          | 126/10682 [02:52<1:27:23,  2.01it/s]
  1%|          | 127/10682 [02:52<1:27:20,  2.01it/s]
  1%|          | 128/10682 [02:53<1:27:19,  2.01it/s]
  1%|          | 129/10682 [02:53<1:27:17,  2.01it/s]
  1%|          | 130/10682 [02:54<1:27:17,  2.01it/s]
  1%|          | 131/10682 [02:54<1:28:17,  1.99it/s]
  1%|          | 132/10682 [02:55<1:27:58,  2.00it/s]
  1%|          | 133/10682 [02:55<1:27:43,  2.00it/s]
  1%|▏         | 134/10682 [02:56<1:27:35,  2.01it/s]
  1%|▏         | 135/10682 [02:56<1:27:25,  2.01it/s]
  1%|▏         | 136/10682 [02:57<1:27:29,  2.01it/s]
  1%|▏         | 137/10682 [02:57<1:27:35,  2.01it/s]
  1%|▏         | 138/10682 [02:58<1:27:25,  2.01it/s]
  1%|▏         | 139/10682 [02:58<1:27:23,  2.01it/s]
  1%|▏         | 140/10682 [02:59<1:27:12,  2.01it/s]
  1%|▏         | 141/10682 [02:59<1:27:10,  2.02it/s]
  1%|▏         | 142/10682 [02:59<1:27:05,  2.02it/s]
  1%|▏         | 143/10682 [03:00<1:27:05,  2.02it/s]
  1%|▏         | 144/10682 [03:00<1:26:58,  2.02it/s]
  1%|▏         | 145/10682 [03:01<1:26:55,  2.02it/s]
  1%|▏         | 146/10682 [03:01<1:26:53,  2.02it/s]
  1%|▏         | 147/10682 [03:02<1:26:52,  2.02it/s]
  1%|▏         | 148/10682 [03:02<1:27:04,  2.02it/s]
  1%|▏         | 149/10682 [03:03<1:26:59,  2.02it/s]
  1%|▏         | 150/10682 [03:03<1:27:03,  2.02it/s]
                                                     
{'loss': 7.3088, 'grad_norm': 0.5318506956100464, 'learning_rate': 0.0001403180542563143, 'epoch': 0.2}

  1%|▏         | 150/10682 [03:03<1:27:03,  2.02it/s]
  1%|▏         | 151/10682 [03:04<1:27:05,  2.02it/s]
  1%|▏         | 152/10682 [03:04<1:27:03,  2.02it/s]
  1%|▏         | 153/10682 [03:05<1:26:58,  2.02it/s]
  1%|▏         | 154/10682 [03:05<1:26:57,  2.02it/s]
  1%|▏         | 155/10682 [03:06<1:26:57,  2.02it/s]
  1%|▏         | 156/10682 [03:06<1:26:55,  2.02it/s]
  1%|▏         | 157/10682 [03:07<1:26:51,  2.02it/s]
  1%|▏         | 158/10682 [03:07<1:26:51,  2.02it/s]
  1%|▏         | 159/10682 [03:08<1:26:48,  2.02it/s]
  1%|▏         | 160/10682 [03:08<1:26:47,  2.02it/s]
  2%|▏         | 161/10682 [03:09<1:26:44,  2.02it/s]
  2%|▏         | 162/10682 [03:09<1:26:45,  2.02it/s]
  2%|▏         | 163/10682 [03:10<1:26:42,  2.02it/s]
  2%|▏         | 164/10682 [03:10<1:26:40,  2.02it/s]
  2%|▏         | 165/10682 [03:11<1:26:42,  2.02it/s]
  2%|▏         | 166/10682 [03:11<1:26:41,  2.02it/s]
  2%|▏         | 167/10682 [03:12<1:26:41,  2.02it/s]
  2%|▏         | 168/10682 [03:12<1:26:43,  2.02it/s]
  2%|▏         | 169/10682 [03:13<1:26:46,  2.02it/s]
  2%|▏         | 170/10682 [03:13<1:26:48,  2.02it/s]
  2%|▏         | 171/10682 [03:14<1:26:56,  2.01it/s]
  2%|▏         | 172/10682 [03:14<1:26:57,  2.01it/s]
  2%|▏         | 173/10682 [03:15<1:26:50,  2.02it/s]
  2%|▏         | 174/10682 [03:15<1:26:47,  2.02it/s]
  2%|▏         | 175/10682 [03:16<1:26:43,  2.02it/s]
                                                     {'loss': 6.8934, 'grad_norm': 0.41131341457366943, 'learning_rate': 0.00016370439663236668, 'epoch': 0.23}


  2%|▏         | 175/10682 [03:16<1:26:43,  2.02it/s]
  2%|▏         | 176/10682 [03:16<1:27:07,  2.01it/s]
  2%|▏         | 177/10682 [03:17<1:27:00,  2.01it/s]
  2%|▏         | 178/10682 [03:17<1:26:55,  2.01it/s]
  2%|▏         | 179/10682 [03:18<1:26:50,  2.02it/s]
  2%|▏         | 180/10682 [03:18<1:26:45,  2.02it/s]
  2%|▏         | 181/10682 [03:19<1:26:43,  2.02it/s]
  2%|▏         | 182/10682 [03:19<1:26:44,  2.02it/s]
  2%|▏         | 183/10682 [03:20<1:26:41,  2.02it/s]
  2%|▏         | 184/10682 [03:20<1:26:43,  2.02it/s]
  2%|▏         | 185/10682 [03:21<1:26:39,  2.02it/s]
  2%|▏         | 186/10682 [03:21<1:26:40,  2.02it/s]
  2%|▏         | 187/10682 [03:22<1:26:36,  2.02it/s]
  2%|▏         | 188/10682 [03:22<1:26:36,  2.02it/s]
  2%|▏         | 189/10682 [03:23<1:26:33,  2.02it/s]
  2%|▏         | 190/10682 [03:23<1:26:39,  2.02it/s]
  2%|▏         | 191/10682 [03:24<1:26:31,  2.02it/s]
  2%|▏         | 192/10682 [03:24<1:26:31,  2.02it/s]
  2%|▏         | 193/10682 [03:25<1:26:27,  2.02it/s]
  2%|▏         | 194/10682 [03:25<1:26:29,  2.02it/s]
  2%|▏         | 195/10682 [03:26<1:26:29,  2.02it/s]
  2%|▏         | 196/10682 [03:26<1:26:30,  2.02it/s]
  2%|▏         | 197/10682 [03:27<1:26:29,  2.02it/s]
  2%|▏         | 198/10682 [03:27<1:26:29,  2.02it/s]
  2%|▏         | 199/10682 [03:28<1:26:27,  2.02it/s]
  2%|▏         | 200/10682 [03:28<1:26:28,  2.02it/s]
                                                     
{'loss': 6.5356, 'grad_norm': 0.5325525403022766, 'learning_rate': 0.00018709073900841907, 'epoch': 0.26}

  2%|▏         | 200/10682 [03:28<1:26:28,  2.02it/s]
  2%|▏         | 201/10682 [03:29<1:26:34,  2.02it/s]
  2%|▏         | 202/10682 [03:29<1:26:32,  2.02it/s]
  2%|▏         | 203/10682 [03:30<1:26:30,  2.02it/s]
  2%|▏         | 204/10682 [03:30<1:26:28,  2.02it/s]
  2%|▏         | 205/10682 [03:31<1:26:26,  2.02it/s]
  2%|▏         | 206/10682 [03:31<1:26:28,  2.02it/s]
  2%|▏         | 207/10682 [03:32<1:26:31,  2.02it/s]
  2%|▏         | 208/10682 [03:32<1:26:32,  2.02it/s]
  2%|▏         | 209/10682 [03:33<1:26:34,  2.02it/s]
  2%|▏         | 210/10682 [03:33<1:26:28,  2.02it/s]
  2%|▏         | 211/10682 [03:34<1:26:28,  2.02it/s]
  2%|▏         | 212/10682 [03:34<1:26:22,  2.02it/s]
  2%|▏         | 213/10682 [03:35<1:26:26,  2.02it/s]
  2%|▏         | 214/10682 [03:35<1:26:22,  2.02it/s]
  2%|▏         | 215/10682 [03:36<1:26:24,  2.02it/s]
  2%|▏         | 216/10682 [03:36<1:26:19,  2.02it/s]
  2%|▏         | 217/10682 [03:37<1:26:23,  2.02it/s]
  2%|▏         | 218/10682 [03:37<1:26:34,  2.01it/s]
  2%|▏         | 219/10682 [03:38<1:26:29,  2.02it/s]
  2%|▏         | 220/10682 [03:38<1:26:26,  2.02it/s]
  2%|▏         | 221/10682 [03:39<1:26:20,  2.02it/s]
  2%|▏         | 222/10682 [03:39<1:26:25,  2.02it/s]
  2%|▏         | 223/10682 [03:40<1:26:21,  2.02it/s]
  2%|▏         | 224/10682 [03:40<1:26:19,  2.02it/s]
  2%|▏         | 225/10682 [03:41<1:26:18,  2.02it/s]{'loss': 6.2746, 'grad_norm': 0.6378172636032104, 'learning_rate': 0.00021047708138447147, 'epoch': 0.29}

                                                     

  2%|▏         | 225/10682 [03:41<1:26:18,  2.02it/s]
  2%|▏         | 226/10682 [03:41<1:26:22,  2.02it/s]
  2%|▏         | 227/10682 [03:42<1:26:20,  2.02it/s]
  2%|▏         | 228/10682 [03:42<1:26:16,  2.02it/s]
  2%|▏         | 229/10682 [03:43<1:26:15,  2.02it/s]
  2%|▏         | 230/10682 [03:43<1:26:14,  2.02it/s]
  2%|▏         | 231/10682 [03:44<1:26:19,  2.02it/s]
  2%|▏         | 232/10682 [03:44<1:26:16,  2.02it/s]
  2%|▏         | 233/10682 [03:45<1:26:19,  2.02it/s]
  2%|▏         | 234/10682 [03:45<1:26:14,  2.02it/s]
  2%|▏         | 235/10682 [03:46<1:26:18,  2.02it/s]
  2%|▏         | 236/10682 [03:46<1:26:15,  2.02it/s]
  2%|▏         | 237/10682 [03:47<1:26:17,  2.02it/s]
  2%|▏         | 238/10682 [03:47<1:26:15,  2.02it/s]
  2%|▏         | 239/10682 [03:48<1:26:12,  2.02it/s]
  2%|▏         | 240/10682 [03:48<1:26:12,  2.02it/s]
  2%|▏         | 241/10682 [03:49<1:26:14,  2.02it/s]
  2%|▏         | 242/10682 [03:49<1:26:12,  2.02it/s]
  2%|▏         | 243/10682 [03:50<1:26:14,  2.02it/s]
  2%|▏         | 244/10682 [03:50<1:26:08,  2.02it/s]
  2%|▏         | 245/10682 [03:51<1:26:09,  2.02it/s]
  2%|▏         | 246/10682 [03:51<1:26:02,  2.02it/s]
  2%|▏         | 247/10682 [03:52<1:26:08,  2.02it/s]
  2%|▏         | 248/10682 [03:52<1:26:05,  2.02it/s]
  2%|▏         | 249/10682 [03:53<1:26:10,  2.02it/s]
  2%|▏         | 250/10682 [03:53<1:26:13,  2.02it/s]
                                                     {'loss': 6.0641, 'grad_norm': 0.664199709892273, 'learning_rate': 0.00023386342376052386, 'epoch': 0.33}


  2%|▏         | 250/10682 [03:53<1:26:13,  2.02it/s]
  2%|▏         | 251/10682 [03:54<1:26:20,  2.01it/s]
  2%|▏         | 252/10682 [03:54<1:26:16,  2.01it/s]
  2%|▏         | 253/10682 [03:54<1:26:18,  2.01it/s]
  2%|▏         | 254/10682 [03:55<1:26:13,  2.02it/s]
  2%|▏         | 255/10682 [03:55<1:26:14,  2.01it/s]
  2%|▏         | 256/10682 [03:56<1:26:10,  2.02it/s]
  2%|▏         | 257/10682 [03:56<1:26:11,  2.02it/s]
  2%|▏         | 258/10682 [03:57<1:26:08,  2.02it/s]
  2%|▏         | 259/10682 [03:57<1:26:08,  2.02it/s]
  2%|▏         | 260/10682 [03:58<1:26:00,  2.02it/s]
  2%|▏         | 261/10682 [03:58<1:26:10,  2.02it/s]
  2%|▏         | 262/10682 [03:59<1:26:03,  2.02it/s]
  2%|▏         | 263/10682 [03:59<1:26:02,  2.02it/s]
  2%|▏         | 264/10682 [04:00<1:26:01,  2.02it/s]
  2%|▏         | 265/10682 [04:00<1:25:58,  2.02it/s]
  2%|▏         | 266/10682 [04:01<1:25:53,  2.02it/s]
  2%|▏         | 267/10682 [04:01<1:25:53,  2.02it/s]
  3%|▎         | 268/10682 [04:02<1:25:51,  2.02it/s]
  3%|▎         | 269/10682 [04:02<1:25:52,  2.02it/s]
  3%|▎         | 270/10682 [04:03<1:25:59,  2.02it/s]
  3%|▎         | 271/10682 [04:03<1:25:55,  2.02it/s]
  3%|▎         | 272/10682 [04:04<1:25:55,  2.02it/s]
  3%|▎         | 273/10682 [04:04<1:25:53,  2.02it/s]
  3%|▎         | 274/10682 [04:05<1:25:56,  2.02it/s]
  3%|▎         | 275/10682 [04:05<1:25:52,  2.02it/s]{'loss': 5.8867, 'grad_norm': 0.7667765021324158, 'learning_rate': 0.00025724976613657625, 'epoch': 0.36}

                                                     

  3%|▎         | 275/10682 [04:05<1:25:52,  2.02it/s]
  3%|▎         | 276/10682 [04:06<1:26:00,  2.02it/s]
  3%|▎         | 277/10682 [04:06<1:25:55,  2.02it/s]
  3%|▎         | 278/10682 [04:07<1:25:58,  2.02it/s]
  3%|▎         | 279/10682 [04:07<1:25:56,  2.02it/s]
  3%|▎         | 280/10682 [04:08<1:25:54,  2.02it/s]
  3%|▎         | 281/10682 [04:08<1:25:53,  2.02it/s]
  3%|▎         | 282/10682 [04:09<1:25:50,  2.02it/s]
  3%|▎         | 283/10682 [04:09<1:25:51,  2.02it/s]
  3%|▎         | 284/10682 [04:10<1:25:48,  2.02it/s]
  3%|▎         | 285/10682 [04:10<1:25:50,  2.02it/s]
  3%|▎         | 286/10682 [04:11<1:25:55,  2.02it/s]
  3%|▎         | 287/10682 [04:11<1:25:57,  2.02it/s]
  3%|▎         | 288/10682 [04:12<1:25:56,  2.02it/s]
  3%|▎         | 289/10682 [04:12<1:25:51,  2.02it/s]
  3%|▎         | 290/10682 [04:13<1:25:57,  2.02it/s]
  3%|▎         | 291/10682 [04:13<1:25:52,  2.02it/s]
  3%|▎         | 292/10682 [04:14<1:25:53,  2.02it/s]
  3%|▎         | 293/10682 [04:14<1:25:51,  2.02it/s]
  3%|▎         | 294/10682 [04:15<1:25:50,  2.02it/s]
  3%|▎         | 295/10682 [04:15<1:25:48,  2.02it/s]
  3%|▎         | 296/10682 [04:16<1:25:44,  2.02it/s]
  3%|▎         | 297/10682 [04:16<1:25:45,  2.02it/s]
  3%|▎         | 298/10682 [04:17<1:25:41,  2.02it/s]
  3%|▎         | 299/10682 [04:17<1:25:46,  2.02it/s]
  3%|▎         | 300/10682 [04:18<1:25:41,  2.02it/s]{'loss': 5.736, 'grad_norm': 0.7596151232719421, 'learning_rate': 0.0002806361085126286, 'epoch': 0.39}

                                                     

  3%|▎         | 300/10682 [04:18<1:25:41,  2.02it/s]
  3%|▎         | 301/10682 [04:18<1:25:51,  2.02it/s]
  3%|▎         | 302/10682 [04:19<1:25:46,  2.02it/s]
  3%|▎         | 303/10682 [04:19<1:25:45,  2.02it/s]
  3%|▎         | 304/10682 [04:20<1:25:42,  2.02it/s]
  3%|▎         | 305/10682 [04:20<1:25:42,  2.02it/s]
  3%|▎         | 306/10682 [04:21<1:25:41,  2.02it/s]
  3%|▎         | 307/10682 [04:21<1:25:39,  2.02it/s]
  3%|▎         | 308/10682 [04:22<1:25:38,  2.02it/s]
  3%|▎         | 309/10682 [04:22<1:25:36,  2.02it/s]
  3%|▎         | 310/10682 [04:23<1:25:36,  2.02it/s]
  3%|▎         | 311/10682 [04:23<1:25:36,  2.02it/s]
  3%|▎         | 312/10682 [04:24<1:25:38,  2.02it/s]
  3%|▎         | 313/10682 [04:24<1:25:35,  2.02it/s]
  3%|▎         | 314/10682 [04:25<1:25:38,  2.02it/s]
  3%|▎         | 315/10682 [04:25<1:25:34,  2.02it/s]
  3%|▎         | 316/10682 [04:26<1:25:38,  2.02it/s]
  3%|▎         | 317/10682 [04:26<1:25:32,  2.02it/s]
  3%|▎         | 318/10682 [04:27<1:25:34,  2.02it/s]
  3%|▎         | 319/10682 [04:27<1:25:30,  2.02it/s]
  3%|▎         | 320/10682 [04:28<1:25:33,  2.02it/s]
  3%|▎         | 321/10682 [04:28<1:25:26,  2.02it/s]
  3%|▎         | 322/10682 [04:29<1:25:29,  2.02it/s]
  3%|▎         | 323/10682 [04:29<1:25:22,  2.02it/s]
  3%|▎         | 324/10682 [04:30<1:25:28,  2.02it/s]
  3%|▎         | 325/10682 [04:30<1:25:22,  2.02it/s]
                                                     
{'loss': 5.6214, 'grad_norm': 0.7320352792739868, 'learning_rate': 0.00030402245088868103, 'epoch': 0.43}

  3%|▎         | 325/10682 [04:30<1:25:22,  2.02it/s]
  3%|▎         | 326/10682 [04:31<1:25:30,  2.02it/s]
  3%|▎         | 327/10682 [04:31<1:25:25,  2.02it/s]
  3%|▎         | 328/10682 [04:32<1:25:26,  2.02it/s]
  3%|▎         | 329/10682 [04:32<1:25:26,  2.02it/s]
  3%|▎         | 330/10682 [04:33<1:25:29,  2.02it/s]
  3%|▎         | 331/10682 [04:33<1:25:26,  2.02it/s]
  3%|▎         | 332/10682 [04:34<1:25:31,  2.02it/s]
  3%|▎         | 333/10682 [04:34<1:25:27,  2.02it/s]
  3%|▎         | 334/10682 [04:35<1:25:26,  2.02it/s]
  3%|▎         | 335/10682 [04:35<1:25:23,  2.02it/s]
  3%|▎         | 336/10682 [04:36<1:25:27,  2.02it/s]
  3%|▎         | 337/10682 [04:36<1:25:19,  2.02it/s]
  3%|▎         | 338/10682 [04:37<1:25:21,  2.02it/s]
  3%|▎         | 339/10682 [04:37<1:25:18,  2.02it/s]
  3%|▎         | 340/10682 [04:38<1:25:20,  2.02it/s]
  3%|▎         | 341/10682 [04:38<1:25:13,  2.02it/s]
  3%|▎         | 342/10682 [04:39<1:25:18,  2.02it/s]
  3%|▎         | 343/10682 [04:39<1:25:15,  2.02it/s]
  3%|▎         | 344/10682 [04:40<1:25:20,  2.02it/s]
  3%|▎         | 345/10682 [04:40<1:25:14,  2.02it/s]
  3%|▎         | 346/10682 [04:41<1:25:16,  2.02it/s]
  3%|▎         | 347/10682 [04:41<1:25:17,  2.02it/s]
  3%|▎         | 348/10682 [04:42<1:25:17,  2.02it/s]
  3%|▎         | 349/10682 [04:42<1:25:16,  2.02it/s]
  3%|▎         | 350/10682 [04:43<1:25:14,  2.02it/s]
                                                     
{'loss': 5.5113, 'grad_norm': 0.6104658842086792, 'learning_rate': 0.00032740879326473337, 'epoch': 0.46}

  3%|▎         | 350/10682 [04:43<1:25:14,  2.02it/s]
  3%|▎         | 351/10682 [04:43<1:25:23,  2.02it/s]
  3%|▎         | 352/10682 [04:44<1:25:17,  2.02it/s]
  3%|▎         | 353/10682 [04:44<1:25:18,  2.02it/s]
  3%|▎         | 354/10682 [04:45<1:25:15,  2.02it/s]
  3%|▎         | 355/10682 [04:45<1:25:21,  2.02it/s]
  3%|▎         | 356/10682 [04:46<1:25:19,  2.02it/s]
  3%|▎         | 357/10682 [04:46<1:25:22,  2.02it/s]
  3%|▎         | 358/10682 [04:47<1:25:24,  2.01it/s]
  3%|▎         | 359/10682 [04:47<1:25:22,  2.02it/s]
  3%|▎         | 360/10682 [04:48<1:25:22,  2.02it/s]
  3%|▎         | 361/10682 [04:48<1:25:18,  2.02it/s]
  3%|▎         | 362/10682 [04:48<1:25:20,  2.02it/s]
  3%|▎         | 363/10682 [04:49<1:25:15,  2.02it/s]
  3%|▎         | 364/10682 [04:49<1:25:16,  2.02it/s]
  3%|▎         | 365/10682 [04:50<1:25:16,  2.02it/s]
  3%|▎         | 366/10682 [04:50<1:25:16,  2.02it/s]
  3%|▎         | 367/10682 [04:51<1:25:13,  2.02it/s]
  3%|▎         | 368/10682 [04:51<1:25:15,  2.02it/s]
  3%|▎         | 369/10682 [04:52<1:25:11,  2.02it/s]
  3%|▎         | 370/10682 [04:52<1:25:13,  2.02it/s]
  3%|▎         | 371/10682 [04:53<1:25:15,  2.02it/s]
  3%|▎         | 372/10682 [04:53<1:25:13,  2.02it/s]
  3%|▎         | 373/10682 [04:54<1:25:13,  2.02it/s]
  4%|▎         | 374/10682 [04:54<1:25:08,  2.02it/s]
  4%|▎         | 375/10682 [04:55<1:25:08,  2.02it/s]{'loss': 5.4065, 'grad_norm': 0.6258764863014221, 'learning_rate': 0.0003507951356407858, 'epoch': 0.49}

                                                     

  4%|▎         | 375/10682 [04:55<1:25:08,  2.02it/s]
  4%|▎         | 376/10682 [04:55<1:25:08,  2.02it/s]
  4%|▎         | 377/10682 [04:56<1:25:10,  2.02it/s]
  4%|▎         | 378/10682 [04:56<1:25:06,  2.02it/s]
  4%|▎         | 379/10682 [04:57<1:25:07,  2.02it/s]
  4%|▎         | 380/10682 [04:57<1:24:59,  2.02it/s]
  4%|▎         | 381/10682 [04:58<1:25:01,  2.02it/s]
  4%|▎         | 382/10682 [04:58<1:24:57,  2.02it/s]
  4%|▎         | 383/10682 [04:59<1:24:59,  2.02it/s]
  4%|▎         | 384/10682 [04:59<1:24:55,  2.02it/s]
  4%|▎         | 385/10682 [05:00<1:24:58,  2.02it/s]
  4%|▎         | 386/10682 [05:00<1:24:56,  2.02it/s]
  4%|▎         | 387/10682 [05:01<1:24:56,  2.02it/s]
  4%|▎         | 388/10682 [05:01<1:24:56,  2.02it/s]
  4%|▎         | 389/10682 [05:02<1:24:56,  2.02it/s]
  4%|▎         | 390/10682 [05:02<1:24:57,  2.02it/s]
  4%|▎         | 391/10682 [05:03<1:24:59,  2.02it/s]
  4%|▎         | 392/10682 [05:03<1:25:00,  2.02it/s]
  4%|▎         | 393/10682 [05:04<1:24:56,  2.02it/s]
  4%|▎         | 394/10682 [05:04<1:24:57,  2.02it/s]
  4%|▎         | 395/10682 [05:05<1:24:52,  2.02it/s]
  4%|▎         | 396/10682 [05:05<1:24:57,  2.02it/s]
  4%|▎         | 397/10682 [05:06<1:24:52,  2.02it/s]
  4%|▎         | 398/10682 [05:06<1:24:51,  2.02it/s]
  4%|▎         | 399/10682 [05:07<1:24:47,  2.02it/s]
  4%|▎         | 400/10682 [05:07<1:24:50,  2.02it/s]
                                                     
{'loss': 5.3179, 'grad_norm': 0.6563537120819092, 'learning_rate': 0.00037418147801683815, 'epoch': 0.52}

  4%|▎         | 400/10682 [05:07<1:24:50,  2.02it/s]
  4%|▍         | 401/10682 [05:08<1:24:59,  2.02it/s]
  4%|▍         | 402/10682 [05:08<1:24:56,  2.02it/s]
  4%|▍         | 403/10682 [05:09<1:24:55,  2.02it/s]
  4%|▍         | 404/10682 [05:09<1:24:52,  2.02it/s]
  4%|▍         | 405/10682 [05:10<1:24:52,  2.02it/s]
  4%|▍         | 406/10682 [05:10<1:24:45,  2.02it/s]
  4%|▍         | 407/10682 [05:11<1:24:47,  2.02it/s]
  4%|▍         | 408/10682 [05:11<1:24:44,  2.02it/s]
  4%|▍         | 409/10682 [05:12<1:24:46,  2.02it/s]
  4%|▍         | 410/10682 [05:12<1:24:42,  2.02it/s]
  4%|▍         | 411/10682 [05:13<1:24:46,  2.02it/s]
  4%|▍         | 412/10682 [05:13<1:24:42,  2.02it/s]
  4%|▍         | 413/10682 [05:14<1:24:47,  2.02it/s]
  4%|▍         | 414/10682 [05:14<1:24:46,  2.02it/s]
  4%|▍         | 415/10682 [05:15<1:24:46,  2.02it/s]
  4%|▍         | 416/10682 [05:15<1:24:49,  2.02it/s]
  4%|▍         | 417/10682 [05:16<1:24:45,  2.02it/s]
  4%|▍         | 418/10682 [05:16<1:24:44,  2.02it/s]
  4%|▍         | 419/10682 [05:17<1:24:40,  2.02it/s]
  4%|▍         | 420/10682 [05:17<1:24:42,  2.02it/s]
  4%|▍         | 421/10682 [05:18<1:24:39,  2.02it/s]
  4%|▍         | 422/10682 [05:18<1:24:43,  2.02it/s]
  4%|▍         | 423/10682 [05:19<1:24:36,  2.02it/s]
  4%|▍         | 424/10682 [05:19<1:24:41,  2.02it/s]
  4%|▍         | 425/10682 [05:20<1:24:36,  2.02it/s]{'loss': 5.2373, 'grad_norm': 0.6940412521362305, 'learning_rate': 0.0003975678203928906, 'epoch': 0.56}
                                                     

  4%|▍         | 425/10682 [05:20<1:24:36,  2.02it/s]
  4%|▍         | 426/10682 [05:20<1:24:45,  2.02it/s]
  4%|▍         | 427/10682 [05:21<1:24:41,  2.02it/s]
  4%|▍         | 428/10682 [05:21<1:24:42,  2.02it/s]
  4%|▍         | 429/10682 [05:22<1:24:41,  2.02it/s]
  4%|▍         | 430/10682 [05:22<1:24:37,  2.02it/s]
  4%|▍         | 431/10682 [05:23<1:24:42,  2.02it/s]
  4%|▍         | 432/10682 [05:23<1:24:36,  2.02it/s]
  4%|▍         | 433/10682 [05:24<1:24:38,  2.02it/s]
  4%|▍         | 434/10682 [05:24<1:24:34,  2.02it/s]
  4%|▍         | 435/10682 [05:25<1:24:39,  2.02it/s]
  4%|▍         | 436/10682 [05:25<1:24:32,  2.02it/s]
  4%|▍         | 437/10682 [05:26<1:24:34,  2.02it/s]
  4%|▍         | 438/10682 [05:26<1:24:35,  2.02it/s]
  4%|▍         | 439/10682 [05:27<1:24:39,  2.02it/s]
  4%|▍         | 440/10682 [05:27<1:24:34,  2.02it/s]
  4%|▍         | 441/10682 [05:28<1:24:34,  2.02it/s]
  4%|▍         | 442/10682 [05:28<1:24:31,  2.02it/s]
  4%|▍         | 443/10682 [05:29<1:24:29,  2.02it/s]
  4%|▍         | 444/10682 [05:29<1:24:28,  2.02it/s]
  4%|▍         | 445/10682 [05:30<1:24:31,  2.02it/s]
  4%|▍         | 446/10682 [05:30<1:24:32,  2.02it/s]
  4%|▍         | 447/10682 [05:31<1:24:35,  2.02it/s]
  4%|▍         | 448/10682 [05:31<1:24:33,  2.02it/s]
  4%|▍         | 449/10682 [05:32<1:24:36,  2.02it/s]
  4%|▍         | 450/10682 [05:32<1:24:39,  2.01it/s]
                                                     
{'loss': 5.156, 'grad_norm': 0.6822566986083984, 'learning_rate': 0.00042095416276894293, 'epoch': 0.59}

  4%|▍         | 450/10682 [05:32<1:24:39,  2.01it/s]
  4%|▍         | 451/10682 [05:33<1:24:41,  2.01it/s]
  4%|▍         | 452/10682 [05:33<1:24:40,  2.01it/s]
  4%|▍         | 453/10682 [05:34<1:24:35,  2.02it/s]
  4%|▍         | 454/10682 [05:34<1:24:36,  2.01it/s]
  4%|▍         | 455/10682 [05:35<1:24:31,  2.02it/s]
  4%|▍         | 456/10682 [05:35<1:24:31,  2.02it/s]
  4%|▍         | 457/10682 [05:36<1:24:24,  2.02it/s]
  4%|▍         | 458/10682 [05:36<1:24:27,  2.02it/s]
  4%|▍         | 459/10682 [05:37<1:24:24,  2.02it/s]
  4%|▍         | 460/10682 [05:37<1:24:21,  2.02it/s]
  4%|▍         | 461/10682 [05:38<1:24:22,  2.02it/s]
  4%|▍         | 462/10682 [05:38<1:24:24,  2.02it/s]
  4%|▍         | 463/10682 [05:39<1:24:21,  2.02it/s]
  4%|▍         | 464/10682 [05:39<1:24:21,  2.02it/s]
  4%|▍         | 465/10682 [05:40<1:24:26,  2.02it/s]
  4%|▍         | 466/10682 [05:40<1:24:20,  2.02it/s]
  4%|▍         | 467/10682 [05:41<1:24:22,  2.02it/s]
  4%|▍         | 468/10682 [05:41<1:24:18,  2.02it/s]
  4%|▍         | 469/10682 [05:42<1:24:16,  2.02it/s]
  4%|▍         | 470/10682 [05:42<1:24:14,  2.02it/s]
  4%|▍         | 471/10682 [05:42<1:24:11,  2.02it/s]
  4%|▍         | 472/10682 [05:43<1:24:17,  2.02it/s]
  4%|▍         | 473/10682 [05:43<1:24:20,  2.02it/s]
  4%|▍         | 474/10682 [05:44<1:24:23,  2.02it/s]
  4%|▍         | 475/10682 [05:44<1:24:24,  2.02it/s]
                                                     {'loss': 5.0844, 'grad_norm': 0.5810084342956543, 'learning_rate': 0.0004443405051449954, 'epoch': 0.62}


  4%|▍         | 475/10682 [05:44<1:24:24,  2.02it/s]
  4%|▍         | 476/10682 [05:45<1:24:22,  2.02it/s]
  4%|▍         | 477/10682 [05:45<1:24:24,  2.02it/s]
  4%|▍         | 478/10682 [05:46<1:24:19,  2.02it/s]
  4%|▍         | 479/10682 [05:46<1:24:18,  2.02it/s]
  4%|▍         | 480/10682 [05:47<1:24:12,  2.02it/s]
  5%|▍         | 481/10682 [05:47<1:24:09,  2.02it/s]
  5%|▍         | 482/10682 [05:48<1:24:04,  2.02it/s]
  5%|▍         | 483/10682 [05:48<1:24:09,  2.02it/s]
  5%|▍         | 484/10682 [05:49<1:24:05,  2.02it/s]
  5%|▍         | 485/10682 [05:49<1:24:08,  2.02it/s]
  5%|▍         | 486/10682 [05:50<1:24:10,  2.02it/s]
  5%|▍         | 487/10682 [05:50<1:24:07,  2.02it/s]
  5%|▍         | 488/10682 [05:51<1:24:08,  2.02it/s]
  5%|▍         | 489/10682 [05:51<1:24:05,  2.02it/s]
  5%|▍         | 490/10682 [05:52<1:24:08,  2.02it/s]
  5%|▍         | 491/10682 [05:52<1:24:00,  2.02it/s]
  5%|▍         | 492/10682 [05:53<1:24:04,  2.02it/s]
  5%|▍         | 493/10682 [05:53<1:23:59,  2.02it/s]
  5%|▍         | 494/10682 [05:54<1:24:01,  2.02it/s]
  5%|▍         | 495/10682 [05:54<1:23:59,  2.02it/s]
  5%|▍         | 496/10682 [05:55<1:24:02,  2.02it/s]
  5%|▍         | 497/10682 [05:55<1:24:01,  2.02it/s]
  5%|▍         | 498/10682 [05:56<1:24:04,  2.02it/s]
  5%|▍         | 499/10682 [05:56<1:23:56,  2.02it/s]
  5%|▍         | 500/10682 [05:57<1:24:00,  2.02it/s]
                                                     {'loss': 5.0179, 'grad_norm': 0.7091294527053833, 'learning_rate': 0.0004677268475210477, 'epoch': 0.66}


  5%|▍         | 500/10682 [05:57<1:24:00,  2.02it/s]
  5%|▍         | 501/10682 [05:57<1:24:05,  2.02it/s]
  5%|▍         | 502/10682 [05:58<1:24:04,  2.02it/s]
  5%|▍         | 503/10682 [05:58<1:24:02,  2.02it/s]
  5%|▍         | 504/10682 [05:59<1:24:03,  2.02it/s]
  5%|▍         | 505/10682 [05:59<1:24:02,  2.02it/s]
  5%|▍         | 506/10682 [06:00<1:24:00,  2.02it/s]
  5%|▍         | 507/10682 [06:00<1:24:05,  2.02it/s]
  5%|▍         | 508/10682 [06:01<1:23:59,  2.02it/s]
  5%|▍         | 509/10682 [06:01<1:24:01,  2.02it/s]
  5%|▍         | 510/10682 [06:02<1:23:58,  2.02it/s]
  5%|▍         | 511/10682 [06:02<1:23:57,  2.02it/s]
  5%|▍         | 512/10682 [06:03<1:23:56,  2.02it/s]
  5%|▍         | 513/10682 [06:03<1:23:56,  2.02it/s]
  5%|▍         | 514/10682 [06:04<1:23:58,  2.02it/s]
  5%|▍         | 515/10682 [06:04<1:23:57,  2.02it/s]
  5%|▍         | 516/10682 [06:05<1:23:59,  2.02it/s]
  5%|▍         | 517/10682 [06:05<1:23:53,  2.02it/s]
  5%|▍         | 518/10682 [06:06<1:23:57,  2.02it/s]
  5%|▍         | 519/10682 [06:06<1:23:53,  2.02it/s]
  5%|▍         | 520/10682 [06:07<1:23:57,  2.02it/s]
  5%|▍         | 521/10682 [06:07<1:23:54,  2.02it/s]
  5%|▍         | 522/10682 [06:08<1:23:58,  2.02it/s]
  5%|▍         | 523/10682 [06:08<1:23:53,  2.02it/s]
  5%|▍         | 524/10682 [06:09<1:23:57,  2.02it/s]
  5%|▍         | 525/10682 [06:09<1:23:57,  2.02it/s]{'loss': 4.9582, 'grad_norm': 0.6471843719482422, 'learning_rate': 0.0004911131898971, 'epoch': 0.69}
                                                     

  5%|▍         | 525/10682 [06:09<1:23:57,  2.02it/s]
  5%|▍         | 526/10682 [06:10<1:24:02,  2.01it/s]
  5%|▍         | 527/10682 [06:10<1:24:01,  2.01it/s]
  5%|▍         | 528/10682 [06:11<1:23:57,  2.02it/s]
  5%|▍         | 529/10682 [06:11<1:23:52,  2.02it/s]
  5%|▍         | 530/10682 [06:12<1:23:54,  2.02it/s]
  5%|▍         | 531/10682 [06:12<1:23:52,  2.02it/s]
  5%|▍         | 532/10682 [06:13<1:23:53,  2.02it/s]
  5%|▍         | 533/10682 [06:13<1:23:52,  2.02it/s]
  5%|▍         | 534/10682 [06:14<1:23:53,  2.02it/s]
  5%|▌         | 535/10682 [06:14<1:23:55,  2.01it/s]
  5%|▌         | 536/10682 [06:15<1:23:51,  2.02it/s]
  5%|▌         | 537/10682 [06:15<1:23:52,  2.02it/s]
  5%|▌         | 538/10682 [06:16<1:23:50,  2.02it/s]
  5%|▌         | 539/10682 [06:16<1:23:51,  2.02it/s]
  5%|▌         | 540/10682 [06:17<1:23:47,  2.02it/s]
  5%|▌         | 541/10682 [06:17<1:23:47,  2.02it/s]
  5%|▌         | 542/10682 [06:18<1:23:49,  2.02it/s]
  5%|▌         | 543/10682 [06:18<1:23:47,  2.02it/s]
  5%|▌         | 544/10682 [06:19<1:23:49,  2.02it/s]
  5%|▌         | 545/10682 [06:19<1:23:46,  2.02it/s]
  5%|▌         | 546/10682 [06:20<1:23:48,  2.02it/s]
  5%|▌         | 547/10682 [06:20<1:23:41,  2.02it/s]
  5%|▌         | 548/10682 [06:21<1:23:43,  2.02it/s]
  5%|▌         | 549/10682 [06:21<1:23:41,  2.02it/s]
  5%|▌         | 550/10682 [06:22<1:23:43,  2.02it/s]
                                                     
{'loss': 4.8969, 'grad_norm': 0.6236402988433838, 'learning_rate': 0.0005144995322731525, 'epoch': 0.72}

  5%|▌         | 550/10682 [06:22<1:23:43,  2.02it/s]
  5%|▌         | 551/10682 [06:22<1:23:49,  2.01it/s]
  5%|▌         | 552/10682 [06:23<1:23:43,  2.02it/s]
  5%|▌         | 553/10682 [06:23<1:23:44,  2.02it/s]
  5%|▌         | 554/10682 [06:24<1:23:40,  2.02it/s]
  5%|▌         | 555/10682 [06:24<1:23:41,  2.02it/s]
  5%|▌         | 556/10682 [06:25<1:23:37,  2.02it/s]
  5%|▌         | 557/10682 [06:25<1:23:39,  2.02it/s]
  5%|▌         | 558/10682 [06:26<1:23:33,  2.02it/s]
  5%|▌         | 559/10682 [06:26<1:23:33,  2.02it/s]
  5%|▌         | 560/10682 [06:27<1:23:33,  2.02it/s]
  5%|▌         | 561/10682 [06:27<1:23:33,  2.02it/s]
  5%|▌         | 562/10682 [06:28<1:23:32,  2.02it/s]
  5%|▌         | 563/10682 [06:28<1:23:28,  2.02it/s]
  5%|▌         | 564/10682 [06:29<1:23:34,  2.02it/s]
  5%|▌         | 565/10682 [06:29<1:23:30,  2.02it/s]
  5%|▌         | 566/10682 [06:30<1:23:37,  2.02it/s]
  5%|▌         | 567/10682 [06:30<1:23:33,  2.02it/s]
  5%|▌         | 568/10682 [06:31<1:23:36,  2.02it/s]
  5%|▌         | 569/10682 [06:31<1:23:39,  2.01it/s]
  5%|▌         | 570/10682 [06:32<1:23:34,  2.02it/s]
  5%|▌         | 571/10682 [06:32<1:23:32,  2.02it/s]
  5%|▌         | 572/10682 [06:33<1:23:30,  2.02it/s]
  5%|▌         | 573/10682 [06:33<1:23:29,  2.02it/s]
  5%|▌         | 574/10682 [06:34<1:23:25,  2.02it/s]
  5%|▌         | 575/10682 [06:34<1:23:27,  2.02it/s]{'loss': 4.8462, 'grad_norm': 0.6192285418510437, 'learning_rate': 0.0005378858746492049, 'epoch': 0.75}

                                                     

  5%|▌         | 575/10682 [06:34<1:23:27,  2.02it/s]
  5%|▌         | 576/10682 [06:35<1:23:28,  2.02it/s]
  5%|▌         | 577/10682 [06:35<1:23:31,  2.02it/s]
  5%|▌         | 578/10682 [06:36<1:23:29,  2.02it/s]
  5%|▌         | 579/10682 [06:36<1:23:27,  2.02it/s]
  5%|▌         | 580/10682 [06:37<1:23:27,  2.02it/s]
  5%|▌         | 581/10682 [06:37<1:23:25,  2.02it/s]
  5%|▌         | 582/10682 [06:38<1:23:26,  2.02it/s]
  5%|▌         | 583/10682 [06:38<1:23:22,  2.02it/s]
  5%|▌         | 584/10682 [06:39<1:23:24,  2.02it/s]
  5%|▌         | 585/10682 [06:39<1:23:21,  2.02it/s]
  5%|▌         | 586/10682 [06:39<1:23:23,  2.02it/s]
  5%|▌         | 587/10682 [06:40<1:23:21,  2.02it/s]
  6%|▌         | 588/10682 [06:40<1:23:21,  2.02it/s]
  6%|▌         | 589/10682 [06:41<1:23:23,  2.02it/s]
  6%|▌         | 590/10682 [06:41<1:23:25,  2.02it/s]
  6%|▌         | 591/10682 [06:42<1:23:26,  2.02it/s]
  6%|▌         | 592/10682 [06:42<1:23:24,  2.02it/s]
  6%|▌         | 593/10682 [06:43<1:23:22,  2.02it/s]
  6%|▌         | 594/10682 [06:43<1:23:22,  2.02it/s]
  6%|▌         | 595/10682 [06:44<1:23:22,  2.02it/s]
  6%|▌         | 596/10682 [06:44<1:23:19,  2.02it/s]
  6%|▌         | 597/10682 [06:45<1:23:17,  2.02it/s]
  6%|▌         | 598/10682 [06:45<1:23:19,  2.02it/s]
  6%|▌         | 599/10682 [06:46<1:23:15,  2.02it/s]
  6%|▌         | 600/10682 [06:46<1:23:16,  2.02it/s]
                                                     {'loss': 4.8063, 'grad_norm': 0.48572197556495667, 'learning_rate': 0.0005612722170252572, 'epoch': 0.79}

  6%|▌         | 600/10682 [06:46<1:23:16,  2.02it/s]
  6%|▌         | 601/10682 [06:47<1:23:26,  2.01it/s]
  6%|▌         | 602/10682 [06:47<1:23:22,  2.01it/s]
  6%|▌         | 603/10682 [06:48<1:23:23,  2.01it/s]
  6%|▌         | 604/10682 [06:48<1:23:26,  2.01it/s]
  6%|▌         | 605/10682 [06:49<1:23:21,  2.01it/s]
  6%|▌         | 606/10682 [06:49<1:23:22,  2.01it/s]
  6%|▌         | 607/10682 [06:50<1:23:15,  2.02it/s]
  6%|▌         | 608/10682 [06:50<1:23:18,  2.02it/s]
  6%|▌         | 609/10682 [06:51<1:30:19,  1.86it/s]
  6%|▌         | 610/10682 [06:52<1:28:14,  1.90it/s]
  6%|▌         | 611/10682 [06:52<1:26:47,  1.93it/s]
  6%|▌         | 612/10682 [06:53<1:25:43,  1.96it/s]
  6%|▌         | 613/10682 [06:53<1:24:56,  1.98it/s]
  6%|▌         | 614/10682 [06:54<1:24:19,  1.99it/s]
  6%|▌         | 615/10682 [06:54<1:24:01,  2.00it/s]
  6%|▌         | 616/10682 [06:55<1:31:17,  1.84it/s]
  6%|▌         | 617/10682 [06:55<1:28:42,  1.89it/s]
  6%|▌         | 618/10682 [06:56<1:27:04,  1.93it/s]
  6%|▌         | 619/10682 [06:56<1:25:48,  1.95it/s]
  6%|▌         | 620/10682 [06:57<1:25:03,  1.97it/s]
  6%|▌         | 621/10682 [06:57<1:24:26,  1.99it/s]
  6%|▌         | 622/10682 [06:58<1:24:02,  2.00it/s]
  6%|▌         | 623/10682 [06:58<1:23:41,  2.00it/s]
  6%|▌         | 624/10682 [06:59<1:23:32,  2.01it/s]
  6%|▌         | 625/10682 [06:59<1:23:24,  2.01it/s]
                                                     {'loss': 4.758, 'grad_norm': 0.5485813617706299, 'learning_rate': 0.0005846585594013096, 'epoch': 0.82}


  6%|▌         | 625/10682 [06:59<1:23:24,  2.01it/s]
  6%|▌         | 626/10682 [07:00<1:23:18,  2.01it/s]
  6%|▌         | 627/10682 [07:00<1:23:12,  2.01it/s]
  6%|▌         | 628/10682 [07:01<1:23:05,  2.02it/s]
  6%|▌         | 629/10682 [07:01<1:23:04,  2.02it/s]
  6%|▌         | 630/10682 [07:02<1:23:00,  2.02it/s]
  6%|▌         | 631/10682 [07:02<1:22:59,  2.02it/s]
  6%|▌         | 632/10682 [07:03<1:22:58,  2.02it/s]
  6%|▌         | 633/10682 [07:03<1:23:01,  2.02it/s]
  6%|▌         | 634/10682 [07:04<1:22:57,  2.02it/s]
  6%|▌         | 635/10682 [07:04<1:23:00,  2.02it/s]
  6%|▌         | 636/10682 [07:05<1:23:00,  2.02it/s]
  6%|▌         | 637/10682 [07:05<1:22:59,  2.02it/s]
  6%|▌         | 638/10682 [07:06<1:22:56,  2.02it/s]
  6%|▌         | 639/10682 [07:06<1:22:56,  2.02it/s]
  6%|▌         | 640/10682 [07:07<1:22:52,  2.02it/s]
  6%|▌         | 641/10682 [07:07<1:22:54,  2.02it/s]
  6%|▌         | 642/10682 [07:08<1:22:47,  2.02it/s]
  6%|▌         | 643/10682 [07:08<1:22:49,  2.02it/s]
  6%|▌         | 644/10682 [07:09<1:22:45,  2.02it/s]
  6%|▌         | 645/10682 [07:09<1:22:49,  2.02it/s]
  6%|▌         | 646/10682 [07:10<1:22:45,  2.02it/s]
  6%|▌         | 647/10682 [07:10<1:22:48,  2.02it/s]
  6%|▌         | 648/10682 [07:11<1:22:46,  2.02it/s]
  6%|▌         | 649/10682 [07:11<1:22:50,  2.02it/s]
  6%|▌         | 650/10682 [07:12<1:22:48,  2.02it/s]{'loss': 4.7245, 'grad_norm': 0.5011232495307922, 'learning_rate': 0.0006080449017773621, 'epoch': 0.85}

                                                     

  6%|▌         | 650/10682 [07:12<1:22:48,  2.02it/s]
  6%|▌         | 651/10682 [07:12<1:22:55,  2.02it/s]
  6%|▌         | 652/10682 [07:12<1:22:50,  2.02it/s]
  6%|▌         | 653/10682 [07:13<1:22:50,  2.02it/s]
  6%|▌         | 654/10682 [07:13<1:22:49,  2.02it/s]
  6%|▌         | 655/10682 [07:14<1:22:50,  2.02it/s]
  6%|▌         | 656/10682 [07:14<1:22:50,  2.02it/s]
  6%|▌         | 657/10682 [07:15<1:22:50,  2.02it/s]
  6%|▌         | 658/10682 [07:15<1:22:51,  2.02it/s]
  6%|▌         | 659/10682 [07:16<1:22:56,  2.01it/s]
  6%|▌         | 660/10682 [07:16<1:22:52,  2.02it/s]
  6%|▌         | 661/10682 [07:17<1:22:55,  2.01it/s]
  6%|▌         | 662/10682 [07:17<1:22:49,  2.02it/s]
  6%|▌         | 663/10682 [07:18<1:22:50,  2.02it/s]
  6%|▌         | 664/10682 [07:18<1:22:45,  2.02it/s]
  6%|▌         | 665/10682 [07:19<1:22:48,  2.02it/s]
  6%|▌         | 666/10682 [07:19<1:22:43,  2.02it/s]
  6%|▌         | 667/10682 [07:20<1:22:46,  2.02it/s]
  6%|▋         | 668/10682 [07:20<1:22:40,  2.02it/s]
  6%|▋         | 669/10682 [07:21<1:22:43,  2.02it/s]
  6%|▋         | 670/10682 [07:21<1:22:38,  2.02it/s]
  6%|▋         | 671/10682 [07:22<1:22:41,  2.02it/s]
  6%|▋         | 672/10682 [07:22<1:22:39,  2.02it/s]
  6%|▋         | 673/10682 [07:23<1:22:40,  2.02it/s]
  6%|▋         | 674/10682 [07:23<1:22:44,  2.02it/s]
  6%|▋         | 675/10682 [07:24<1:22:41,  2.02it/s]
                                                     
{'loss': 4.6818, 'grad_norm': 0.47154542803764343, 'learning_rate': 0.0006314312441534145, 'epoch': 0.88}

  6%|▋         | 675/10682 [07:24<1:22:41,  2.02it/s]
  6%|▋         | 676/10682 [07:24<1:22:48,  2.01it/s]
  6%|▋         | 677/10682 [07:25<1:22:43,  2.02it/s]
  6%|▋         | 678/10682 [07:25<1:22:41,  2.02it/s]
  6%|▋         | 679/10682 [07:26<1:22:41,  2.02it/s]
  6%|▋         | 680/10682 [07:26<1:22:37,  2.02it/s]
  6%|▋         | 681/10682 [07:27<1:22:42,  2.02it/s]
  6%|▋         | 682/10682 [07:27<1:22:35,  2.02it/s]
  6%|▋         | 683/10682 [07:28<1:22:40,  2.02it/s]
  6%|▋         | 684/10682 [07:28<1:22:35,  2.02it/s]
  6%|▋         | 685/10682 [07:29<1:22:38,  2.02it/s]
  6%|▋         | 686/10682 [07:29<1:22:44,  2.01it/s]
  6%|▋         | 687/10682 [07:30<1:22:39,  2.02it/s]
  6%|▋         | 688/10682 [07:30<1:22:37,  2.02it/s]
  6%|▋         | 689/10682 [07:31<1:22:37,  2.02it/s]
  6%|▋         | 690/10682 [07:31<1:22:38,  2.02it/s]
  6%|▋         | 691/10682 [07:32<1:22:37,  2.02it/s]
  6%|▋         | 692/10682 [07:32<1:22:38,  2.01it/s]
  6%|▋         | 693/10682 [07:33<1:22:37,  2.01it/s]
  6%|▋         | 694/10682 [07:33<1:22:36,  2.01it/s]
  7%|▋         | 695/10682 [07:34<1:22:32,  2.02it/s]
  7%|▋         | 696/10682 [07:34<1:22:30,  2.02it/s]
  7%|▋         | 697/10682 [07:35<1:22:27,  2.02it/s]
  7%|▋         | 698/10682 [07:35<1:22:28,  2.02it/s]
  7%|▋         | 699/10682 [07:36<1:22:26,  2.02it/s]
  7%|▋         | 700/10682 [07:36<1:22:27,  2.02it/s]
                                                     {'loss': 4.6438, 'grad_norm': 0.4610711336135864, 'learning_rate': 0.0006548175865294667, 'epoch': 0.92}


  7%|▋         | 700/10682 [07:36<1:22:27,  2.02it/s]
  7%|▋         | 701/10682 [07:37<1:22:33,  2.02it/s]
  7%|▋         | 702/10682 [07:37<1:22:33,  2.01it/s]
  7%|▋         | 703/10682 [07:38<1:22:30,  2.02it/s]
  7%|▋         | 704/10682 [07:38<1:22:30,  2.02it/s]
  7%|▋         | 705/10682 [07:39<1:22:27,  2.02it/s]
  7%|▋         | 706/10682 [07:39<1:22:30,  2.02it/s]
  7%|▋         | 707/10682 [07:40<1:22:25,  2.02it/s]
  7%|▋         | 708/10682 [07:40<1:22:27,  2.02it/s]
  7%|▋         | 709/10682 [07:41<1:22:31,  2.01it/s]
  7%|▋         | 710/10682 [07:41<1:22:26,  2.02it/s]
  7%|▋         | 711/10682 [07:42<1:22:30,  2.01it/s]
  7%|▋         | 712/10682 [07:42<1:22:26,  2.02it/s]
  7%|▋         | 713/10682 [07:43<1:22:29,  2.01it/s]
  7%|▋         | 714/10682 [07:43<1:22:20,  2.02it/s]
  7%|▋         | 715/10682 [07:44<1:22:23,  2.02it/s]
  7%|▋         | 716/10682 [07:44<1:22:17,  2.02it/s]
  7%|▋         | 717/10682 [07:45<1:22:21,  2.02it/s]
  7%|▋         | 718/10682 [07:45<1:22:12,  2.02it/s]
  7%|▋         | 719/10682 [07:46<1:22:17,  2.02it/s]
  7%|▋         | 720/10682 [07:46<1:22:15,  2.02it/s]
  7%|▋         | 721/10682 [07:47<1:22:15,  2.02it/s]
  7%|▋         | 722/10682 [07:47<1:22:08,  2.02it/s]
  7%|▋         | 723/10682 [07:48<1:22:18,  2.02it/s]
  7%|▋         | 724/10682 [07:48<1:22:23,  2.01it/s]
  7%|▋         | 725/10682 [07:49<1:22:25,  2.01it/s]{'loss': 4.6151, 'grad_norm': 0.5662415623664856, 'learning_rate': 0.0006782039289055192, 'epoch': 0.95}

                                                     

  7%|▋         | 725/10682 [07:49<1:22:25,  2.01it/s]
  7%|▋         | 726/10682 [07:49<1:22:33,  2.01it/s]
  7%|▋         | 727/10682 [07:50<1:22:22,  2.01it/s]
  7%|▋         | 728/10682 [07:50<1:22:20,  2.01it/s]
  7%|▋         | 729/10682 [07:51<1:22:11,  2.02it/s]
  7%|▋         | 730/10682 [07:51<1:22:14,  2.02it/s]
  7%|▋         | 731/10682 [07:52<1:22:03,  2.02it/s]
  7%|▋         | 732/10682 [07:52<1:22:08,  2.02it/s]
  7%|▋         | 733/10682 [07:53<1:22:14,  2.02it/s]
  7%|▋         | 734/10682 [07:53<1:22:13,  2.02it/s]
  7%|▋         | 735/10682 [07:54<1:22:12,  2.02it/s]
  7%|▋         | 736/10682 [07:54<1:22:06,  2.02it/s]
  7%|▋         | 737/10682 [07:55<1:22:00,  2.02it/s]
  7%|▋         | 738/10682 [07:55<1:22:01,  2.02it/s]
  7%|▋         | 739/10682 [07:56<1:22:02,  2.02it/s]
  7%|▋         | 740/10682 [07:56<1:22:03,  2.02it/s]
  7%|▋         | 741/10682 [07:57<1:21:58,  2.02it/s]
  7%|▋         | 742/10682 [07:57<1:21:59,  2.02it/s]
  7%|▋         | 743/10682 [07:58<1:21:52,  2.02it/s]
  7%|▋         | 744/10682 [07:58<1:22:02,  2.02it/s]
  7%|▋         | 745/10682 [07:59<1:21:57,  2.02it/s]
  7%|▋         | 746/10682 [07:59<1:22:02,  2.02it/s]
  7%|▋         | 747/10682 [08:00<1:21:56,  2.02it/s]
  7%|▋         | 748/10682 [08:00<1:22:01,  2.02it/s]
  7%|▋         | 749/10682 [08:01<1:22:05,  2.02it/s]
  7%|▋         | 750/10682 [08:01<1:22:11,  2.01it/s]{'loss': 4.5735, 'grad_norm': 0.44774675369262695, 'learning_rate': 0.0007015902712815716, 'epoch': 0.98}
                                                     

  7%|▋         | 750/10682 [08:01<1:22:11,  2.01it/s]
  7%|▋         | 751/10682 [08:02<1:22:24,  2.01it/s]
  7%|▋         | 752/10682 [08:02<1:22:19,  2.01it/s]
  7%|▋         | 753/10682 [08:03<1:22:11,  2.01it/s]
  7%|▋         | 754/10682 [08:03<1:22:06,  2.02it/s]
  7%|▋         | 755/10682 [08:04<1:21:59,  2.02it/s]
  7%|▋         | 756/10682 [08:04<1:22:06,  2.02it/s]
  7%|▋         | 757/10682 [08:05<1:22:02,  2.02it/s]
  7%|▋         | 758/10682 [08:05<1:22:01,  2.02it/s]
  7%|▋         | 759/10682 [08:06<1:22:00,  2.02it/s]
  7%|▋         | 760/10682 [08:06<1:21:59,  2.02it/s]
  7%|▋         | 761/10682 [08:07<1:21:54,  2.02it/s]
  7%|▋         | 762/10682 [08:07<1:22:03,  2.01it/s]
  7%|▋         | 763/10682 [08:08<1:21:41,  2.02it/s]
  7%|▋         | 764/10682 [08:32<21:00:59,  7.63s/it]
  7%|▋         | 765/10682 [08:32<15:07:00,  5.49s/it]
  7%|▋         | 766/10682 [08:33<10:59:31,  3.99s/it]
  7%|▋         | 767/10682 [08:33<8:06:05,  2.94s/it] 
  7%|▋         | 768/10682 [08:34<6:04:38,  2.21s/it]
  7%|▋         | 769/10682 [08:34<4:39:36,  1.69s/it]
  7%|▋         | 770/10682 [08:35<3:40:10,  1.33s/it]
  7%|▋         | 771/10682 [08:35<2:58:28,  1.08s/it]
  7%|▋         | 772/10682 [08:36<2:29:20,  1.11it/s]
  7%|▋         | 773/10682 [08:36<2:09:02,  1.28it/s]
  7%|▋         | 774/10682 [08:37<1:55:10,  1.43it/s]
  7%|▋         | 775/10682 [08:37<1:45:06,  1.57it/s]
                                                     {'loss': 4.5249, 'grad_norm': 0.44595780968666077, 'learning_rate': 0.0007249766136576241, 'epoch': 1.02}


  7%|▋         | 775/10682 [08:37<1:45:06,  1.57it/s]
  7%|▋         | 776/10682 [08:38<1:38:11,  1.68it/s]
  7%|▋         | 777/10682 [08:38<1:33:09,  1.77it/s]
  7%|▋         | 778/10682 [08:39<1:29:41,  1.84it/s]
  7%|▋         | 779/10682 [08:39<1:27:20,  1.89it/s]
  7%|▋         | 780/10682 [08:40<1:25:38,  1.93it/s]
  7%|▋         | 781/10682 [08:40<1:24:27,  1.95it/s]
  7%|▋         | 782/10682 [08:41<1:23:35,  1.97it/s]
  7%|▋         | 783/10682 [08:41<1:23:08,  1.98it/s]
  7%|▋         | 784/10682 [08:42<1:22:40,  2.00it/s]
  7%|▋         | 785/10682 [08:42<1:22:23,  2.00it/s]
  7%|▋         | 786/10682 [08:43<1:22:08,  2.01it/s]
  7%|▋         | 787/10682 [08:43<1:22:06,  2.01it/s]
  7%|▋         | 788/10682 [08:44<1:21:54,  2.01it/s]
  7%|▋         | 789/10682 [08:44<1:21:57,  2.01it/s]
  7%|▋         | 790/10682 [08:45<1:21:53,  2.01it/s]