[2024-10-18 22:43:34,450][04690] Saving configuration to /content/train_dir/default_experiment/config.json... [2024-10-18 22:43:34,453][04690] Rollout worker 0 uses device cpu [2024-10-18 22:43:34,456][04690] Rollout worker 1 uses device cpu [2024-10-18 22:43:34,457][04690] Rollout worker 2 uses device cpu [2024-10-18 22:43:34,460][04690] Rollout worker 3 uses device cpu [2024-10-18 22:43:34,462][04690] Rollout worker 4 uses device cpu [2024-10-18 22:43:34,464][04690] Rollout worker 5 uses device cpu [2024-10-18 22:43:34,467][04690] Rollout worker 6 uses device cpu [2024-10-18 22:43:34,468][04690] Rollout worker 7 uses device cpu [2024-10-18 22:45:34,714][04690] Environment doom_basic already registered, overwriting... [2024-10-18 22:45:34,718][04690] Environment doom_two_colors_easy already registered, overwriting... [2024-10-18 22:45:34,720][04690] Environment doom_two_colors_hard already registered, overwriting... [2024-10-18 22:45:34,722][04690] Environment doom_dm already registered, overwriting... [2024-10-18 22:45:34,724][04690] Environment doom_dwango5 already registered, overwriting... [2024-10-18 22:45:34,726][04690] Environment doom_my_way_home_flat_actions already registered, overwriting... [2024-10-18 22:45:34,728][04690] Environment doom_defend_the_center_flat_actions already registered, overwriting... [2024-10-18 22:45:34,730][04690] Environment doom_my_way_home already registered, overwriting... [2024-10-18 22:45:34,731][04690] Environment doom_deadly_corridor already registered, overwriting... [2024-10-18 22:45:34,734][04690] Environment doom_defend_the_center already registered, overwriting... [2024-10-18 22:45:34,736][04690] Environment doom_defend_the_line already registered, overwriting... [2024-10-18 22:45:34,738][04690] Environment doom_health_gathering already registered, overwriting... [2024-10-18 22:45:34,740][04690] Environment doom_health_gathering_supreme already registered, overwriting... [2024-10-18 22:45:34,741][04690] Environment doom_battle already registered, overwriting... [2024-10-18 22:45:34,743][04690] Environment doom_battle2 already registered, overwriting... [2024-10-18 22:45:34,744][04690] Environment doom_duel_bots already registered, overwriting... [2024-10-18 22:45:34,746][04690] Environment doom_deathmatch_bots already registered, overwriting... [2024-10-18 22:45:34,747][04690] Environment doom_duel already registered, overwriting... [2024-10-18 22:45:34,749][04690] Environment doom_deathmatch_full already registered, overwriting... [2024-10-18 22:45:34,750][04690] Environment doom_benchmark already registered, overwriting... [2024-10-18 22:45:34,752][04690] register_encoder_factory: [2024-10-18 22:45:34,784][04690] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-10-18 22:45:34,787][04690] Overriding arg 'device' with value 'cpu' passed from command line [2024-10-18 22:45:34,795][04690] Experiment dir /content/train_dir/default_experiment already exists! [2024-10-18 22:45:34,798][04690] Resuming existing experiment from /content/train_dir/default_experiment... [2024-10-18 22:45:34,800][04690] Weights and Biases integration disabled [2024-10-18 22:45:34,805][04690] Environment var CUDA_VISIBLE_DEVICES is [2024-10-18 22:45:36,853][04690] Starting experiment with the following configuration: help=False algo=APPO env=doom_health_gathering_supreme experiment=default_experiment train_dir=/content/train_dir restart_behavior=resume device=cpu seed=None num_policies=1 async_rl=True serial_mode=False batched_sampling=False num_batches_to_accumulate=2 worker_num_splits=2 policy_workers_per_policy=1 max_policy_lag=1000 num_workers=8 num_envs_per_worker=4 batch_size=1024 num_batches_per_epoch=1 num_epochs=1 rollout=32 recurrence=32 shuffle_minibatches=False gamma=0.99 reward_scale=1.0 reward_clip=1000.0 value_bootstrap=False normalize_returns=True exploration_loss_coeff=0.001 value_loss_coeff=0.5 kl_loss_coeff=0.0 exploration_loss=symmetric_kl gae_lambda=0.95 ppo_clip_ratio=0.1 ppo_clip_value=0.2 with_vtrace=False vtrace_rho=1.0 vtrace_c=1.0 optimizer=adam adam_eps=1e-06 adam_beta1=0.9 adam_beta2=0.999 max_grad_norm=4.0 learning_rate=0.0001 lr_schedule=constant lr_schedule_kl_threshold=0.008 lr_adaptive_min=1e-06 lr_adaptive_max=0.01 obs_subtract_mean=0.0 obs_scale=255.0 normalize_input=True normalize_input_keys=None decorrelate_experience_max_seconds=0 decorrelate_envs_on_one_worker=True actor_worker_gpus=[] set_workers_cpu_affinity=True force_envs_single_thread=False default_niceness=0 log_to_file=True experiment_summaries_interval=10 flush_summaries_interval=30 stats_avg=100 summaries_use_frameskip=True heartbeat_interval=20 heartbeat_reporting_interval=600 train_for_env_steps=4000000 train_for_seconds=10000000000 save_every_sec=120 keep_checkpoints=2 load_checkpoint_kind=latest save_milestones_sec=-1 save_best_every_sec=5 save_best_metric=reward save_best_after=100000 benchmark=False encoder_mlp_layers=[512, 512] encoder_conv_architecture=convnet_simple encoder_conv_mlp_layers=[512] use_rnn=True rnn_size=512 rnn_type=gru rnn_num_layers=1 decoder_mlp_layers=[] nonlinearity=elu policy_initialization=orthogonal policy_init_gain=1.0 actor_critic_share_weights=True adaptive_stddev=True continuous_tanh_scale=0.0 initial_stddev=1.0 use_env_info_cache=False env_gpu_actions=False env_gpu_observations=True env_frameskip=4 env_framestack=1 pixel_format=CHW use_record_episode_statistics=False with_wandb=False wandb_user=None wandb_project=sample_factory wandb_group=None wandb_job_type=SF wandb_tags=[] with_pbt=False pbt_mix_policies_in_one_env=True pbt_period_env_steps=5000000 pbt_start_mutation=20000000 pbt_replace_fraction=0.3 pbt_mutation_rate=0.15 pbt_replace_reward_gap=0.1 pbt_replace_reward_gap_absolute=1e-06 pbt_optimize_gamma=False pbt_target_objective=true_objective pbt_perturb_min=1.1 pbt_perturb_max=1.5 num_agents=-1 num_humans=0 num_bots=-1 start_bot_difficulty=None timelimit=None res_w=128 res_h=72 wide_aspect_ratio=False eval_env_frameskip=1 fps=35 command_line=--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=4 --train_for_env_steps=4000000 cli_args={'env': 'doom_health_gathering_supreme', 'num_workers': 8, 'num_envs_per_worker': 4, 'train_for_env_steps': 4000000} git_hash=unknown git_repo_name=not a git repository [2024-10-18 22:45:36,856][04690] Saving configuration to /content/train_dir/default_experiment/config.json... [2024-10-18 22:45:36,859][04690] Rollout worker 0 uses device cpu [2024-10-18 22:45:36,864][04690] Rollout worker 1 uses device cpu [2024-10-18 22:45:36,865][04690] Rollout worker 2 uses device cpu [2024-10-18 22:45:36,866][04690] Rollout worker 3 uses device cpu [2024-10-18 22:45:36,867][04690] Rollout worker 4 uses device cpu [2024-10-18 22:45:36,872][04690] Rollout worker 5 uses device cpu [2024-10-18 22:45:36,874][04690] Rollout worker 6 uses device cpu [2024-10-18 22:45:36,875][04690] Rollout worker 7 uses device cpu [2024-10-18 22:45:37,124][04690] InferenceWorker_p0-w0: min num requests: 2 [2024-10-18 22:45:37,177][04690] Starting all processes... [2024-10-18 22:45:37,180][04690] Starting process learner_proc0 [2024-10-18 22:45:37,248][04690] Starting all processes... [2024-10-18 22:45:37,261][04690] Starting process inference_proc0-0 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc0 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc1 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc2 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc3 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc4 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc5 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc6 [2024-10-18 22:45:37,266][04690] Starting process rollout_proc7 [2024-10-18 22:45:53,858][05483] Worker 5 uses CPU cores [1] [2024-10-18 22:45:53,871][05464] Starting seed is not provided [2024-10-18 22:45:53,872][05464] Initializing actor-critic model on device cpu [2024-10-18 22:45:53,873][05464] RunningMeanStd input shape: (3, 72, 128) [2024-10-18 22:45:53,875][05464] RunningMeanStd input shape: (1,) [2024-10-18 22:45:54,150][05464] ConvEncoder: input_channels=3 [2024-10-18 22:45:54,558][05482] Worker 4 uses CPU cores [0] [2024-10-18 22:45:54,606][05480] Worker 3 uses CPU cores [1] [2024-10-18 22:45:54,669][05484] Worker 6 uses CPU cores [0] [2024-10-18 22:45:54,798][05477] Worker 0 uses CPU cores [0] [2024-10-18 22:45:55,081][05479] Worker 1 uses CPU cores [1] [2024-10-18 22:45:55,323][05481] Worker 2 uses CPU cores [0] [2024-10-18 22:45:55,439][05485] Worker 7 uses CPU cores [1] [2024-10-18 22:45:55,532][05464] Conv encoder output size: 512 [2024-10-18 22:45:55,534][05464] Policy head output size: 512 [2024-10-18 22:45:55,604][05464] Created Actor Critic model with architecture: [2024-10-18 22:45:55,606][05464] ActorCriticSharedWeights( (obs_normalizer): ObservationNormalizer( (running_mean_std): RunningMeanStdDictInPlace( (running_mean_std): ModuleDict( (obs): RunningMeanStdInPlace() ) ) ) (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) (encoder): VizdoomEncoder( (basic_encoder): ConvEncoder( (enc): RecursiveScriptModule( original_name=ConvEncoderImpl (conv_head): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Conv2d) (1): RecursiveScriptModule(original_name=ELU) (2): RecursiveScriptModule(original_name=Conv2d) (3): RecursiveScriptModule(original_name=ELU) (4): RecursiveScriptModule(original_name=Conv2d) (5): RecursiveScriptModule(original_name=ELU) ) (mlp_layers): RecursiveScriptModule( original_name=Sequential (0): RecursiveScriptModule(original_name=Linear) (1): RecursiveScriptModule(original_name=ELU) ) ) ) ) (core): ModelCoreRNN( (core): GRU(512, 512) ) (decoder): MlpDecoder( (mlp): Identity() ) (critic_linear): Linear(in_features=512, out_features=1, bias=True) (action_parameterization): ActionParameterizationDefault( (distribution_linear): Linear(in_features=512, out_features=5, bias=True) ) ) [2024-10-18 22:45:57,125][04690] Heartbeat connected on InferenceWorker_p0-w0 [2024-10-18 22:45:57,134][04690] Heartbeat connected on RolloutWorker_w0 [2024-10-18 22:45:57,141][04690] Heartbeat connected on RolloutWorker_w1 [2024-10-18 22:45:57,147][04690] Heartbeat connected on RolloutWorker_w2 [2024-10-18 22:45:57,153][04690] Heartbeat connected on RolloutWorker_w3 [2024-10-18 22:45:57,160][04690] Heartbeat connected on RolloutWorker_w4 [2024-10-18 22:45:57,165][04690] Heartbeat connected on RolloutWorker_w5 [2024-10-18 22:45:57,174][04690] Heartbeat connected on RolloutWorker_w6 [2024-10-18 22:45:57,182][04690] Heartbeat connected on RolloutWorker_w7 [2024-10-18 22:45:57,366][04690] Heartbeat connected on Batcher_0 [2024-10-18 22:45:57,389][05464] Using optimizer [2024-10-18 22:45:57,391][05464] No checkpoints found [2024-10-18 22:45:57,391][05464] Did not load from checkpoint, starting from scratch! [2024-10-18 22:45:57,392][05464] Initialized policy 0 weights for model version 0 [2024-10-18 22:45:57,401][05478] RunningMeanStd input shape: (3, 72, 128) [2024-10-18 22:45:57,403][05478] RunningMeanStd input shape: (1,) [2024-10-18 22:45:57,402][05464] LearnerWorker_p0 finished initialization! [2024-10-18 22:45:57,405][04690] Heartbeat connected on LearnerWorker_p0 [2024-10-18 22:45:57,525][05478] ConvEncoder: input_channels=3 [2024-10-18 22:45:57,721][05478] Conv encoder output size: 512 [2024-10-18 22:45:57,722][05478] Policy head output size: 512 [2024-10-18 22:45:57,752][04690] Inference worker 0-0 is ready! [2024-10-18 22:45:57,755][04690] All inference workers are ready! Signal rollout workers to start! [2024-10-18 22:45:57,860][05485] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:57,859][05483] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:57,861][05479] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:57,863][05480] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:57,986][05484] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:58,000][05482] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:58,011][05481] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:58,002][05477] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-18 22:45:59,805][04690] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-10-18 22:46:00,140][05479] Decorrelating experience for 0 frames... [2024-10-18 22:46:00,138][05483] Decorrelating experience for 0 frames... [2024-10-18 22:46:00,548][05484] Decorrelating experience for 0 frames... [2024-10-18 22:46:00,560][05477] Decorrelating experience for 0 frames... [2024-10-18 22:46:00,563][05481] Decorrelating experience for 0 frames... [2024-10-18 22:46:00,566][05482] Decorrelating experience for 0 frames... [2024-10-18 22:46:00,733][05485] Decorrelating experience for 0 frames... [2024-10-18 22:46:01,884][05484] Decorrelating experience for 32 frames... [2024-10-18 22:46:01,893][05477] Decorrelating experience for 32 frames... [2024-10-18 22:46:01,897][05479] Decorrelating experience for 32 frames... [2024-10-18 22:46:01,906][05482] Decorrelating experience for 32 frames... [2024-10-18 22:46:02,156][05485] Decorrelating experience for 32 frames... [2024-10-18 22:46:02,202][05480] Decorrelating experience for 0 frames... [2024-10-18 22:46:03,177][05479] Decorrelating experience for 64 frames... [2024-10-18 22:46:03,351][05481] Decorrelating experience for 32 frames... [2024-10-18 22:46:03,348][05485] Decorrelating experience for 64 frames... [2024-10-18 22:46:03,527][05477] Decorrelating experience for 64 frames... [2024-10-18 22:46:03,529][05484] Decorrelating experience for 64 frames... [2024-10-18 22:46:04,266][05483] Decorrelating experience for 32 frames... [2024-10-18 22:46:04,446][05485] Decorrelating experience for 96 frames... [2024-10-18 22:46:04,805][04690] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-10-18 22:46:05,152][05482] Decorrelating experience for 64 frames... [2024-10-18 22:46:05,211][05481] Decorrelating experience for 64 frames... [2024-10-18 22:46:05,376][05477] Decorrelating experience for 96 frames... [2024-10-18 22:46:05,993][05484] Decorrelating experience for 96 frames... [2024-10-18 22:46:07,044][05479] Decorrelating experience for 96 frames... [2024-10-18 22:46:07,376][05483] Decorrelating experience for 64 frames... [2024-10-18 22:46:08,668][05481] Decorrelating experience for 96 frames... [2024-10-18 22:46:09,809][04690] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 45.4. Samples: 454. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-10-18 22:46:09,814][04690] Avg episode reward: [(0, '2.794')] [2024-10-18 22:46:10,150][05482] Decorrelating experience for 96 frames... [2024-10-18 22:46:13,104][05480] Decorrelating experience for 32 frames... [2024-10-18 22:46:14,276][05483] Decorrelating experience for 96 frames... [2024-10-18 22:46:14,808][04690] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 104.2. Samples: 1564. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) [2024-10-18 22:46:14,810][04690] Avg episode reward: [(0, '2.768')] [2024-10-18 22:46:16,637][05464] Signal inference workers to stop experience collection... [2024-10-18 22:46:16,665][05478] InferenceWorker_p0-w0: stopping experience collection [2024-10-18 22:46:16,880][05480] Decorrelating experience for 64 frames... [2024-10-18 22:46:17,259][05464] Signal inference workers to resume experience collection... [2024-10-18 22:46:17,263][05478] InferenceWorker_p0-w0: resuming experience collection [2024-10-18 22:46:17,390][05480] Decorrelating experience for 96 frames... [2024-10-18 22:46:19,805][04690] Fps is (10 sec: 409.8, 60 sec: 204.8, 300 sec: 204.8). Total num frames: 4096. Throughput: 0: 142.3. Samples: 2846. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2024-10-18 22:46:19,808][04690] Avg episode reward: [(0, '3.025')] [2024-10-18 22:46:24,805][04690] Fps is (10 sec: 819.4, 60 sec: 327.7, 300 sec: 327.7). Total num frames: 8192. Throughput: 0: 184.6. Samples: 4614. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) [2024-10-18 22:46:24,812][04690] Avg episode reward: [(0, '3.359')] [2024-10-18 22:46:29,806][04690] Fps is (10 sec: 819.1, 60 sec: 409.6, 300 sec: 409.6). Total num frames: 12288. Throughput: 0: 155.0. Samples: 4650. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:46:29,809][04690] Avg episode reward: [(0, '3.367')] [2024-10-18 22:46:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 468.1, 300 sec: 468.1). Total num frames: 16384. Throughput: 0: 161.8. Samples: 5662. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:46:34,813][04690] Avg episode reward: [(0, '3.485')] [2024-10-18 22:46:39,805][04690] Fps is (10 sec: 819.3, 60 sec: 512.0, 300 sec: 512.0). Total num frames: 20480. Throughput: 0: 160.6. Samples: 6424. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:46:39,813][04690] Avg episode reward: [(0, '3.905')] [2024-10-18 22:46:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 546.1, 300 sec: 546.1). Total num frames: 24576. Throughput: 0: 173.4. Samples: 7804. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:46:44,812][04690] Avg episode reward: [(0, '3.914')] [2024-10-18 22:46:49,808][04690] Fps is (10 sec: 818.9, 60 sec: 573.4, 300 sec: 573.4). Total num frames: 28672. Throughput: 0: 205.3. Samples: 9240. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:46:49,819][04690] Avg episode reward: [(0, '3.893')] [2024-10-18 22:46:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 595.8, 300 sec: 595.8). Total num frames: 32768. Throughput: 0: 206.8. Samples: 9758. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:46:54,808][04690] Avg episode reward: [(0, '3.927')] [2024-10-18 22:46:59,805][04690] Fps is (10 sec: 819.5, 60 sec: 614.4, 300 sec: 614.4). Total num frames: 36864. Throughput: 0: 204.1. Samples: 10746. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:46:59,809][04690] Avg episode reward: [(0, '3.999')] [2024-10-18 22:47:03,130][05478] Updated weights for policy 0, policy_version 10 (0.3191) [2024-10-18 22:47:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 682.7, 300 sec: 630.2). Total num frames: 40960. Throughput: 0: 205.9. Samples: 12110. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:47:04,808][04690] Avg episode reward: [(0, '4.325')] [2024-10-18 22:47:09,806][04690] Fps is (10 sec: 819.1, 60 sec: 751.0, 300 sec: 643.7). Total num frames: 45056. Throughput: 0: 181.4. Samples: 12778. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:47:09,808][04690] Avg episode reward: [(0, '4.364')] [2024-10-18 22:47:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 655.4). Total num frames: 49152. Throughput: 0: 203.2. Samples: 13794. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:47:14,808][04690] Avg episode reward: [(0, '4.471')] [2024-10-18 22:47:19,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 665.6). Total num frames: 53248. Throughput: 0: 213.1. Samples: 15250. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:47:19,808][04690] Avg episode reward: [(0, '4.490')] [2024-10-18 22:47:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 674.6). Total num frames: 57344. Throughput: 0: 211.6. Samples: 15946. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:47:24,811][04690] Avg episode reward: [(0, '4.388')] [2024-10-18 22:47:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 682.7). Total num frames: 61440. Throughput: 0: 207.1. Samples: 17124. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:47:29,811][04690] Avg episode reward: [(0, '4.352')] [2024-10-18 22:47:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 689.9). Total num frames: 65536. Throughput: 0: 207.8. Samples: 18590. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:47:34,813][04690] Avg episode reward: [(0, '4.477')] [2024-10-18 22:47:39,482][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000018_73728.pth... [2024-10-18 22:47:39,812][04690] Fps is (10 sec: 1227.9, 60 sec: 887.4, 300 sec: 737.2). Total num frames: 73728. Throughput: 0: 215.3. Samples: 19450. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:47:39,819][04690] Avg episode reward: [(0, '4.490')] [2024-10-18 22:47:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 702.2). Total num frames: 73728. Throughput: 0: 227.5. Samples: 20982. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:47:44,808][04690] Avg episode reward: [(0, '4.493')] [2024-10-18 22:47:49,805][04690] Fps is (10 sec: 409.9, 60 sec: 819.2, 300 sec: 707.5). Total num frames: 77824. Throughput: 0: 216.5. Samples: 21854. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:47:49,813][04690] Avg episode reward: [(0, '4.468')] [2024-10-18 22:47:50,374][05478] Updated weights for policy 0, policy_version 20 (0.1486) [2024-10-18 22:47:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 748.0). Total num frames: 86016. Throughput: 0: 218.4. Samples: 22608. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:47:54,808][04690] Avg episode reward: [(0, '4.468')] [2024-10-18 22:47:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 750.9). Total num frames: 90112. Throughput: 0: 228.1. Samples: 24058. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:47:59,810][04690] Avg episode reward: [(0, '4.420')] [2024-10-18 22:48:04,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 753.7). Total num frames: 94208. Throughput: 0: 219.6. Samples: 25134. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:48:04,812][04690] Avg episode reward: [(0, '4.420')] [2024-10-18 22:48:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 756.2). Total num frames: 98304. Throughput: 0: 240.7. Samples: 26776. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:48:09,809][04690] Avg episode reward: [(0, '4.377')] [2024-10-18 22:48:14,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 758.5). Total num frames: 102400. Throughput: 0: 230.1. Samples: 27478. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:48:14,810][04690] Avg episode reward: [(0, '4.291')] [2024-10-18 22:48:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 760.7). Total num frames: 106496. Throughput: 0: 231.2. Samples: 28992. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:48:19,809][04690] Avg episode reward: [(0, '4.288')] [2024-10-18 22:48:21,295][05464] Saving new best policy, reward=4.291! [2024-10-18 22:48:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 762.7). Total num frames: 110592. Throughput: 0: 218.1. Samples: 29262. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:48:24,808][04690] Avg episode reward: [(0, '4.272')] [2024-10-18 22:48:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 764.6). Total num frames: 114688. Throughput: 0: 214.8. Samples: 30646. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:48:29,808][04690] Avg episode reward: [(0, '4.232')] [2024-10-18 22:48:34,103][05478] Updated weights for policy 0, policy_version 30 (0.0983) [2024-10-18 22:48:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 792.8). Total num frames: 122880. Throughput: 0: 229.8. Samples: 32196. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:48:34,807][04690] Avg episode reward: [(0, '4.192')] [2024-10-18 22:48:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.3, 300 sec: 768.0). Total num frames: 122880. Throughput: 0: 235.9. Samples: 33224. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:48:39,809][04690] Avg episode reward: [(0, '4.172')] [2024-10-18 22:48:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 794.4). Total num frames: 131072. Throughput: 0: 224.8. Samples: 34176. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 22:48:44,810][04690] Avg episode reward: [(0, '4.246')] [2024-10-18 22:48:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 795.1). Total num frames: 135168. Throughput: 0: 234.9. Samples: 35704. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 22:48:49,813][04690] Avg episode reward: [(0, '4.279')] [2024-10-18 22:48:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 795.8). Total num frames: 139264. Throughput: 0: 213.2. Samples: 36372. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:48:54,810][04690] Avg episode reward: [(0, '4.275')] [2024-10-18 22:48:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 796.4). Total num frames: 143360. Throughput: 0: 221.5. Samples: 37444. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:48:59,809][04690] Avg episode reward: [(0, '4.256')] [2024-10-18 22:49:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 797.1). Total num frames: 147456. Throughput: 0: 227.7. Samples: 39240. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:49:04,807][04690] Avg episode reward: [(0, '4.352')] [2024-10-18 22:49:09,432][05464] Saving new best policy, reward=4.352! [2024-10-18 22:49:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 819.2). Total num frames: 155648. Throughput: 0: 232.9. Samples: 39742. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:49:09,812][04690] Avg episode reward: [(0, '4.368')] [2024-10-18 22:49:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 798.2). Total num frames: 155648. Throughput: 0: 234.1. Samples: 41180. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:49:14,807][04690] Avg episode reward: [(0, '4.453')] [2024-10-18 22:49:15,476][05464] Saving new best policy, reward=4.368! [2024-10-18 22:49:19,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 798.7). Total num frames: 159744. Throughput: 0: 226.5. Samples: 42388. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:49:19,813][04690] Avg episode reward: [(0, '4.450')] [2024-10-18 22:49:21,073][05464] Saving new best policy, reward=4.453! [2024-10-18 22:49:21,080][05478] Updated weights for policy 0, policy_version 40 (0.0534) [2024-10-18 22:49:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 799.2). Total num frames: 163840. Throughput: 0: 210.0. Samples: 42672. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:49:24,808][04690] Avg episode reward: [(0, '4.451')] [2024-10-18 22:49:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 799.7). Total num frames: 167936. Throughput: 0: 218.1. Samples: 43990. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:49:29,813][04690] Avg episode reward: [(0, '4.412')] [2024-10-18 22:49:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 800.1). Total num frames: 172032. Throughput: 0: 206.7. Samples: 45006. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:49:34,808][04690] Avg episode reward: [(0, '4.451')] [2024-10-18 22:49:36,665][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000043_176128.pth... [2024-10-18 22:49:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 800.6). Total num frames: 176128. Throughput: 0: 225.9. Samples: 46536. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:49:39,808][04690] Avg episode reward: [(0, '4.496')] [2024-10-18 22:49:44,067][05464] Saving new best policy, reward=4.496! [2024-10-18 22:49:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 819.2). Total num frames: 184320. Throughput: 0: 221.2. Samples: 47396. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:49:44,813][04690] Avg episode reward: [(0, '4.617')] [2024-10-18 22:49:49,508][05464] Saving new best policy, reward=4.617! [2024-10-18 22:49:49,807][04690] Fps is (10 sec: 1228.5, 60 sec: 887.4, 300 sec: 819.2). Total num frames: 188416. Throughput: 0: 208.3. Samples: 48616. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:49:49,811][04690] Avg episode reward: [(0, '4.562')] [2024-10-18 22:49:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 819.2). Total num frames: 192512. Throughput: 0: 219.8. Samples: 49634. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:49:54,809][04690] Avg episode reward: [(0, '4.549')] [2024-10-18 22:49:59,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 819.2). Total num frames: 196608. Throughput: 0: 209.5. Samples: 50608. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:49:59,810][04690] Avg episode reward: [(0, '4.434')] [2024-10-18 22:50:04,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 819.2). Total num frames: 200704. Throughput: 0: 217.2. Samples: 52164. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:04,821][04690] Avg episode reward: [(0, '4.432')] [2024-10-18 22:50:09,806][04690] Fps is (10 sec: 409.6, 60 sec: 750.9, 300 sec: 802.8). Total num frames: 200704. Throughput: 0: 218.8. Samples: 52520. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:09,828][04690] Avg episode reward: [(0, '4.487')] [2024-10-18 22:50:13,114][05478] Updated weights for policy 0, policy_version 50 (0.1459) [2024-10-18 22:50:14,807][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 803.1). Total num frames: 204800. Throughput: 0: 193.6. Samples: 52704. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:50:14,810][04690] Avg episode reward: [(0, '4.511')] [2024-10-18 22:50:16,543][05464] Signal inference workers to stop experience collection... (50 times) [2024-10-18 22:50:16,602][05478] InferenceWorker_p0-w0: stopping experience collection (50 times) [2024-10-18 22:50:17,952][05464] Signal inference workers to resume experience collection... (50 times) [2024-10-18 22:50:17,954][05478] InferenceWorker_p0-w0: resuming experience collection (50 times) [2024-10-18 22:50:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 803.4). Total num frames: 208896. Throughput: 0: 197.4. Samples: 53888. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:50:19,816][04690] Avg episode reward: [(0, '4.429')] [2024-10-18 22:50:24,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 803.7). Total num frames: 212992. Throughput: 0: 204.8. Samples: 55754. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:24,814][04690] Avg episode reward: [(0, '4.325')] [2024-10-18 22:50:29,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 804.0). Total num frames: 217088. Throughput: 0: 196.8. Samples: 56252. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:29,811][04690] Avg episode reward: [(0, '4.289')] [2024-10-18 22:50:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 804.3). Total num frames: 221184. Throughput: 0: 193.6. Samples: 57326. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:34,814][04690] Avg episode reward: [(0, '4.253')] [2024-10-18 22:50:39,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 804.6). Total num frames: 225280. Throughput: 0: 203.6. Samples: 58794. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:39,813][04690] Avg episode reward: [(0, '4.275')] [2024-10-18 22:50:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 233472. Throughput: 0: 204.5. Samples: 59810. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:44,811][04690] Avg episode reward: [(0, '4.282')] [2024-10-18 22:50:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 751.0, 300 sec: 805.1). Total num frames: 233472. Throughput: 0: 193.9. Samples: 60888. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:49,808][04690] Avg episode reward: [(0, '4.358')] [2024-10-18 22:50:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 819.2). Total num frames: 241664. Throughput: 0: 213.1. Samples: 62108. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:50:54,817][04690] Avg episode reward: [(0, '4.348')] [2024-10-18 22:50:58,031][05478] Updated weights for policy 0, policy_version 60 (0.0516) [2024-10-18 22:50:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 833.1). Total num frames: 245760. Throughput: 0: 226.8. Samples: 62908. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:50:59,812][04690] Avg episode reward: [(0, '4.285')] [2024-10-18 22:51:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 249856. Throughput: 0: 232.7. Samples: 64360. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:51:04,814][04690] Avg episode reward: [(0, '4.318')] [2024-10-18 22:51:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 253952. Throughput: 0: 216.3. Samples: 65488. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:51:09,808][04690] Avg episode reward: [(0, '4.344')] [2024-10-18 22:51:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 258048. Throughput: 0: 221.5. Samples: 66218. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:51:14,812][04690] Avg episode reward: [(0, '4.343')] [2024-10-18 22:51:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 262144. Throughput: 0: 240.7. Samples: 68158. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:51:19,810][04690] Avg episode reward: [(0, '4.366')] [2024-10-18 22:51:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 266240. Throughput: 0: 230.4. Samples: 69160. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:51:24,809][04690] Avg episode reward: [(0, '4.382')] [2024-10-18 22:51:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 270336. Throughput: 0: 218.4. Samples: 69638. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:51:29,812][04690] Avg episode reward: [(0, '4.398')] [2024-10-18 22:51:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 278528. Throughput: 0: 228.5. Samples: 71172. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:51:34,808][04690] Avg episode reward: [(0, '4.316')] [2024-10-18 22:51:38,434][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth... [2024-10-18 22:51:38,603][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000018_73728.pth [2024-10-18 22:51:39,806][04690] Fps is (10 sec: 1228.7, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 282624. Throughput: 0: 222.8. Samples: 72134. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:51:39,813][04690] Avg episode reward: [(0, '4.382')] [2024-10-18 22:51:44,405][05478] Updated weights for policy 0, policy_version 70 (0.0052) [2024-10-18 22:51:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 286720. Throughput: 0: 226.6. Samples: 73106. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:51:44,807][04690] Avg episode reward: [(0, '4.399')] [2024-10-18 22:51:49,805][04690] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 290816. Throughput: 0: 216.5. Samples: 74102. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:51:49,808][04690] Avg episode reward: [(0, '4.393')] [2024-10-18 22:51:54,806][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 290816. Throughput: 0: 214.0. Samples: 75116. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:51:54,809][04690] Avg episode reward: [(0, '4.461')] [2024-10-18 22:51:59,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 294912. Throughput: 0: 213.6. Samples: 75828. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:51:59,808][04690] Avg episode reward: [(0, '4.490')] [2024-10-18 22:52:04,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 299008. Throughput: 0: 198.4. Samples: 77088. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:04,815][04690] Avg episode reward: [(0, '4.495')] [2024-10-18 22:52:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 307200. Throughput: 0: 190.6. Samples: 77738. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:09,816][04690] Avg episode reward: [(0, '4.593')] [2024-10-18 22:52:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 311296. Throughput: 0: 214.5. Samples: 79292. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:14,809][04690] Avg episode reward: [(0, '4.642')] [2024-10-18 22:52:19,206][05464] Saving new best policy, reward=4.642! [2024-10-18 22:52:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 315392. Throughput: 0: 202.2. Samples: 80270. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:19,812][04690] Avg episode reward: [(0, '4.610')] [2024-10-18 22:52:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 319488. Throughput: 0: 216.2. Samples: 81864. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:24,808][04690] Avg episode reward: [(0, '4.623')] [2024-10-18 22:52:29,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 323584. Throughput: 0: 206.4. Samples: 82396. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:29,816][04690] Avg episode reward: [(0, '4.616')] [2024-10-18 22:52:31,503][05478] Updated weights for policy 0, policy_version 80 (0.1046) [2024-10-18 22:52:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 327680. Throughput: 0: 217.1. Samples: 83870. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:34,808][04690] Avg episode reward: [(0, '4.609')] [2024-10-18 22:52:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 331776. Throughput: 0: 205.3. Samples: 84354. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:39,808][04690] Avg episode reward: [(0, '4.550')] [2024-10-18 22:52:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 335872. Throughput: 0: 220.7. Samples: 85758. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:44,813][04690] Avg episode reward: [(0, '4.491')] [2024-10-18 22:52:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 344064. Throughput: 0: 229.7. Samples: 87424. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:49,812][04690] Avg episode reward: [(0, '4.373')] [2024-10-18 22:52:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 344064. Throughput: 0: 237.8. Samples: 88438. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:54,808][04690] Avg episode reward: [(0, '4.390')] [2024-10-18 22:52:59,806][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 352256. Throughput: 0: 223.6. Samples: 89356. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:52:59,809][04690] Avg episode reward: [(0, '4.319')] [2024-10-18 22:53:04,806][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 356352. Throughput: 0: 232.1. Samples: 90716. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:04,810][04690] Avg episode reward: [(0, '4.541')] [2024-10-18 22:53:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 360448. Throughput: 0: 221.9. Samples: 91848. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:09,807][04690] Avg episode reward: [(0, '4.599')] [2024-10-18 22:53:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 364544. Throughput: 0: 226.3. Samples: 92580. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:14,808][04690] Avg episode reward: [(0, '4.586')] [2024-10-18 22:53:17,458][05478] Updated weights for policy 0, policy_version 90 (0.1216) [2024-10-18 22:53:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 368640. Throughput: 0: 229.3. Samples: 94188. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:19,809][04690] Avg episode reward: [(0, '4.540')] [2024-10-18 22:53:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 372736. Throughput: 0: 252.3. Samples: 95708. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:24,808][04690] Avg episode reward: [(0, '4.560')] [2024-10-18 22:53:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 376832. Throughput: 0: 230.7. Samples: 96140. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:29,808][04690] Avg episode reward: [(0, '4.586')] [2024-10-18 22:53:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 380928. Throughput: 0: 221.4. Samples: 97388. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:53:34,813][04690] Avg episode reward: [(0, '4.603')] [2024-10-18 22:53:39,469][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000095_389120.pth... [2024-10-18 22:53:39,584][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000043_176128.pth [2024-10-18 22:53:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 389120. Throughput: 0: 234.0. Samples: 98968. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:53:39,812][04690] Avg episode reward: [(0, '4.586')] [2024-10-18 22:53:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 393216. Throughput: 0: 233.3. Samples: 99856. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:53:44,813][04690] Avg episode reward: [(0, '4.429')] [2024-10-18 22:53:49,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 393216. Throughput: 0: 224.8. Samples: 100830. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:53:49,808][04690] Avg episode reward: [(0, '4.391')] [2024-10-18 22:53:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 401408. Throughput: 0: 216.6. Samples: 101596. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:53:54,812][04690] Avg episode reward: [(0, '4.406')] [2024-10-18 22:53:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 405504. Throughput: 0: 231.1. Samples: 102978. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:53:59,809][04690] Avg episode reward: [(0, '4.455')] [2024-10-18 22:54:04,641][05478] Updated weights for policy 0, policy_version 100 (0.1152) [2024-10-18 22:54:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 409600. Throughput: 0: 218.5. Samples: 104022. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:54:04,813][04690] Avg episode reward: [(0, '4.514')] [2024-10-18 22:54:08,175][05464] Signal inference workers to stop experience collection... (100 times) [2024-10-18 22:54:08,215][05478] InferenceWorker_p0-w0: stopping experience collection (100 times) [2024-10-18 22:54:09,398][05464] Signal inference workers to resume experience collection... (100 times) [2024-10-18 22:54:09,400][05478] InferenceWorker_p0-w0: resuming experience collection (100 times) [2024-10-18 22:54:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 413696. Throughput: 0: 196.3. Samples: 104540. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:54:09,816][04690] Avg episode reward: [(0, '4.486')] [2024-10-18 22:54:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 417792. Throughput: 0: 220.8. Samples: 106074. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:54:14,807][04690] Avg episode reward: [(0, '4.470')] [2024-10-18 22:54:19,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 421888. Throughput: 0: 224.7. Samples: 107502. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:54:19,810][04690] Avg episode reward: [(0, '4.496')] [2024-10-18 22:54:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 425984. Throughput: 0: 203.3. Samples: 108116. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:54:24,811][04690] Avg episode reward: [(0, '4.509')] [2024-10-18 22:54:29,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 425984. Throughput: 0: 201.4. Samples: 108920. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:54:29,808][04690] Avg episode reward: [(0, '4.483')] [2024-10-18 22:54:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 434176. Throughput: 0: 208.8. Samples: 110226. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:54:34,807][04690] Avg episode reward: [(0, '4.532')] [2024-10-18 22:54:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 438272. Throughput: 0: 225.7. Samples: 111752. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:54:39,811][04690] Avg episode reward: [(0, '4.503')] [2024-10-18 22:54:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 442368. Throughput: 0: 205.8. Samples: 112240. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:54:44,813][04690] Avg episode reward: [(0, '4.510')] [2024-10-18 22:54:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 446464. Throughput: 0: 206.2. Samples: 113302. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:54:49,807][04690] Avg episode reward: [(0, '4.464')] [2024-10-18 22:54:52,959][05478] Updated weights for policy 0, policy_version 110 (0.1470) [2024-10-18 22:54:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 450560. Throughput: 0: 234.1. Samples: 115076. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:54:54,810][04690] Avg episode reward: [(0, '4.437')] [2024-10-18 22:54:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 454656. Throughput: 0: 215.7. Samples: 115782. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:54:59,816][04690] Avg episode reward: [(0, '4.516')] [2024-10-18 22:55:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 458752. Throughput: 0: 203.2. Samples: 116644. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:04,811][04690] Avg episode reward: [(0, '4.496')] [2024-10-18 22:55:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 462848. Throughput: 0: 229.1. Samples: 118426. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:09,814][04690] Avg episode reward: [(0, '4.613')] [2024-10-18 22:55:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 466944. Throughput: 0: 222.4. Samples: 118928. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:14,812][04690] Avg episode reward: [(0, '4.577')] [2024-10-18 22:55:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 471040. Throughput: 0: 227.5. Samples: 120464. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:19,808][04690] Avg episode reward: [(0, '4.587')] [2024-10-18 22:55:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 475136. Throughput: 0: 216.1. Samples: 121476. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:55:24,811][04690] Avg episode reward: [(0, '4.554')] [2024-10-18 22:55:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 483328. Throughput: 0: 223.3. Samples: 122290. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:29,814][04690] Avg episode reward: [(0, '4.522')] [2024-10-18 22:55:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 487424. Throughput: 0: 233.2. Samples: 123796. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:34,808][04690] Avg episode reward: [(0, '4.545')] [2024-10-18 22:55:38,755][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000120_491520.pth... [2024-10-18 22:55:38,770][05478] Updated weights for policy 0, policy_version 120 (0.2141) [2024-10-18 22:55:38,904][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000069_282624.pth [2024-10-18 22:55:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 491520. Throughput: 0: 210.1. Samples: 124532. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:39,808][04690] Avg episode reward: [(0, '4.499')] [2024-10-18 22:55:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 495616. Throughput: 0: 216.8. Samples: 125536. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:44,810][04690] Avg episode reward: [(0, '4.526')] [2024-10-18 22:55:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 499712. Throughput: 0: 229.6. Samples: 126974. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:49,808][04690] Avg episode reward: [(0, '4.590')] [2024-10-18 22:55:54,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 503808. Throughput: 0: 203.5. Samples: 127584. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:54,812][04690] Avg episode reward: [(0, '4.597')] [2024-10-18 22:55:59,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 507904. Throughput: 0: 218.4. Samples: 128756. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:55:59,811][04690] Avg episode reward: [(0, '4.686')] [2024-10-18 22:56:02,186][05464] Saving new best policy, reward=4.686! [2024-10-18 22:56:04,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 512000. Throughput: 0: 218.3. Samples: 130286. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:56:04,808][04690] Avg episode reward: [(0, '4.693')] [2024-10-18 22:56:09,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 516096. Throughput: 0: 225.6. Samples: 131630. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:56:09,812][04690] Avg episode reward: [(0, '4.736')] [2024-10-18 22:56:09,985][05464] Saving new best policy, reward=4.693! [2024-10-18 22:56:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 520192. Throughput: 0: 225.9. Samples: 132456. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 22:56:14,808][04690] Avg episode reward: [(0, '4.883')] [2024-10-18 22:56:15,508][05464] Saving new best policy, reward=4.736! [2024-10-18 22:56:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 524288. Throughput: 0: 213.6. Samples: 133408. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:56:19,815][04690] Avg episode reward: [(0, '4.968')] [2024-10-18 22:56:20,487][05464] Saving new best policy, reward=4.883! [2024-10-18 22:56:24,340][05464] Saving new best policy, reward=4.968! [2024-10-18 22:56:24,344][05478] Updated weights for policy 0, policy_version 130 (0.1022) [2024-10-18 22:56:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 532480. Throughput: 0: 213.3. Samples: 134132. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:56:24,810][04690] Avg episode reward: [(0, '4.995')] [2024-10-18 22:56:28,124][05464] Saving new best policy, reward=4.995! [2024-10-18 22:56:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 536576. Throughput: 0: 223.3. Samples: 135586. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:56:29,810][04690] Avg episode reward: [(0, '4.971')] [2024-10-18 22:56:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 540672. Throughput: 0: 214.6. Samples: 136632. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:56:34,810][04690] Avg episode reward: [(0, '4.997')] [2024-10-18 22:56:38,837][05464] Saving new best policy, reward=4.997! [2024-10-18 22:56:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 544768. Throughput: 0: 216.7. Samples: 137336. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:56:39,808][04690] Avg episode reward: [(0, '4.944')] [2024-10-18 22:56:44,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 548864. Throughput: 0: 221.7. Samples: 138734. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:56:44,811][04690] Avg episode reward: [(0, '4.888')] [2024-10-18 22:56:49,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 552960. Throughput: 0: 217.9. Samples: 140090. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:56:49,810][04690] Avg episode reward: [(0, '4.930')] [2024-10-18 22:56:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 557056. Throughput: 0: 201.4. Samples: 140692. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:56:54,812][04690] Avg episode reward: [(0, '4.999')] [2024-10-18 22:56:59,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 557056. Throughput: 0: 204.7. Samples: 141666. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:56:59,807][04690] Avg episode reward: [(0, '4.929')] [2024-10-18 22:57:00,494][05464] Saving new best policy, reward=4.999! [2024-10-18 22:57:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 565248. Throughput: 0: 207.8. Samples: 142760. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:04,809][04690] Avg episode reward: [(0, '4.793')] [2024-10-18 22:57:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 569344. Throughput: 0: 213.8. Samples: 143752. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:09,808][04690] Avg episode reward: [(0, '4.799')] [2024-10-18 22:57:12,870][05478] Updated weights for policy 0, policy_version 140 (0.0722) [2024-10-18 22:57:14,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 573440. Throughput: 0: 205.2. Samples: 144822. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:14,816][04690] Avg episode reward: [(0, '4.798')] [2024-10-18 22:57:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 577536. Throughput: 0: 207.8. Samples: 145982. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:57:19,813][04690] Avg episode reward: [(0, '4.885')] [2024-10-18 22:57:24,805][04690] Fps is (10 sec: 819.5, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 581632. Throughput: 0: 213.2. Samples: 146930. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:57:24,811][04690] Avg episode reward: [(0, '4.863')] [2024-10-18 22:57:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 585728. Throughput: 0: 219.5. Samples: 148612. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:57:29,810][04690] Avg episode reward: [(0, '4.865')] [2024-10-18 22:57:34,806][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 589824. Throughput: 0: 210.7. Samples: 149572. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:57:34,814][04690] Avg episode reward: [(0, '4.843')] [2024-10-18 22:57:36,801][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000145_593920.pth... [2024-10-18 22:57:36,910][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000095_389120.pth [2024-10-18 22:57:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 593920. Throughput: 0: 228.4. Samples: 150972. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:39,811][04690] Avg episode reward: [(0, '4.764')] [2024-10-18 22:57:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 602112. Throughput: 0: 227.6. Samples: 151910. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:44,809][04690] Avg episode reward: [(0, '4.788')] [2024-10-18 22:57:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 606208. Throughput: 0: 228.0. Samples: 153018. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:49,814][04690] Avg episode reward: [(0, '4.895')] [2024-10-18 22:57:54,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 606208. Throughput: 0: 228.8. Samples: 154048. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:57:54,809][04690] Avg episode reward: [(0, '4.943')] [2024-10-18 22:57:58,723][05478] Updated weights for policy 0, policy_version 150 (0.0530) [2024-10-18 22:57:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 614400. Throughput: 0: 226.9. Samples: 155034. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:57:59,810][04690] Avg episode reward: [(0, '5.005')] [2024-10-18 22:58:01,065][05464] Signal inference workers to stop experience collection... (150 times) [2024-10-18 22:58:01,099][05478] InferenceWorker_p0-w0: stopping experience collection (150 times) [2024-10-18 22:58:02,474][05464] Signal inference workers to resume experience collection... (150 times) [2024-10-18 22:58:02,476][05464] Saving new best policy, reward=5.005! [2024-10-18 22:58:02,476][05478] InferenceWorker_p0-w0: resuming experience collection (150 times) [2024-10-18 22:58:04,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 618496. Throughput: 0: 235.6. Samples: 156584. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:58:04,810][04690] Avg episode reward: [(0, '4.939')] [2024-10-18 22:58:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 622592. Throughput: 0: 227.0. Samples: 157146. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:58:09,812][04690] Avg episode reward: [(0, '4.882')] [2024-10-18 22:58:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 626688. Throughput: 0: 213.8. Samples: 158234. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:58:14,819][04690] Avg episode reward: [(0, '4.918')] [2024-10-18 22:58:19,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 630784. Throughput: 0: 226.4. Samples: 159762. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 22:58:19,814][04690] Avg episode reward: [(0, '5.062')] [2024-10-18 22:58:21,018][05464] Saving new best policy, reward=5.062! [2024-10-18 22:58:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 634880. Throughput: 0: 228.9. Samples: 161274. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:58:24,808][04690] Avg episode reward: [(0, '4.979')] [2024-10-18 22:58:29,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 638976. Throughput: 0: 214.9. Samples: 161582. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:58:29,812][04690] Avg episode reward: [(0, '4.989')] [2024-10-18 22:58:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 643072. Throughput: 0: 229.2. Samples: 163330. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:58:34,809][04690] Avg episode reward: [(0, '4.926')] [2024-10-18 22:58:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 651264. Throughput: 0: 220.0. Samples: 163948. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:58:39,813][04690] Avg episode reward: [(0, '4.988')] [2024-10-18 22:58:44,660][05478] Updated weights for policy 0, policy_version 160 (0.1030) [2024-10-18 22:58:44,806][04690] Fps is (10 sec: 1228.7, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 655360. Throughput: 0: 228.0. Samples: 165296. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:58:44,809][04690] Avg episode reward: [(0, '4.935')] [2024-10-18 22:58:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 659456. Throughput: 0: 218.9. Samples: 166434. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:58:49,812][04690] Avg episode reward: [(0, '4.928')] [2024-10-18 22:58:54,805][04690] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 663552. Throughput: 0: 236.5. Samples: 167790. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:58:54,812][04690] Avg episode reward: [(0, '4.969')] [2024-10-18 22:58:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 667648. Throughput: 0: 228.1. Samples: 168500. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:58:59,810][04690] Avg episode reward: [(0, '4.956')] [2024-10-18 22:59:04,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 671744. Throughput: 0: 221.0. Samples: 169708. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:59:04,811][04690] Avg episode reward: [(0, '4.928')] [2024-10-18 22:59:09,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 675840. Throughput: 0: 204.1. Samples: 170458. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:59:09,812][04690] Avg episode reward: [(0, '4.928')] [2024-10-18 22:59:14,806][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 679936. Throughput: 0: 225.6. Samples: 171734. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:59:14,808][04690] Avg episode reward: [(0, '4.885')] [2024-10-18 22:59:19,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 684032. Throughput: 0: 215.3. Samples: 173020. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:59:19,809][04690] Avg episode reward: [(0, '4.869')] [2024-10-18 22:59:24,807][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 688128. Throughput: 0: 215.1. Samples: 173626. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:59:24,816][04690] Avg episode reward: [(0, '4.755')] [2024-10-18 22:59:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 692224. Throughput: 0: 204.3. Samples: 174488. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:59:29,813][04690] Avg episode reward: [(0, '4.634')] [2024-10-18 22:59:33,453][05478] Updated weights for policy 0, policy_version 170 (0.2554) [2024-10-18 22:59:34,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 696320. Throughput: 0: 209.1. Samples: 175842. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:59:34,811][04690] Avg episode reward: [(0, '4.548')] [2024-10-18 22:59:37,378][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000171_700416.pth... [2024-10-18 22:59:37,484][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000120_491520.pth [2024-10-18 22:59:39,808][04690] Fps is (10 sec: 818.9, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 700416. Throughput: 0: 198.0. Samples: 176702. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:59:39,811][04690] Avg episode reward: [(0, '4.619')] [2024-10-18 22:59:44,811][04690] Fps is (10 sec: 818.8, 60 sec: 819.1, 300 sec: 874.7). Total num frames: 704512. Throughput: 0: 206.4. Samples: 177788. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 22:59:44,813][04690] Avg episode reward: [(0, '4.801')] [2024-10-18 22:59:49,805][04690] Fps is (10 sec: 819.5, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 708608. Throughput: 0: 209.5. Samples: 179136. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:59:49,807][04690] Avg episode reward: [(0, '4.926')] [2024-10-18 22:59:54,805][04690] Fps is (10 sec: 819.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 712704. Throughput: 0: 229.2. Samples: 180770. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 22:59:54,813][04690] Avg episode reward: [(0, '4.967')] [2024-10-18 22:59:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 716800. Throughput: 0: 214.7. Samples: 181394. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 22:59:59,808][04690] Avg episode reward: [(0, '5.013')] [2024-10-18 23:00:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 720896. Throughput: 0: 211.0. Samples: 182516. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:04,814][04690] Avg episode reward: [(0, '5.304')] [2024-10-18 23:00:06,083][05464] Saving new best policy, reward=5.304! [2024-10-18 23:00:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 724992. Throughput: 0: 227.6. Samples: 183868. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:09,808][04690] Avg episode reward: [(0, '5.364')] [2024-10-18 23:00:13,921][05464] Saving new best policy, reward=5.364! [2024-10-18 23:00:14,807][04690] Fps is (10 sec: 1228.6, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 733184. Throughput: 0: 231.8. Samples: 184920. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:14,830][04690] Avg episode reward: [(0, '5.517')] [2024-10-18 23:00:19,807][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 733184. Throughput: 0: 217.1. Samples: 185614. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:19,822][04690] Avg episode reward: [(0, '5.527')] [2024-10-18 23:00:24,705][05464] Saving new best policy, reward=5.517! [2024-10-18 23:00:24,691][05478] Updated weights for policy 0, policy_version 180 (0.0526) [2024-10-18 23:00:24,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 737280. Throughput: 0: 203.7. Samples: 185868. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:00:24,811][04690] Avg episode reward: [(0, '5.559')] [2024-10-18 23:00:24,919][05464] Saving new best policy, reward=5.527! [2024-10-18 23:00:29,720][05464] Saving new best policy, reward=5.559! [2024-10-18 23:00:29,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 741376. Throughput: 0: 197.1. Samples: 186658. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:00:29,808][04690] Avg episode reward: [(0, '5.737')] [2024-10-18 23:00:33,603][05464] Saving new best policy, reward=5.737! [2024-10-18 23:00:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 745472. Throughput: 0: 196.5. Samples: 187978. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:00:34,810][04690] Avg episode reward: [(0, '5.717')] [2024-10-18 23:00:39,808][04690] Fps is (10 sec: 818.9, 60 sec: 819.2, 300 sec: 860.8). Total num frames: 749568. Throughput: 0: 181.0. Samples: 188916. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:39,817][04690] Avg episode reward: [(0, '5.619')] [2024-10-18 23:00:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.3, 300 sec: 860.9). Total num frames: 753664. Throughput: 0: 191.4. Samples: 190008. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:44,812][04690] Avg episode reward: [(0, '5.764')] [2024-10-18 23:00:48,346][05464] Saving new best policy, reward=5.764! [2024-10-18 23:00:49,805][04690] Fps is (10 sec: 819.5, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 757760. Throughput: 0: 194.0. Samples: 191248. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:49,816][04690] Avg episode reward: [(0, '5.754')] [2024-10-18 23:00:54,806][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 761856. Throughput: 0: 204.2. Samples: 193056. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:00:54,815][04690] Avg episode reward: [(0, '5.590')] [2024-10-18 23:00:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 765952. Throughput: 0: 190.4. Samples: 193486. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:00:59,812][04690] Avg episode reward: [(0, '5.554')] [2024-10-18 23:01:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 770048. Throughput: 0: 195.4. Samples: 194408. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:01:04,808][04690] Avg episode reward: [(0, '5.554')] [2024-10-18 23:01:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 774144. Throughput: 0: 229.7. Samples: 196206. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:01:09,811][04690] Avg episode reward: [(0, '5.496')] [2024-10-18 23:01:10,581][05478] Updated weights for policy 0, policy_version 190 (0.1165) [2024-10-18 23:01:14,811][04690] Fps is (10 sec: 1228.0, 60 sec: 819.1, 300 sec: 874.7). Total num frames: 782336. Throughput: 0: 233.6. Samples: 197170. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:01:14,814][04690] Avg episode reward: [(0, '5.584')] [2024-10-18 23:01:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 782336. Throughput: 0: 226.9. Samples: 198188. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:01:19,808][04690] Avg episode reward: [(0, '5.567')] [2024-10-18 23:01:24,805][04690] Fps is (10 sec: 819.7, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 790528. Throughput: 0: 230.5. Samples: 199288. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:01:24,813][04690] Avg episode reward: [(0, '5.539')] [2024-10-18 23:01:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 794624. Throughput: 0: 228.3. Samples: 200282. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:01:29,808][04690] Avg episode reward: [(0, '5.496')] [2024-10-18 23:01:34,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 860.8). Total num frames: 798720. Throughput: 0: 230.2. Samples: 201606. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:01:34,813][04690] Avg episode reward: [(0, '5.255')] [2024-10-18 23:01:38,522][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000196_802816.pth... [2024-10-18 23:01:38,609][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000145_593920.pth [2024-10-18 23:01:39,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 802816. Throughput: 0: 212.8. Samples: 202634. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:01:39,811][04690] Avg episode reward: [(0, '5.249')] [2024-10-18 23:01:44,809][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 860.8). Total num frames: 806912. Throughput: 0: 219.6. Samples: 203370. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:01:44,816][04690] Avg episode reward: [(0, '5.545')] [2024-10-18 23:01:49,807][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 860.8). Total num frames: 811008. Throughput: 0: 228.3. Samples: 204680. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:01:49,810][04690] Avg episode reward: [(0, '5.486')] [2024-10-18 23:01:54,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 815104. Throughput: 0: 202.7. Samples: 205328. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:01:54,808][04690] Avg episode reward: [(0, '5.656')] [2024-10-18 23:01:59,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 815104. Throughput: 0: 203.4. Samples: 206322. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:01:59,813][04690] Avg episode reward: [(0, '5.687')] [2024-10-18 23:02:00,030][05478] Updated weights for policy 0, policy_version 200 (0.0050) [2024-10-18 23:02:02,369][05464] Signal inference workers to stop experience collection... (200 times) [2024-10-18 23:02:02,410][05478] InferenceWorker_p0-w0: stopping experience collection (200 times) [2024-10-18 23:02:03,790][05464] Signal inference workers to resume experience collection... (200 times) [2024-10-18 23:02:03,791][05478] InferenceWorker_p0-w0: resuming experience collection (200 times) [2024-10-18 23:02:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 823296. Throughput: 0: 206.2. Samples: 207466. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:02:04,811][04690] Avg episode reward: [(0, '5.749')] [2024-10-18 23:02:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 827392. Throughput: 0: 203.8. Samples: 208458. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:02:09,810][04690] Avg episode reward: [(0, '5.605')] [2024-10-18 23:02:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.3, 300 sec: 860.9). Total num frames: 831488. Throughput: 0: 205.4. Samples: 209524. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:02:14,807][04690] Avg episode reward: [(0, '5.723')] [2024-10-18 23:02:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 835584. Throughput: 0: 203.4. Samples: 210758. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:02:19,807][04690] Avg episode reward: [(0, '5.847')] [2024-10-18 23:02:21,766][05464] Saving new best policy, reward=5.847! [2024-10-18 23:02:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 839680. Throughput: 0: 219.8. Samples: 212526. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:02:24,810][04690] Avg episode reward: [(0, '5.714')] [2024-10-18 23:02:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 843776. Throughput: 0: 217.8. Samples: 213168. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:02:29,807][04690] Avg episode reward: [(0, '5.731')] [2024-10-18 23:02:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 847872. Throughput: 0: 212.9. Samples: 214260. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:02:34,814][04690] Avg episode reward: [(0, '5.718')] [2024-10-18 23:02:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 851968. Throughput: 0: 226.6. Samples: 215524. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:02:39,808][04690] Avg episode reward: [(0, '5.547')] [2024-10-18 23:02:43,841][05478] Updated weights for policy 0, policy_version 210 (0.1054) [2024-10-18 23:02:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 860160. Throughput: 0: 227.2. Samples: 216546. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:02:44,814][04690] Avg episode reward: [(0, '5.334')] [2024-10-18 23:02:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 864256. Throughput: 0: 224.2. Samples: 217556. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:02:49,815][04690] Avg episode reward: [(0, '5.342')] [2024-10-18 23:02:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 868352. Throughput: 0: 216.8. Samples: 218214. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:02:54,808][04690] Avg episode reward: [(0, '5.362')] [2024-10-18 23:02:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 860.9). Total num frames: 872448. Throughput: 0: 224.3. Samples: 219618. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:02:59,812][04690] Avg episode reward: [(0, '5.683')] [2024-10-18 23:03:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 876544. Throughput: 0: 234.2. Samples: 221296. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:03:04,808][04690] Avg episode reward: [(0, '5.570')] [2024-10-18 23:03:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 880640. Throughput: 0: 215.6. Samples: 222230. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:03:09,808][04690] Avg episode reward: [(0, '5.537')] [2024-10-18 23:03:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 884736. Throughput: 0: 211.6. Samples: 222688. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:03:14,808][04690] Avg episode reward: [(0, '5.588')] [2024-10-18 23:03:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 888832. Throughput: 0: 232.3. Samples: 224712. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:19,809][04690] Avg episode reward: [(0, '5.871')] [2024-10-18 23:03:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 892928. Throughput: 0: 226.1. Samples: 225700. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:24,808][04690] Avg episode reward: [(0, '5.814')] [2024-10-18 23:03:25,154][05464] Saving new best policy, reward=5.871! [2024-10-18 23:03:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 897024. Throughput: 0: 216.3. Samples: 226280. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:29,813][04690] Avg episode reward: [(0, '5.952')] [2024-10-18 23:03:30,481][05478] Updated weights for policy 0, policy_version 220 (0.1025) [2024-10-18 23:03:34,249][05464] Saving new best policy, reward=5.952! [2024-10-18 23:03:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 860.9). Total num frames: 905216. Throughput: 0: 227.2. Samples: 227780. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:34,808][04690] Avg episode reward: [(0, '5.820')] [2024-10-18 23:03:38,074][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000222_909312.pth... [2024-10-18 23:03:38,186][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000171_700416.pth [2024-10-18 23:03:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 860.9). Total num frames: 909312. Throughput: 0: 234.8. Samples: 228778. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:39,813][04690] Avg episode reward: [(0, '6.151')] [2024-10-18 23:03:43,582][05464] Saving new best policy, reward=6.151! [2024-10-18 23:03:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 913408. Throughput: 0: 226.6. Samples: 229814. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:44,818][04690] Avg episode reward: [(0, '5.997')] [2024-10-18 23:03:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 917504. Throughput: 0: 212.0. Samples: 230834. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:49,809][04690] Avg episode reward: [(0, '5.768')] [2024-10-18 23:03:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 921600. Throughput: 0: 213.1. Samples: 231818. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:54,808][04690] Avg episode reward: [(0, '5.690')] [2024-10-18 23:03:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 925696. Throughput: 0: 233.5. Samples: 233196. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:03:59,811][04690] Avg episode reward: [(0, '5.696')] [2024-10-18 23:04:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 929792. Throughput: 0: 214.9. Samples: 234384. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:04:04,811][04690] Avg episode reward: [(0, '5.679')] [2024-10-18 23:04:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 933888. Throughput: 0: 226.8. Samples: 235906. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:04:09,808][04690] Avg episode reward: [(0, '5.797')] [2024-10-18 23:04:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 937984. Throughput: 0: 230.5. Samples: 236652. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:04:14,811][04690] Avg episode reward: [(0, '5.834')] [2024-10-18 23:04:15,271][05478] Updated weights for policy 0, policy_version 230 (0.1022) [2024-10-18 23:04:19,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 860.9). Total num frames: 942080. Throughput: 0: 220.1. Samples: 237686. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:04:19,816][04690] Avg episode reward: [(0, '5.859')] [2024-10-18 23:04:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 946176. Throughput: 0: 204.5. Samples: 237982. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:04:24,812][04690] Avg episode reward: [(0, '5.983')] [2024-10-18 23:04:29,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 950272. Throughput: 0: 204.7. Samples: 239026. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:04:29,808][04690] Avg episode reward: [(0, '6.150')] [2024-10-18 23:04:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 954368. Throughput: 0: 219.2. Samples: 240698. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:04:34,811][04690] Avg episode reward: [(0, '5.862')] [2024-10-18 23:04:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 958464. Throughput: 0: 229.9. Samples: 242162. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:04:39,808][04690] Avg episode reward: [(0, '5.985')] [2024-10-18 23:04:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 962560. Throughput: 0: 208.9. Samples: 242596. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:04:44,809][04690] Avg episode reward: [(0, '5.936')] [2024-10-18 23:04:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 966656. Throughput: 0: 217.6. Samples: 244178. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:04:49,813][04690] Avg episode reward: [(0, '5.950')] [2024-10-18 23:04:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 974848. Throughput: 0: 199.0. Samples: 244860. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:04:54,808][04690] Avg episode reward: [(0, '5.993')] [2024-10-18 23:04:59,808][04690] Fps is (10 sec: 1228.4, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 978944. Throughput: 0: 211.7. Samples: 246180. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:04:59,815][04690] Avg episode reward: [(0, '6.075')] [2024-10-18 23:05:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 983040. Throughput: 0: 214.0. Samples: 247316. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:05:04,815][04690] Avg episode reward: [(0, '5.831')] [2024-10-18 23:05:04,954][05478] Updated weights for policy 0, policy_version 240 (0.1494) [2024-10-18 23:05:09,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 987136. Throughput: 0: 224.0. Samples: 248060. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:05:09,814][04690] Avg episode reward: [(0, '5.860')] [2024-10-18 23:05:14,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 991232. Throughput: 0: 235.3. Samples: 249614. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:05:14,815][04690] Avg episode reward: [(0, '5.625')] [2024-10-18 23:05:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 995328. Throughput: 0: 221.1. Samples: 250648. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:05:19,814][04690] Avg episode reward: [(0, '5.527')] [2024-10-18 23:05:24,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 999424. Throughput: 0: 204.4. Samples: 251362. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:05:24,808][04690] Avg episode reward: [(0, '5.482')] [2024-10-18 23:05:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1003520. Throughput: 0: 231.0. Samples: 252990. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:05:29,815][04690] Avg episode reward: [(0, '5.557')] [2024-10-18 23:05:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1007616. Throughput: 0: 228.0. Samples: 254438. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:05:34,809][04690] Avg episode reward: [(0, '5.481')] [2024-10-18 23:05:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1011712. Throughput: 0: 236.3. Samples: 255494. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:05:39,807][04690] Avg episode reward: [(0, '5.458')] [2024-10-18 23:05:40,588][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000248_1015808.pth... [2024-10-18 23:05:40,698][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000196_802816.pth [2024-10-18 23:05:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1019904. Throughput: 0: 227.7. Samples: 256424. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:05:44,808][04690] Avg episode reward: [(0, '5.686')] [2024-10-18 23:05:48,325][05478] Updated weights for policy 0, policy_version 250 (0.0057) [2024-10-18 23:05:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1024000. Throughput: 0: 230.2. Samples: 257676. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:05:49,810][04690] Avg episode reward: [(0, '5.852')] [2024-10-18 23:05:52,334][05464] Signal inference workers to stop experience collection... (250 times) [2024-10-18 23:05:52,442][05478] InferenceWorker_p0-w0: stopping experience collection (250 times) [2024-10-18 23:05:54,078][05464] Signal inference workers to resume experience collection... (250 times) [2024-10-18 23:05:54,080][05478] InferenceWorker_p0-w0: resuming experience collection (250 times) [2024-10-18 23:05:54,810][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 1028096. Throughput: 0: 229.7. Samples: 258396. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:05:54,814][04690] Avg episode reward: [(0, '5.774')] [2024-10-18 23:05:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1032192. Throughput: 0: 221.3. Samples: 259574. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:05:59,813][04690] Avg episode reward: [(0, '5.971')] [2024-10-18 23:06:04,805][04690] Fps is (10 sec: 819.6, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1036288. Throughput: 0: 233.2. Samples: 261142. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:04,808][04690] Avg episode reward: [(0, '5.733')] [2024-10-18 23:06:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 1040384. Throughput: 0: 231.6. Samples: 261782. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:09,810][04690] Avg episode reward: [(0, '5.845')] [2024-10-18 23:06:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1044480. Throughput: 0: 215.2. Samples: 262674. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:06:14,807][04690] Avg episode reward: [(0, '5.926')] [2024-10-18 23:06:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1048576. Throughput: 0: 224.6. Samples: 264546. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:06:19,812][04690] Avg episode reward: [(0, '6.090')] [2024-10-18 23:06:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1056768. Throughput: 0: 216.8. Samples: 265248. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:24,814][04690] Avg episode reward: [(0, '6.061')] [2024-10-18 23:06:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1056768. Throughput: 0: 226.2. Samples: 266604. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:29,812][04690] Avg episode reward: [(0, '6.115')] [2024-10-18 23:06:34,745][05478] Updated weights for policy 0, policy_version 260 (0.0638) [2024-10-18 23:06:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1064960. Throughput: 0: 224.6. Samples: 267782. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:34,807][04690] Avg episode reward: [(0, '6.228')] [2024-10-18 23:06:38,548][05464] Saving new best policy, reward=6.228! [2024-10-18 23:06:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1069056. Throughput: 0: 227.5. Samples: 268632. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:06:39,808][04690] Avg episode reward: [(0, '6.248')] [2024-10-18 23:06:43,953][05464] Saving new best policy, reward=6.248! [2024-10-18 23:06:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1073152. Throughput: 0: 228.5. Samples: 269858. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:06:44,808][04690] Avg episode reward: [(0, '6.425')] [2024-10-18 23:06:49,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1073152. Throughput: 0: 215.3. Samples: 270830. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:06:49,808][04690] Avg episode reward: [(0, '6.622')] [2024-10-18 23:06:50,899][05464] Saving new best policy, reward=6.425! [2024-10-18 23:06:54,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.3, 300 sec: 888.6). Total num frames: 1077248. Throughput: 0: 205.9. Samples: 271048. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:54,815][04690] Avg episode reward: [(0, '6.566')] [2024-10-18 23:06:56,202][05464] Saving new best policy, reward=6.622! [2024-10-18 23:06:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1081344. Throughput: 0: 218.7. Samples: 272516. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:06:59,812][04690] Avg episode reward: [(0, '6.596')] [2024-10-18 23:07:04,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1089536. Throughput: 0: 209.1. Samples: 273956. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:04,810][04690] Avg episode reward: [(0, '7.103')] [2024-10-18 23:07:09,215][05464] Saving new best policy, reward=7.103! [2024-10-18 23:07:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1093632. Throughput: 0: 212.1. Samples: 274792. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:09,810][04690] Avg episode reward: [(0, '7.292')] [2024-10-18 23:07:14,281][05464] Saving new best policy, reward=7.292! [2024-10-18 23:07:14,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1097728. Throughput: 0: 205.3. Samples: 275842. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:14,809][04690] Avg episode reward: [(0, '7.180')] [2024-10-18 23:07:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1101824. Throughput: 0: 211.3. Samples: 277292. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:19,808][04690] Avg episode reward: [(0, '7.194')] [2024-10-18 23:07:22,326][05478] Updated weights for policy 0, policy_version 270 (0.0538) [2024-10-18 23:07:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1105920. Throughput: 0: 208.8. Samples: 278028. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:24,808][04690] Avg episode reward: [(0, '7.102')] [2024-10-18 23:07:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1110016. Throughput: 0: 207.3. Samples: 279188. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:29,812][04690] Avg episode reward: [(0, '7.122')] [2024-10-18 23:07:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1114112. Throughput: 0: 219.0. Samples: 280686. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:34,810][04690] Avg episode reward: [(0, '6.615')] [2024-10-18 23:07:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1118208. Throughput: 0: 246.6. Samples: 282144. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:39,811][04690] Avg episode reward: [(0, '6.593')] [2024-10-18 23:07:39,829][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000274_1122304.pth... [2024-10-18 23:07:39,937][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000222_909312.pth [2024-10-18 23:07:44,807][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1122304. Throughput: 0: 233.5. Samples: 283024. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:44,810][04690] Avg episode reward: [(0, '6.637')] [2024-10-18 23:07:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1126400. Throughput: 0: 222.5. Samples: 283968. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:07:49,814][04690] Avg episode reward: [(0, '6.755')] [2024-10-18 23:07:54,805][04690] Fps is (10 sec: 1229.0, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1134592. Throughput: 0: 234.5. Samples: 285346. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:54,817][04690] Avg episode reward: [(0, '6.734')] [2024-10-18 23:07:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1138688. Throughput: 0: 232.1. Samples: 286288. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:07:59,814][04690] Avg episode reward: [(0, '6.665')] [2024-10-18 23:08:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1142784. Throughput: 0: 224.9. Samples: 287412. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:08:04,812][04690] Avg episode reward: [(0, '6.628')] [2024-10-18 23:08:08,359][05478] Updated weights for policy 0, policy_version 280 (0.0988) [2024-10-18 23:08:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1146880. Throughput: 0: 224.2. Samples: 288116. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:08:09,807][04690] Avg episode reward: [(0, '6.691')] [2024-10-18 23:08:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1150976. Throughput: 0: 231.7. Samples: 289616. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:08:14,808][04690] Avg episode reward: [(0, '7.050')] [2024-10-18 23:08:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1155072. Throughput: 0: 234.2. Samples: 291226. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:08:19,808][04690] Avg episode reward: [(0, '7.215')] [2024-10-18 23:08:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1159168. Throughput: 0: 223.3. Samples: 292194. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:08:24,808][04690] Avg episode reward: [(0, '7.207')] [2024-10-18 23:08:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1163264. Throughput: 0: 224.1. Samples: 293110. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:08:29,813][04690] Avg episode reward: [(0, '7.709')] [2024-10-18 23:08:33,572][05464] Saving new best policy, reward=7.709! [2024-10-18 23:08:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1171456. Throughput: 0: 234.1. Samples: 294502. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:08:34,810][04690] Avg episode reward: [(0, '7.613')] [2024-10-18 23:08:39,811][04690] Fps is (10 sec: 1228.0, 60 sec: 955.6, 300 sec: 888.6). Total num frames: 1175552. Throughput: 0: 222.1. Samples: 295342. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:08:39,814][04690] Avg episode reward: [(0, '7.714')] [2024-10-18 23:08:44,030][05464] Saving new best policy, reward=7.714! [2024-10-18 23:08:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.8, 300 sec: 888.6). Total num frames: 1179648. Throughput: 0: 224.1. Samples: 296374. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:08:44,815][04690] Avg episode reward: [(0, '7.481')] [2024-10-18 23:08:49,805][04690] Fps is (10 sec: 819.7, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1183744. Throughput: 0: 235.3. Samples: 298000. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:08:49,813][04690] Avg episode reward: [(0, '7.270')] [2024-10-18 23:08:51,843][05478] Updated weights for policy 0, policy_version 290 (0.1140) [2024-10-18 23:08:54,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 1187840. Throughput: 0: 232.8. Samples: 298592. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:08:54,812][04690] Avg episode reward: [(0, '7.233')] [2024-10-18 23:08:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1191936. Throughput: 0: 227.8. Samples: 299866. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:08:59,808][04690] Avg episode reward: [(0, '7.076')] [2024-10-18 23:09:04,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1196032. Throughput: 0: 224.3. Samples: 301320. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:09:04,808][04690] Avg episode reward: [(0, '6.785')] [2024-10-18 23:09:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1204224. Throughput: 0: 217.9. Samples: 302000. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:09,813][04690] Avg episode reward: [(0, '6.758')] [2024-10-18 23:09:14,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 1204224. Throughput: 0: 226.3. Samples: 303292. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:14,812][04690] Avg episode reward: [(0, '6.653')] [2024-10-18 23:09:19,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1208320. Throughput: 0: 213.0. Samples: 304086. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:19,808][04690] Avg episode reward: [(0, '6.738')] [2024-10-18 23:09:24,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1212416. Throughput: 0: 228.9. Samples: 305642. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:24,808][04690] Avg episode reward: [(0, '6.526')] [2024-10-18 23:09:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1216512. Throughput: 0: 217.6. Samples: 306164. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:29,807][04690] Avg episode reward: [(0, '6.474')] [2024-10-18 23:09:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1224704. Throughput: 0: 215.8. Samples: 307712. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:34,811][04690] Avg episode reward: [(0, '6.666')] [2024-10-18 23:09:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.3, 300 sec: 888.6). Total num frames: 1224704. Throughput: 0: 225.7. Samples: 308750. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:09:39,812][04690] Avg episode reward: [(0, '6.717')] [2024-10-18 23:09:40,195][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000300_1228800.pth... [2024-10-18 23:09:40,200][05478] Updated weights for policy 0, policy_version 300 (0.1450) [2024-10-18 23:09:40,306][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000248_1015808.pth [2024-10-18 23:09:42,574][05464] Signal inference workers to stop experience collection... (300 times) [2024-10-18 23:09:42,609][05478] InferenceWorker_p0-w0: stopping experience collection (300 times) [2024-10-18 23:09:44,152][05464] Signal inference workers to resume experience collection... (300 times) [2024-10-18 23:09:44,155][05478] InferenceWorker_p0-w0: resuming experience collection (300 times) [2024-10-18 23:09:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1232896. Throughput: 0: 220.5. Samples: 309788. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:09:44,808][04690] Avg episode reward: [(0, '6.988')] [2024-10-18 23:09:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1236992. Throughput: 0: 217.7. Samples: 311118. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:09:49,812][04690] Avg episode reward: [(0, '7.627')] [2024-10-18 23:09:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1241088. Throughput: 0: 218.6. Samples: 311838. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:09:54,808][04690] Avg episode reward: [(0, '7.689')] [2024-10-18 23:09:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1245184. Throughput: 0: 213.1. Samples: 312880. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:09:59,807][04690] Avg episode reward: [(0, '7.970')] [2024-10-18 23:10:02,125][05464] Saving new best policy, reward=7.970! [2024-10-18 23:10:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1249280. Throughput: 0: 233.6. Samples: 314596. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:04,815][04690] Avg episode reward: [(0, '8.457')] [2024-10-18 23:10:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1253376. Throughput: 0: 228.1. Samples: 315906. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:09,808][04690] Avg episode reward: [(0, '8.622')] [2024-10-18 23:10:10,978][05464] Saving new best policy, reward=8.457! [2024-10-18 23:10:14,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1257472. Throughput: 0: 226.3. Samples: 316346. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:14,811][04690] Avg episode reward: [(0, '8.829')] [2024-10-18 23:10:16,595][05464] Saving new best policy, reward=8.622! [2024-10-18 23:10:16,721][05464] Saving new best policy, reward=8.829! [2024-10-18 23:10:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1261568. Throughput: 0: 226.8. Samples: 317920. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:19,810][04690] Avg episode reward: [(0, '9.143')] [2024-10-18 23:10:24,291][05464] Saving new best policy, reward=9.143! [2024-10-18 23:10:24,296][05478] Updated weights for policy 0, policy_version 310 (0.1944) [2024-10-18 23:10:24,806][04690] Fps is (10 sec: 1229.1, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1269760. Throughput: 0: 213.0. Samples: 318334. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:24,825][04690] Avg episode reward: [(0, '9.447')] [2024-10-18 23:10:29,811][04690] Fps is (10 sec: 818.7, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 1269760. Throughput: 0: 218.2. Samples: 319610. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:29,842][04690] Avg episode reward: [(0, '9.404')] [2024-10-18 23:10:34,805][04690] Fps is (10 sec: 0.0, 60 sec: 750.9, 300 sec: 874.7). Total num frames: 1269760. Throughput: 0: 198.1. Samples: 320034. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:34,811][04690] Avg episode reward: [(0, '9.371')] [2024-10-18 23:10:34,884][05464] Saving new best policy, reward=9.447! [2024-10-18 23:10:39,805][04690] Fps is (10 sec: 819.7, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1277952. Throughput: 0: 193.3. Samples: 320538. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:39,807][04690] Avg episode reward: [(0, '9.590')] [2024-10-18 23:10:43,432][05464] Saving new best policy, reward=9.590! [2024-10-18 23:10:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1282048. Throughput: 0: 204.6. Samples: 322086. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:44,814][04690] Avg episode reward: [(0, '9.844')] [2024-10-18 23:10:47,711][05464] Saving new best policy, reward=9.844! [2024-10-18 23:10:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.8). Total num frames: 1286144. Throughput: 0: 194.1. Samples: 323330. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:10:49,812][04690] Avg episode reward: [(0, '9.808')] [2024-10-18 23:10:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1290240. Throughput: 0: 179.6. Samples: 323988. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:10:54,807][04690] Avg episode reward: [(0, '9.393')] [2024-10-18 23:10:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1294336. Throughput: 0: 196.0. Samples: 325166. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:10:59,808][04690] Avg episode reward: [(0, '9.397')] [2024-10-18 23:11:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1298432. Throughput: 0: 205.4. Samples: 327164. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:04,808][04690] Avg episode reward: [(0, '9.117')] [2024-10-18 23:11:09,809][04690] Fps is (10 sec: 818.9, 60 sec: 819.1, 300 sec: 874.7). Total num frames: 1302528. Throughput: 0: 203.2. Samples: 327478. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:09,812][04690] Avg episode reward: [(0, '9.167')] [2024-10-18 23:11:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1306624. Throughput: 0: 200.5. Samples: 328630. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:14,808][04690] Avg episode reward: [(0, '8.979')] [2024-10-18 23:11:15,563][05478] Updated weights for policy 0, policy_version 320 (0.0554) [2024-10-18 23:11:19,805][04690] Fps is (10 sec: 1229.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1314816. Throughput: 0: 226.5. Samples: 330226. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:19,808][04690] Avg episode reward: [(0, '9.110')] [2024-10-18 23:11:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1318912. Throughput: 0: 235.2. Samples: 331124. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:24,810][04690] Avg episode reward: [(0, '9.098')] [2024-10-18 23:11:29,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1323008. Throughput: 0: 227.8. Samples: 332338. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:29,813][04690] Avg episode reward: [(0, '8.693')] [2024-10-18 23:11:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 1327104. Throughput: 0: 227.1. Samples: 333548. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:11:34,812][04690] Avg episode reward: [(0, '8.868')] [2024-10-18 23:11:38,807][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000325_1331200.pth... [2024-10-18 23:11:38,936][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000274_1122304.pth [2024-10-18 23:11:39,809][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 1331200. Throughput: 0: 228.4. Samples: 334268. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:11:39,815][04690] Avg episode reward: [(0, '8.847')] [2024-10-18 23:11:44,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1331200. Throughput: 0: 222.9. Samples: 335198. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:11:44,808][04690] Avg episode reward: [(0, '8.671')] [2024-10-18 23:11:49,805][04690] Fps is (10 sec: 409.8, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1335296. Throughput: 0: 201.1. Samples: 336214. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:11:49,812][04690] Avg episode reward: [(0, '8.427')] [2024-10-18 23:11:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1343488. Throughput: 0: 226.1. Samples: 337650. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:11:54,812][04690] Avg episode reward: [(0, '8.761')] [2024-10-18 23:11:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1347584. Throughput: 0: 218.0. Samples: 338438. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:11:59,808][04690] Avg episode reward: [(0, '9.296')] [2024-10-18 23:12:03,923][05478] Updated weights for policy 0, policy_version 330 (0.1186) [2024-10-18 23:12:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1351680. Throughput: 0: 206.2. Samples: 339506. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:04,808][04690] Avg episode reward: [(0, '9.184')] [2024-10-18 23:12:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1355776. Throughput: 0: 200.6. Samples: 340150. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:09,821][04690] Avg episode reward: [(0, '9.222')] [2024-10-18 23:12:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1359872. Throughput: 0: 210.9. Samples: 341830. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:14,808][04690] Avg episode reward: [(0, '9.453')] [2024-10-18 23:12:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1363968. Throughput: 0: 218.3. Samples: 343370. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:19,808][04690] Avg episode reward: [(0, '9.522')] [2024-10-18 23:12:24,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1368064. Throughput: 0: 209.2. Samples: 343680. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:24,812][04690] Avg episode reward: [(0, '9.193')] [2024-10-18 23:12:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1372160. Throughput: 0: 223.1. Samples: 345238. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:12:29,810][04690] Avg episode reward: [(0, '9.431')] [2024-10-18 23:12:34,805][04690] Fps is (10 sec: 1228.9, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1380352. Throughput: 0: 231.6. Samples: 346636. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:34,808][04690] Avg episode reward: [(0, '9.757')] [2024-10-18 23:12:39,810][04690] Fps is (10 sec: 1228.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1384448. Throughput: 0: 219.1. Samples: 347512. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:39,827][04690] Avg episode reward: [(0, '10.186')] [2024-10-18 23:12:44,207][05464] Saving new best policy, reward=10.186! [2024-10-18 23:12:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1388544. Throughput: 0: 224.0. Samples: 348518. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:44,808][04690] Avg episode reward: [(0, '10.225')] [2024-10-18 23:12:48,146][05464] Saving new best policy, reward=10.225! [2024-10-18 23:12:48,151][05478] Updated weights for policy 0, policy_version 340 (0.0043) [2024-10-18 23:12:49,805][04690] Fps is (10 sec: 819.6, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 1392640. Throughput: 0: 233.0. Samples: 349990. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:49,807][04690] Avg episode reward: [(0, '10.259')] [2024-10-18 23:12:52,027][05464] Saving new best policy, reward=10.259! [2024-10-18 23:12:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1396736. Throughput: 0: 235.2. Samples: 350736. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:54,813][04690] Avg episode reward: [(0, '10.691')] [2024-10-18 23:12:57,597][05464] Saving new best policy, reward=10.691! [2024-10-18 23:12:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1400832. Throughput: 0: 221.2. Samples: 351786. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:12:59,812][04690] Avg episode reward: [(0, '11.076')] [2024-10-18 23:13:02,596][05464] Saving new best policy, reward=11.076! [2024-10-18 23:13:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1404928. Throughput: 0: 220.0. Samples: 353270. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:04,807][04690] Avg episode reward: [(0, '10.899')] [2024-10-18 23:13:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1409024. Throughput: 0: 247.4. Samples: 354814. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:09,817][04690] Avg episode reward: [(0, '10.591')] [2024-10-18 23:13:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1413120. Throughput: 0: 228.4. Samples: 355518. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:14,810][04690] Avg episode reward: [(0, '10.759')] [2024-10-18 23:13:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1417216. Throughput: 0: 220.0. Samples: 356534. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:19,813][04690] Avg episode reward: [(0, '10.817')] [2024-10-18 23:13:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.8, 300 sec: 888.6). Total num frames: 1425408. Throughput: 0: 218.2. Samples: 357328. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:24,807][04690] Avg episode reward: [(0, '10.861')] [2024-10-18 23:13:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 1429504. Throughput: 0: 231.6. Samples: 358940. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:29,808][04690] Avg episode reward: [(0, '10.927')] [2024-10-18 23:13:33,797][05478] Updated weights for policy 0, policy_version 350 (0.0558) [2024-10-18 23:13:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 1433600. Throughput: 0: 221.0. Samples: 359936. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:34,808][04690] Avg episode reward: [(0, '10.989')] [2024-10-18 23:13:37,425][05464] Signal inference workers to stop experience collection... (350 times) [2024-10-18 23:13:37,490][05478] InferenceWorker_p0-w0: stopping experience collection (350 times) [2024-10-18 23:13:38,567][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000351_1437696.pth... [2024-10-18 23:13:38,569][05464] Signal inference workers to resume experience collection... (350 times) [2024-10-18 23:13:38,576][05478] InferenceWorker_p0-w0: resuming experience collection (350 times) [2024-10-18 23:13:38,681][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000300_1228800.pth [2024-10-18 23:13:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1437696. Throughput: 0: 220.5. Samples: 360658. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:39,807][04690] Avg episode reward: [(0, '10.938')] [2024-10-18 23:13:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1441792. Throughput: 0: 229.8. Samples: 362126. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:44,809][04690] Avg episode reward: [(0, '10.800')] [2024-10-18 23:13:49,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1445888. Throughput: 0: 231.1. Samples: 363670. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:49,812][04690] Avg episode reward: [(0, '10.754')] [2024-10-18 23:13:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1449984. Throughput: 0: 204.9. Samples: 364034. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:13:54,813][04690] Avg episode reward: [(0, '10.809')] [2024-10-18 23:13:59,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1454080. Throughput: 0: 220.3. Samples: 365430. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:13:59,815][04690] Avg episode reward: [(0, '10.951')] [2024-10-18 23:14:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 1458176. Throughput: 0: 219.4. Samples: 366408. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:14:04,810][04690] Avg episode reward: [(0, '11.067')] [2024-10-18 23:14:09,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1462272. Throughput: 0: 216.8. Samples: 367084. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:09,810][04690] Avg episode reward: [(0, '10.895')] [2024-10-18 23:14:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1466368. Throughput: 0: 202.4. Samples: 368048. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:14,811][04690] Avg episode reward: [(0, '10.577')] [2024-10-18 23:14:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1470464. Throughput: 0: 217.2. Samples: 369712. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:19,808][04690] Avg episode reward: [(0, '11.048')] [2024-10-18 23:14:21,768][05478] Updated weights for policy 0, policy_version 360 (0.1189) [2024-10-18 23:14:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1474560. Throughput: 0: 212.5. Samples: 370222. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:24,808][04690] Avg episode reward: [(0, '11.409')] [2024-10-18 23:14:26,477][05464] Saving new best policy, reward=11.409! [2024-10-18 23:14:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 1478656. Throughput: 0: 209.4. Samples: 371548. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:14:29,810][04690] Avg episode reward: [(0, '11.497')] [2024-10-18 23:14:31,803][05464] Saving new best policy, reward=11.497! [2024-10-18 23:14:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1482752. Throughput: 0: 209.3. Samples: 373090. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:14:34,813][04690] Avg episode reward: [(0, '11.366')] [2024-10-18 23:14:39,811][04690] Fps is (10 sec: 1228.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 1490944. Throughput: 0: 228.6. Samples: 374324. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:39,814][04690] Avg episode reward: [(0, '11.387')] [2024-10-18 23:14:44,813][04690] Fps is (10 sec: 1227.8, 60 sec: 887.3, 300 sec: 874.7). Total num frames: 1495040. Throughput: 0: 216.2. Samples: 375160. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:44,821][04690] Avg episode reward: [(0, '11.747')] [2024-10-18 23:14:49,805][04690] Fps is (10 sec: 409.8, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 1495040. Throughput: 0: 220.5. Samples: 376332. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:49,816][04690] Avg episode reward: [(0, '11.765')] [2024-10-18 23:14:49,887][05464] Saving new best policy, reward=11.747! [2024-10-18 23:14:53,713][05464] Saving new best policy, reward=11.765! [2024-10-18 23:14:54,805][04690] Fps is (10 sec: 819.9, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1503232. Throughput: 0: 224.4. Samples: 377184. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:54,817][04690] Avg episode reward: [(0, '11.816')] [2024-10-18 23:14:57,504][05464] Saving new best policy, reward=11.816! [2024-10-18 23:14:59,809][04690] Fps is (10 sec: 1228.3, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 1507328. Throughput: 0: 233.5. Samples: 378558. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:14:59,814][04690] Avg episode reward: [(0, '11.621')] [2024-10-18 23:15:04,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1511424. Throughput: 0: 221.4. Samples: 379674. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:04,817][04690] Avg episode reward: [(0, '11.391')] [2024-10-18 23:15:07,903][05478] Updated weights for policy 0, policy_version 370 (0.0534) [2024-10-18 23:15:09,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1515520. Throughput: 0: 246.0. Samples: 381294. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:09,812][04690] Avg episode reward: [(0, '11.650')] [2024-10-18 23:15:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 1519616. Throughput: 0: 231.9. Samples: 381984. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:14,814][04690] Avg episode reward: [(0, '11.563')] [2024-10-18 23:15:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 1523712. Throughput: 0: 227.4. Samples: 383324. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:19,808][04690] Avg episode reward: [(0, '11.605')] [2024-10-18 23:15:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 1527808. Throughput: 0: 210.2. Samples: 383782. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:24,812][04690] Avg episode reward: [(0, '11.935')] [2024-10-18 23:15:29,514][05464] Saving new best policy, reward=11.935! [2024-10-18 23:15:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1536000. Throughput: 0: 226.8. Samples: 385364. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:29,809][04690] Avg episode reward: [(0, '11.568')] [2024-10-18 23:15:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1540096. Throughput: 0: 227.6. Samples: 386576. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:34,810][04690] Avg episode reward: [(0, '11.223')] [2024-10-18 23:15:38,536][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000377_1544192.pth... [2024-10-18 23:15:38,621][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000325_1331200.pth [2024-10-18 23:15:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.6, 300 sec: 888.6). Total num frames: 1544192. Throughput: 0: 228.0. Samples: 387442. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:39,808][04690] Avg episode reward: [(0, '10.985')] [2024-10-18 23:15:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.6, 300 sec: 888.6). Total num frames: 1548288. Throughput: 0: 222.0. Samples: 388546. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:44,808][04690] Avg episode reward: [(0, '10.762')] [2024-10-18 23:15:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1552384. Throughput: 0: 231.1. Samples: 390072. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:49,814][04690] Avg episode reward: [(0, '10.919')] [2024-10-18 23:15:51,940][05478] Updated weights for policy 0, policy_version 380 (0.1502) [2024-10-18 23:15:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1556480. Throughput: 0: 208.6. Samples: 390682. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:15:54,809][04690] Avg episode reward: [(0, '10.604')] [2024-10-18 23:15:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1560576. Throughput: 0: 221.8. Samples: 391966. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:15:59,811][04690] Avg episode reward: [(0, '10.951')] [2024-10-18 23:16:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1564672. Throughput: 0: 225.7. Samples: 393482. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:16:04,808][04690] Avg episode reward: [(0, '11.410')] [2024-10-18 23:16:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1572864. Throughput: 0: 246.8. Samples: 394886. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:16:09,808][04690] Avg episode reward: [(0, '11.063')] [2024-10-18 23:16:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1576960. Throughput: 0: 230.0. Samples: 395716. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:16:14,813][04690] Avg episode reward: [(0, '11.086')] [2024-10-18 23:16:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1581056. Throughput: 0: 226.0. Samples: 396748. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:16:19,808][04690] Avg episode reward: [(0, '10.889')] [2024-10-18 23:16:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 1585152. Throughput: 0: 242.2. Samples: 398340. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:16:24,812][04690] Avg episode reward: [(0, '10.981')] [2024-10-18 23:16:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1589248. Throughput: 0: 233.7. Samples: 399062. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:16:29,812][04690] Avg episode reward: [(0, '10.797')] [2024-10-18 23:16:34,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1593344. Throughput: 0: 226.5. Samples: 400266. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:16:34,816][04690] Avg episode reward: [(0, '10.746')] [2024-10-18 23:16:38,082][05478] Updated weights for policy 0, policy_version 390 (0.0535) [2024-10-18 23:16:39,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 902.5). Total num frames: 1597440. Throughput: 0: 227.6. Samples: 400926. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:16:39,816][04690] Avg episode reward: [(0, '10.783')] [2024-10-18 23:16:44,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1601536. Throughput: 0: 233.9. Samples: 402490. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:16:44,807][04690] Avg episode reward: [(0, '10.888')] [2024-10-18 23:16:49,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1605632. Throughput: 0: 231.4. Samples: 403896. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:16:49,808][04690] Avg episode reward: [(0, '10.983')] [2024-10-18 23:16:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1609728. Throughput: 0: 222.8. Samples: 404910. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:16:54,808][04690] Avg episode reward: [(0, '11.215')] [2024-10-18 23:16:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1617920. Throughput: 0: 226.9. Samples: 405926. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:16:59,814][04690] Avg episode reward: [(0, '11.430')] [2024-10-18 23:17:04,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1622016. Throughput: 0: 234.9. Samples: 407320. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:17:04,812][04690] Avg episode reward: [(0, '11.280')] [2024-10-18 23:17:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1626112. Throughput: 0: 215.0. Samples: 408016. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:17:09,813][04690] Avg episode reward: [(0, '12.001')] [2024-10-18 23:17:13,343][05464] Saving new best policy, reward=12.001! [2024-10-18 23:17:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1630208. Throughput: 0: 221.9. Samples: 409048. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:17:14,808][04690] Avg episode reward: [(0, '11.942')] [2024-10-18 23:17:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1634304. Throughput: 0: 231.8. Samples: 410696. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:17:19,815][04690] Avg episode reward: [(0, '12.063')] [2024-10-18 23:17:20,967][05478] Updated weights for policy 0, policy_version 400 (0.0515) [2024-10-18 23:17:24,300][05464] Signal inference workers to stop experience collection... (400 times) [2024-10-18 23:17:24,415][05478] InferenceWorker_p0-w0: stopping experience collection (400 times) [2024-10-18 23:17:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1638400. Throughput: 0: 249.3. Samples: 412142. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:17:24,808][04690] Avg episode reward: [(0, '12.070')] [2024-10-18 23:17:26,270][05464] Signal inference workers to resume experience collection... (400 times) [2024-10-18 23:17:26,272][05478] InferenceWorker_p0-w0: resuming experience collection (400 times) [2024-10-18 23:17:26,278][05464] Saving new best policy, reward=12.063! [2024-10-18 23:17:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1642496. Throughput: 0: 223.0. Samples: 412524. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:17:29,813][04690] Avg episode reward: [(0, '12.233')] [2024-10-18 23:17:31,428][05464] Saving new best policy, reward=12.070! [2024-10-18 23:17:31,563][05464] Saving new best policy, reward=12.233! [2024-10-18 23:17:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1646592. Throughput: 0: 228.2. Samples: 414166. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:17:34,815][04690] Avg episode reward: [(0, '13.171')] [2024-10-18 23:17:39,098][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000404_1654784.pth... [2024-10-18 23:17:39,206][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000351_1437696.pth [2024-10-18 23:17:39,224][05464] Saving new best policy, reward=13.171! [2024-10-18 23:17:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.8, 300 sec: 902.5). Total num frames: 1654784. Throughput: 0: 219.8. Samples: 414802. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:17:39,807][04690] Avg episode reward: [(0, '13.009')] [2024-10-18 23:17:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1658880. Throughput: 0: 229.1. Samples: 416236. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:17:44,810][04690] Avg episode reward: [(0, '13.216')] [2024-10-18 23:17:49,625][05464] Saving new best policy, reward=13.216! [2024-10-18 23:17:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1662976. Throughput: 0: 220.7. Samples: 417250. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:17:49,808][04690] Avg episode reward: [(0, '13.133')] [2024-10-18 23:17:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1667072. Throughput: 0: 225.6. Samples: 418166. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:17:54,809][04690] Avg episode reward: [(0, '13.006')] [2024-10-18 23:17:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1671168. Throughput: 0: 229.7. Samples: 419386. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:17:59,809][04690] Avg episode reward: [(0, '13.362')] [2024-10-18 23:18:03,119][05464] Saving new best policy, reward=13.362! [2024-10-18 23:18:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1675264. Throughput: 0: 218.4. Samples: 420526. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:18:04,808][04690] Avg episode reward: [(0, '13.585')] [2024-10-18 23:18:07,862][05464] Saving new best policy, reward=13.585! [2024-10-18 23:18:07,868][05478] Updated weights for policy 0, policy_version 410 (0.1108) [2024-10-18 23:18:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1679360. Throughput: 0: 223.0. Samples: 422176. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:18:09,808][04690] Avg episode reward: [(0, '13.953')] [2024-10-18 23:18:11,649][05464] Saving new best policy, reward=13.953! [2024-10-18 23:18:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1683456. Throughput: 0: 228.8. Samples: 422818. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:18:14,810][04690] Avg episode reward: [(0, '14.139')] [2024-10-18 23:18:19,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1687552. Throughput: 0: 222.6. Samples: 424182. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:18:19,815][04690] Avg episode reward: [(0, '14.276')] [2024-10-18 23:18:21,246][05464] Saving new best policy, reward=14.139! [2024-10-18 23:18:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1691648. Throughput: 0: 234.3. Samples: 425346. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:24,812][04690] Avg episode reward: [(0, '14.406')] [2024-10-18 23:18:26,158][05464] Saving new best policy, reward=14.276! [2024-10-18 23:18:26,302][05464] Saving new best policy, reward=14.406! [2024-10-18 23:18:29,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1695744. Throughput: 0: 219.8. Samples: 426126. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:29,813][04690] Avg episode reward: [(0, '15.035')] [2024-10-18 23:18:33,717][05464] Saving new best policy, reward=15.035! [2024-10-18 23:18:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 1703936. Throughput: 0: 227.6. Samples: 427494. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:18:34,808][04690] Avg episode reward: [(0, '15.091')] [2024-10-18 23:18:39,701][05464] Saving new best policy, reward=15.091! [2024-10-18 23:18:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1708032. Throughput: 0: 223.6. Samples: 428230. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:39,808][04690] Avg episode reward: [(0, '14.400')] [2024-10-18 23:18:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1712128. Throughput: 0: 222.2. Samples: 429386. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:44,808][04690] Avg episode reward: [(0, '15.239')] [2024-10-18 23:18:49,807][05464] Saving new best policy, reward=15.239! [2024-10-18 23:18:49,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1712128. Throughput: 0: 224.0. Samples: 430604. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:49,811][04690] Avg episode reward: [(0, '15.249')] [2024-10-18 23:18:54,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1716224. Throughput: 0: 209.8. Samples: 431616. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:54,819][04690] Avg episode reward: [(0, '15.607')] [2024-10-18 23:18:56,288][05464] Saving new best policy, reward=15.249! [2024-10-18 23:18:56,290][05478] Updated weights for policy 0, policy_version 420 (0.0577) [2024-10-18 23:18:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1720320. Throughput: 0: 204.1. Samples: 432004. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:18:59,809][04690] Avg episode reward: [(0, '15.389')] [2024-10-18 23:19:01,839][05464] Saving new best policy, reward=15.607! [2024-10-18 23:19:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1724416. Throughput: 0: 205.4. Samples: 433426. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:04,814][04690] Avg episode reward: [(0, '15.136')] [2024-10-18 23:19:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1732608. Throughput: 0: 191.0. Samples: 433942. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:09,811][04690] Avg episode reward: [(0, '14.960')] [2024-10-18 23:19:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1736704. Throughput: 0: 213.4. Samples: 435728. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:14,811][04690] Avg episode reward: [(0, '15.349')] [2024-10-18 23:19:19,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1736704. Throughput: 0: 204.5. Samples: 436698. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:19,814][04690] Avg episode reward: [(0, '15.629')] [2024-10-18 23:19:24,011][05464] Saving new best policy, reward=15.629! [2024-10-18 23:19:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 1744896. Throughput: 0: 220.9. Samples: 438172. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:24,807][04690] Avg episode reward: [(0, '15.779')] [2024-10-18 23:19:27,764][05464] Saving new best policy, reward=15.779! [2024-10-18 23:19:29,812][04690] Fps is (10 sec: 1227.9, 60 sec: 887.4, 300 sec: 902.5). Total num frames: 1748992. Throughput: 0: 209.5. Samples: 438816. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:19:29,822][04690] Avg episode reward: [(0, '15.391')] [2024-10-18 23:19:34,807][04690] Fps is (10 sec: 819.0, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1753088. Throughput: 0: 208.7. Samples: 439998. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:19:34,809][04690] Avg episode reward: [(0, '15.221')] [2024-10-18 23:19:38,266][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000429_1757184.pth... [2024-10-18 23:19:38,369][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000377_1544192.pth [2024-10-18 23:19:39,805][04690] Fps is (10 sec: 819.8, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1757184. Throughput: 0: 201.9. Samples: 440702. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:39,807][04690] Avg episode reward: [(0, '14.849')] [2024-10-18 23:19:42,609][05478] Updated weights for policy 0, policy_version 430 (0.1005) [2024-10-18 23:19:44,805][04690] Fps is (10 sec: 819.4, 60 sec: 819.2, 300 sec: 902.5). Total num frames: 1761280. Throughput: 0: 222.4. Samples: 442014. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:44,816][04690] Avg episode reward: [(0, '14.326')] [2024-10-18 23:19:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1765376. Throughput: 0: 225.1. Samples: 443554. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:19:49,810][04690] Avg episode reward: [(0, '15.101')] [2024-10-18 23:19:54,809][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 1769472. Throughput: 0: 224.6. Samples: 444048. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:19:54,815][04690] Avg episode reward: [(0, '14.440')] [2024-10-18 23:19:59,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1773568. Throughput: 0: 211.2. Samples: 445230. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:19:59,813][04690] Avg episode reward: [(0, '14.483')] [2024-10-18 23:20:04,806][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1777664. Throughput: 0: 207.1. Samples: 446018. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:20:04,811][04690] Avg episode reward: [(0, '14.489')] [2024-10-18 23:20:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 1781760. Throughput: 0: 188.2. Samples: 446642. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:20:09,812][04690] Avg episode reward: [(0, '14.458')] [2024-10-18 23:20:14,805][04690] Fps is (10 sec: 409.6, 60 sec: 750.9, 300 sec: 874.7). Total num frames: 1781760. Throughput: 0: 198.4. Samples: 447742. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:20:14,814][04690] Avg episode reward: [(0, '14.606')] [2024-10-18 23:20:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 1789952. Throughput: 0: 200.0. Samples: 448998. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:20:19,809][04690] Avg episode reward: [(0, '13.836')] [2024-10-18 23:20:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1794048. Throughput: 0: 205.6. Samples: 449954. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:20:24,811][04690] Avg episode reward: [(0, '13.603')] [2024-10-18 23:20:29,809][04690] Fps is (10 sec: 818.9, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1798144. Throughput: 0: 199.8. Samples: 451006. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:20:29,812][04690] Avg episode reward: [(0, '13.923')] [2024-10-18 23:20:32,955][05478] Updated weights for policy 0, policy_version 440 (0.2508) [2024-10-18 23:20:34,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1802240. Throughput: 0: 196.6. Samples: 452400. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:20:34,825][04690] Avg episode reward: [(0, '14.008')] [2024-10-18 23:20:39,805][04690] Fps is (10 sec: 819.5, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 1806336. Throughput: 0: 203.0. Samples: 453180. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:20:39,809][04690] Avg episode reward: [(0, '13.926')] [2024-10-18 23:20:44,807][04690] Fps is (10 sec: 409.6, 60 sec: 750.9, 300 sec: 860.8). Total num frames: 1806336. Throughput: 0: 188.9. Samples: 453730. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:20:44,813][04690] Avg episode reward: [(0, '14.238')] [2024-10-18 23:20:49,805][04690] Fps is (10 sec: 409.6, 60 sec: 750.9, 300 sec: 860.9). Total num frames: 1810432. Throughput: 0: 191.5. Samples: 454634. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:20:49,811][04690] Avg episode reward: [(0, '14.746')] [2024-10-18 23:20:54,805][04690] Fps is (10 sec: 819.3, 60 sec: 751.0, 300 sec: 860.9). Total num frames: 1814528. Throughput: 0: 210.9. Samples: 456132. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:20:54,808][04690] Avg episode reward: [(0, '14.640')] [2024-10-18 23:20:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 860.9). Total num frames: 1818624. Throughput: 0: 197.6. Samples: 456634. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:20:59,808][04690] Avg episode reward: [(0, '14.465')] [2024-10-18 23:21:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 847.0). Total num frames: 1822720. Throughput: 0: 203.6. Samples: 458160. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:21:04,808][04690] Avg episode reward: [(0, '14.481')] [2024-10-18 23:21:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 847.0). Total num frames: 1826816. Throughput: 0: 204.0. Samples: 459134. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:21:09,808][04690] Avg episode reward: [(0, '14.574')] [2024-10-18 23:21:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 1830912. Throughput: 0: 196.6. Samples: 459854. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:21:14,812][04690] Avg episode reward: [(0, '14.594')] [2024-10-18 23:21:19,806][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 847.0). Total num frames: 1835008. Throughput: 0: 186.9. Samples: 460812. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:21:19,816][04690] Avg episode reward: [(0, '14.505')] [2024-10-18 23:21:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 847.0). Total num frames: 1839104. Throughput: 0: 179.9. Samples: 461276. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:21:24,808][04690] Avg episode reward: [(0, '15.023')] [2024-10-18 23:21:28,182][05478] Updated weights for policy 0, policy_version 450 (0.1300) [2024-10-18 23:21:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 751.0, 300 sec: 847.0). Total num frames: 1843200. Throughput: 0: 191.7. Samples: 462356. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:21:29,809][04690] Avg episode reward: [(0, '15.278')] [2024-10-18 23:21:30,578][05464] Signal inference workers to stop experience collection... (450 times) [2024-10-18 23:21:30,661][05478] InferenceWorker_p0-w0: stopping experience collection (450 times) [2024-10-18 23:21:32,081][05464] Signal inference workers to resume experience collection... (450 times) [2024-10-18 23:21:32,082][05478] InferenceWorker_p0-w0: resuming experience collection (450 times) [2024-10-18 23:21:34,806][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 847.0). Total num frames: 1847296. Throughput: 0: 211.2. Samples: 464140. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:21:34,817][04690] Avg episode reward: [(0, '14.697')] [2024-10-18 23:21:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 847.0). Total num frames: 1851392. Throughput: 0: 206.7. Samples: 465434. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:21:39,813][04690] Avg episode reward: [(0, '15.395')] [2024-10-18 23:21:40,980][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000453_1855488.pth... [2024-10-18 23:21:41,115][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000404_1654784.pth [2024-10-18 23:21:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 1855488. Throughput: 0: 204.2. Samples: 465824. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:21:44,808][04690] Avg episode reward: [(0, '15.546')] [2024-10-18 23:21:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 1859584. Throughput: 0: 201.2. Samples: 467214. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:21:49,813][04690] Avg episode reward: [(0, '15.533')] [2024-10-18 23:21:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1867776. Throughput: 0: 197.0. Samples: 467998. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:21:54,818][04690] Avg episode reward: [(0, '15.463')] [2024-10-18 23:21:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 833.1). Total num frames: 1867776. Throughput: 0: 208.3. Samples: 469226. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:21:59,809][04690] Avg episode reward: [(0, '15.322')] [2024-10-18 23:22:04,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 833.1). Total num frames: 1871872. Throughput: 0: 214.9. Samples: 470484. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:04,814][04690] Avg episode reward: [(0, '15.140')] [2024-10-18 23:22:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1880064. Throughput: 0: 217.7. Samples: 471074. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:09,808][04690] Avg episode reward: [(0, '14.962')] [2024-10-18 23:22:13,104][05478] Updated weights for policy 0, policy_version 460 (0.2015) [2024-10-18 23:22:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1884160. Throughput: 0: 227.0. Samples: 472572. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:14,808][04690] Avg episode reward: [(0, '15.058')] [2024-10-18 23:22:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1888256. Throughput: 0: 212.2. Samples: 473688. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:19,808][04690] Avg episode reward: [(0, '14.200')] [2024-10-18 23:22:24,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1892352. Throughput: 0: 198.5. Samples: 474366. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:24,813][04690] Avg episode reward: [(0, '14.502')] [2024-10-18 23:22:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1896448. Throughput: 0: 219.7. Samples: 475710. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:22:29,813][04690] Avg episode reward: [(0, '14.170')] [2024-10-18 23:22:34,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 833.1). Total num frames: 1900544. Throughput: 0: 222.6. Samples: 477232. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:22:34,810][04690] Avg episode reward: [(0, '14.312')] [2024-10-18 23:22:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 833.1). Total num frames: 1904640. Throughput: 0: 217.2. Samples: 477774. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:39,812][04690] Avg episode reward: [(0, '14.685')] [2024-10-18 23:22:44,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 833.1). Total num frames: 1908736. Throughput: 0: 221.9. Samples: 479210. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:44,808][04690] Avg episode reward: [(0, '14.343')] [2024-10-18 23:22:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 847.0). Total num frames: 1916928. Throughput: 0: 227.6. Samples: 480728. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:22:49,810][04690] Avg episode reward: [(0, '13.957')] [2024-10-18 23:22:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1921024. Throughput: 0: 232.7. Samples: 481546. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:22:54,811][04690] Avg episode reward: [(0, '13.614')] [2024-10-18 23:22:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 847.0). Total num frames: 1925120. Throughput: 0: 222.2. Samples: 482570. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:22:59,808][04690] Avg episode reward: [(0, '13.486')] [2024-10-18 23:23:00,239][05478] Updated weights for policy 0, policy_version 470 (0.1487) [2024-10-18 23:23:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 847.0). Total num frames: 1929216. Throughput: 0: 227.7. Samples: 483934. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:23:04,809][04690] Avg episode reward: [(0, '13.871')] [2024-10-18 23:23:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1933312. Throughput: 0: 231.9. Samples: 484802. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:23:09,812][04690] Avg episode reward: [(0, '13.732')] [2024-10-18 23:23:14,809][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 847.0). Total num frames: 1937408. Throughput: 0: 225.6. Samples: 485862. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:23:14,813][04690] Avg episode reward: [(0, '13.733')] [2024-10-18 23:23:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1941504. Throughput: 0: 221.4. Samples: 487196. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:23:19,816][04690] Avg episode reward: [(0, '13.681')] [2024-10-18 23:23:24,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1945600. Throughput: 0: 249.0. Samples: 488980. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:23:24,814][04690] Avg episode reward: [(0, '13.995')] [2024-10-18 23:23:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 833.1). Total num frames: 1949696. Throughput: 0: 226.6. Samples: 489406. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:23:29,812][04690] Avg episode reward: [(0, '13.777')] [2024-10-18 23:23:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 833.1). Total num frames: 1953792. Throughput: 0: 219.8. Samples: 490618. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:23:34,811][04690] Avg episode reward: [(0, '13.894')] [2024-10-18 23:23:37,259][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000478_1957888.pth... [2024-10-18 23:23:37,375][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000429_1757184.pth [2024-10-18 23:23:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 833.1). Total num frames: 1957888. Throughput: 0: 211.4. Samples: 491060. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:23:39,814][04690] Avg episode reward: [(0, '13.795')] [2024-10-18 23:23:44,812][04690] Fps is (10 sec: 818.7, 60 sec: 887.4, 300 sec: 847.0). Total num frames: 1961984. Throughput: 0: 212.8. Samples: 492148. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:23:44,816][04690] Avg episode reward: [(0, '14.361')] [2024-10-18 23:23:48,105][05478] Updated weights for policy 0, policy_version 480 (0.1077) [2024-10-18 23:23:49,811][04690] Fps is (10 sec: 818.7, 60 sec: 819.1, 300 sec: 847.0). Total num frames: 1966080. Throughput: 0: 208.0. Samples: 493294. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:23:49,818][04690] Avg episode reward: [(0, '14.437')] [2024-10-18 23:23:54,805][04690] Fps is (10 sec: 819.7, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 1970176. Throughput: 0: 206.6. Samples: 494098. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:23:54,809][04690] Avg episode reward: [(0, '14.346')] [2024-10-18 23:23:59,806][04690] Fps is (10 sec: 819.6, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 1974272. Throughput: 0: 209.5. Samples: 495288. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:23:59,813][04690] Avg episode reward: [(0, '14.764')] [2024-10-18 23:24:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 833.1). Total num frames: 1978368. Throughput: 0: 217.8. Samples: 496998. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:24:04,807][04690] Avg episode reward: [(0, '14.619')] [2024-10-18 23:24:09,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 833.1). Total num frames: 1982464. Throughput: 0: 202.7. Samples: 498100. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:24:09,808][04690] Avg episode reward: [(0, '14.797')] [2024-10-18 23:24:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.3, 300 sec: 847.0). Total num frames: 1986560. Throughput: 0: 209.9. Samples: 498850. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:24:14,816][04690] Avg episode reward: [(0, '14.995')] [2024-10-18 23:24:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1994752. Throughput: 0: 214.3. Samples: 500262. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:19,809][04690] Avg episode reward: [(0, '15.395')] [2024-10-18 23:24:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 1998848. Throughput: 0: 221.5. Samples: 501026. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:24,813][04690] Avg episode reward: [(0, '15.736')] [2024-10-18 23:24:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2002944. Throughput: 0: 220.5. Samples: 502068. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:24:29,816][04690] Avg episode reward: [(0, '16.298')] [2024-10-18 23:24:33,434][05464] Saving new best policy, reward=16.298! [2024-10-18 23:24:33,439][05478] Updated weights for policy 0, policy_version 490 (0.0505) [2024-10-18 23:24:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2007040. Throughput: 0: 228.6. Samples: 503580. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:24:34,807][04690] Avg episode reward: [(0, '16.291')] [2024-10-18 23:24:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2011136. Throughput: 0: 229.1. Samples: 504408. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:39,811][04690] Avg episode reward: [(0, '16.171')] [2024-10-18 23:24:44,809][04690] Fps is (10 sec: 818.9, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2015232. Throughput: 0: 225.5. Samples: 505438. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:44,817][04690] Avg episode reward: [(0, '16.467')] [2024-10-18 23:24:47,839][05464] Saving new best policy, reward=16.467! [2024-10-18 23:24:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.6, 300 sec: 847.0). Total num frames: 2019328. Throughput: 0: 220.4. Samples: 506918. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:49,808][04690] Avg episode reward: [(0, '16.064')] [2024-10-18 23:24:54,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2023424. Throughput: 0: 231.8. Samples: 508532. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:54,813][04690] Avg episode reward: [(0, '16.043')] [2024-10-18 23:24:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2027520. Throughput: 0: 227.5. Samples: 509088. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:24:59,809][04690] Avg episode reward: [(0, '16.468')] [2024-10-18 23:25:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 847.0). Total num frames: 2031616. Throughput: 0: 220.7. Samples: 510194. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:25:04,812][04690] Avg episode reward: [(0, '16.178')] [2024-10-18 23:25:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2039808. Throughput: 0: 219.6. Samples: 510906. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:09,808][04690] Avg episode reward: [(0, '16.585')] [2024-10-18 23:25:13,325][05464] Saving new best policy, reward=16.585! [2024-10-18 23:25:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 860.9). Total num frames: 2043904. Throughput: 0: 233.3. Samples: 512568. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:14,813][04690] Avg episode reward: [(0, '16.602')] [2024-10-18 23:25:19,190][05464] Saving new best policy, reward=16.602! [2024-10-18 23:25:19,190][05478] Updated weights for policy 0, policy_version 500 (0.0538) [2024-10-18 23:25:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2048000. Throughput: 0: 222.5. Samples: 513592. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:19,811][04690] Avg episode reward: [(0, '16.716')] [2024-10-18 23:25:22,526][05464] Signal inference workers to stop experience collection... (500 times) [2024-10-18 23:25:22,576][05478] InferenceWorker_p0-w0: stopping experience collection (500 times) [2024-10-18 23:25:23,992][05464] Signal inference workers to resume experience collection... (500 times) [2024-10-18 23:25:23,993][05478] InferenceWorker_p0-w0: resuming experience collection (500 times) [2024-10-18 23:25:23,996][05464] Saving new best policy, reward=16.716! [2024-10-18 23:25:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2052096. Throughput: 0: 219.4. Samples: 514280. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:24,809][04690] Avg episode reward: [(0, '16.430')] [2024-10-18 23:25:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2056192. Throughput: 0: 226.3. Samples: 515620. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:29,819][04690] Avg episode reward: [(0, '15.972')] [2024-10-18 23:25:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2060288. Throughput: 0: 228.2. Samples: 517186. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:34,812][04690] Avg episode reward: [(0, '15.354')] [2024-10-18 23:25:37,575][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000504_2064384.pth... [2024-10-18 23:25:37,734][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000453_1855488.pth [2024-10-18 23:25:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2064384. Throughput: 0: 217.5. Samples: 518320. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:39,808][04690] Avg episode reward: [(0, '15.449')] [2024-10-18 23:25:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2068480. Throughput: 0: 221.7. Samples: 519066. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:25:44,808][04690] Avg episode reward: [(0, '15.081')] [2024-10-18 23:25:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2076672. Throughput: 0: 235.2. Samples: 520776. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:49,810][04690] Avg episode reward: [(0, '15.072')] [2024-10-18 23:25:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2076672. Throughput: 0: 243.2. Samples: 521850. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:54,807][04690] Avg episode reward: [(0, '15.325')] [2024-10-18 23:25:59,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2080768. Throughput: 0: 214.8. Samples: 522234. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:25:59,812][04690] Avg episode reward: [(0, '15.325')] [2024-10-18 23:26:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2084864. Throughput: 0: 213.0. Samples: 523176. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:04,810][04690] Avg episode reward: [(0, '15.368')] [2024-10-18 23:26:07,163][05478] Updated weights for policy 0, policy_version 510 (0.0064) [2024-10-18 23:26:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2088960. Throughput: 0: 212.7. Samples: 523850. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:09,813][04690] Avg episode reward: [(0, '14.646')] [2024-10-18 23:26:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2093056. Throughput: 0: 209.4. Samples: 525042. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:14,813][04690] Avg episode reward: [(0, '14.861')] [2024-10-18 23:26:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2097152. Throughput: 0: 201.5. Samples: 526254. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:19,809][04690] Avg episode reward: [(0, '15.307')] [2024-10-18 23:26:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2101248. Throughput: 0: 213.9. Samples: 527944. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:26:24,815][04690] Avg episode reward: [(0, '15.362')] [2024-10-18 23:26:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2105344. Throughput: 0: 217.3. Samples: 528844. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:26:29,808][04690] Avg episode reward: [(0, '15.361')] [2024-10-18 23:26:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2109440. Throughput: 0: 197.5. Samples: 529662. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:26:34,814][04690] Avg episode reward: [(0, '15.467')] [2024-10-18 23:26:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2117632. Throughput: 0: 194.7. Samples: 530612. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:26:39,815][04690] Avg episode reward: [(0, '15.407')] [2024-10-18 23:26:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2121728. Throughput: 0: 218.2. Samples: 532054. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:26:44,813][04690] Avg episode reward: [(0, '15.345')] [2024-10-18 23:26:49,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2125824. Throughput: 0: 223.6. Samples: 533236. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:49,811][04690] Avg episode reward: [(0, '15.202')] [2024-10-18 23:26:53,299][05478] Updated weights for policy 0, policy_version 520 (0.1609) [2024-10-18 23:26:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2129920. Throughput: 0: 224.4. Samples: 533948. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:54,815][04690] Avg episode reward: [(0, '15.552')] [2024-10-18 23:26:59,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2134016. Throughput: 0: 225.6. Samples: 535196. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:26:59,813][04690] Avg episode reward: [(0, '15.813')] [2024-10-18 23:27:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2138112. Throughput: 0: 239.4. Samples: 537028. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:27:04,809][04690] Avg episode reward: [(0, '15.777')] [2024-10-18 23:27:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2142208. Throughput: 0: 208.2. Samples: 537314. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:09,810][04690] Avg episode reward: [(0, '16.045')] [2024-10-18 23:27:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2146304. Throughput: 0: 223.2. Samples: 538886. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:14,815][04690] Avg episode reward: [(0, '16.511')] [2024-10-18 23:27:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2154496. Throughput: 0: 235.4. Samples: 540254. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:27:19,808][04690] Avg episode reward: [(0, '17.378')] [2024-10-18 23:27:24,365][05464] Saving new best policy, reward=17.378! [2024-10-18 23:27:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2158592. Throughput: 0: 233.0. Samples: 541096. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:24,810][04690] Avg episode reward: [(0, '17.776')] [2024-10-18 23:27:29,266][05464] Saving new best policy, reward=17.776! [2024-10-18 23:27:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2162688. Throughput: 0: 224.7. Samples: 542166. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:29,808][04690] Avg episode reward: [(0, '17.690')] [2024-10-18 23:27:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2166784. Throughput: 0: 231.6. Samples: 543658. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:34,810][04690] Avg episode reward: [(0, '17.679')] [2024-10-18 23:27:36,818][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000530_2170880.pth... [2024-10-18 23:27:36,824][05478] Updated weights for policy 0, policy_version 530 (0.0985) [2024-10-18 23:27:36,923][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000478_1957888.pth [2024-10-18 23:27:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2170880. Throughput: 0: 231.8. Samples: 544378. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:39,815][04690] Avg episode reward: [(0, '17.531')] [2024-10-18 23:27:44,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 2174976. Throughput: 0: 228.4. Samples: 545474. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:27:44,811][04690] Avg episode reward: [(0, '17.311')] [2024-10-18 23:27:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2179072. Throughput: 0: 222.4. Samples: 547034. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:27:49,811][04690] Avg episode reward: [(0, '18.387')] [2024-10-18 23:27:54,798][05464] Saving new best policy, reward=18.387! [2024-10-18 23:27:54,805][04690] Fps is (10 sec: 1229.0, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2187264. Throughput: 0: 247.3. Samples: 548444. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:54,817][04690] Avg episode reward: [(0, '18.104')] [2024-10-18 23:27:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2187264. Throughput: 0: 230.4. Samples: 549252. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:27:59,808][04690] Avg episode reward: [(0, '18.493')] [2024-10-18 23:28:04,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2191360. Throughput: 0: 224.0. Samples: 550336. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:28:04,810][04690] Avg episode reward: [(0, '18.264')] [2024-10-18 23:28:05,342][05464] Saving new best policy, reward=18.493! [2024-10-18 23:28:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2199552. Throughput: 0: 223.6. Samples: 551160. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:28:09,808][04690] Avg episode reward: [(0, '18.884')] [2024-10-18 23:28:12,881][05464] Saving new best policy, reward=18.884! [2024-10-18 23:28:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2203648. Throughput: 0: 231.7. Samples: 552594. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:28:14,808][04690] Avg episode reward: [(0, '19.073')] [2024-10-18 23:28:18,510][05464] Saving new best policy, reward=19.073! [2024-10-18 23:28:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2207744. Throughput: 0: 223.1. Samples: 553696. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:28:19,809][04690] Avg episode reward: [(0, '19.166')] [2024-10-18 23:28:24,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2207744. Throughput: 0: 227.4. Samples: 554612. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:28:24,808][04690] Avg episode reward: [(0, '18.936')] [2024-10-18 23:28:25,056][05464] Saving new best policy, reward=19.166! [2024-10-18 23:28:25,054][05478] Updated weights for policy 0, policy_version 540 (0.0040) [2024-10-18 23:28:29,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2211840. Throughput: 0: 217.3. Samples: 555252. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:28:29,815][04690] Avg episode reward: [(0, '19.181')] [2024-10-18 23:28:34,340][05464] Saving new best policy, reward=19.181! [2024-10-18 23:28:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2220032. Throughput: 0: 215.5. Samples: 556730. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:28:34,814][04690] Avg episode reward: [(0, '18.621')] [2024-10-18 23:28:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.8). Total num frames: 2220032. Throughput: 0: 206.0. Samples: 557716. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:28:39,808][04690] Avg episode reward: [(0, '19.442')] [2024-10-18 23:28:44,103][05464] Saving new best policy, reward=19.442! [2024-10-18 23:28:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2228224. Throughput: 0: 210.0. Samples: 558700. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:28:44,808][04690] Avg episode reward: [(0, '19.007')] [2024-10-18 23:28:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2232320. Throughput: 0: 217.2. Samples: 560110. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:28:49,813][04690] Avg episode reward: [(0, '19.385')] [2024-10-18 23:28:54,810][04690] Fps is (10 sec: 818.8, 60 sec: 819.1, 300 sec: 888.6). Total num frames: 2236416. Throughput: 0: 213.9. Samples: 560786. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:28:54,813][04690] Avg episode reward: [(0, '19.462')] [2024-10-18 23:28:57,924][05464] Saving new best policy, reward=19.462! [2024-10-18 23:28:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2240512. Throughput: 0: 204.0. Samples: 561774. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:28:59,816][04690] Avg episode reward: [(0, '19.301')] [2024-10-18 23:29:04,805][04690] Fps is (10 sec: 819.6, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2244608. Throughput: 0: 218.7. Samples: 563538. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:29:04,808][04690] Avg episode reward: [(0, '18.842')] [2024-10-18 23:29:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2248704. Throughput: 0: 228.9. Samples: 564912. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:29:09,808][04690] Avg episode reward: [(0, '18.621')] [2024-10-18 23:29:09,974][05478] Updated weights for policy 0, policy_version 550 (0.0691) [2024-10-18 23:29:13,938][05464] Signal inference workers to stop experience collection... (550 times) [2024-10-18 23:29:14,002][05478] InferenceWorker_p0-w0: stopping experience collection (550 times) [2024-10-18 23:29:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2252800. Throughput: 0: 230.8. Samples: 565638. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:29:14,811][04690] Avg episode reward: [(0, '18.368')] [2024-10-18 23:29:15,798][05464] Signal inference workers to resume experience collection... (550 times) [2024-10-18 23:29:15,811][05478] InferenceWorker_p0-w0: resuming experience collection (550 times) [2024-10-18 23:29:19,806][04690] Fps is (10 sec: 1228.7, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2260992. Throughput: 0: 226.4. Samples: 566918. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:29:19,811][04690] Avg episode reward: [(0, '18.868')] [2024-10-18 23:29:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2265088. Throughput: 0: 240.0. Samples: 568518. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:29:24,808][04690] Avg episode reward: [(0, '18.741')] [2024-10-18 23:29:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2269184. Throughput: 0: 229.3. Samples: 569020. Policy #0 lag: (min: 1.0, avg: 1.7, max: 3.0) [2024-10-18 23:29:29,808][04690] Avg episode reward: [(0, '19.370')] [2024-10-18 23:29:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2273280. Throughput: 0: 222.0. Samples: 570102. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:29:34,808][04690] Avg episode reward: [(0, '19.605')] [2024-10-18 23:29:37,338][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000556_2277376.pth... [2024-10-18 23:29:37,467][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000504_2064384.pth [2024-10-18 23:29:37,482][05464] Saving new best policy, reward=19.605! [2024-10-18 23:29:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2277376. Throughput: 0: 247.5. Samples: 571924. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:29:39,816][04690] Avg episode reward: [(0, '19.506')] [2024-10-18 23:29:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2281472. Throughput: 0: 236.5. Samples: 572416. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:29:44,808][04690] Avg episode reward: [(0, '19.391')] [2024-10-18 23:29:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2285568. Throughput: 0: 227.6. Samples: 573780. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:29:49,809][04690] Avg episode reward: [(0, '18.606')] [2024-10-18 23:29:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2289664. Throughput: 0: 222.3. Samples: 574916. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:29:54,811][04690] Avg episode reward: [(0, '18.483')] [2024-10-18 23:29:55,047][05478] Updated weights for policy 0, policy_version 560 (0.1068) [2024-10-18 23:29:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2297856. Throughput: 0: 228.8. Samples: 575936. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:29:59,813][04690] Avg episode reward: [(0, '18.141')] [2024-10-18 23:30:04,806][04690] Fps is (10 sec: 1228.7, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2301952. Throughput: 0: 227.4. Samples: 577150. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:30:04,814][04690] Avg episode reward: [(0, '17.957')] [2024-10-18 23:30:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2306048. Throughput: 0: 207.3. Samples: 577848. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:30:09,813][04690] Avg episode reward: [(0, '18.623')] [2024-10-18 23:30:14,805][04690] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2310144. Throughput: 0: 223.9. Samples: 579096. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:30:14,816][04690] Avg episode reward: [(0, '18.736')] [2024-10-18 23:30:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2314240. Throughput: 0: 242.9. Samples: 581034. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:30:19,808][04690] Avg episode reward: [(0, '18.413')] [2024-10-18 23:30:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2318336. Throughput: 0: 209.5. Samples: 581350. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:30:24,807][04690] Avg episode reward: [(0, '19.456')] [2024-10-18 23:30:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2322432. Throughput: 0: 224.2. Samples: 582506. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:30:29,817][04690] Avg episode reward: [(0, '18.756')] [2024-10-18 23:30:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2330624. Throughput: 0: 233.5. Samples: 584286. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:30:34,807][04690] Avg episode reward: [(0, '18.766')] [2024-10-18 23:30:38,943][05478] Updated weights for policy 0, policy_version 570 (0.0049) [2024-10-18 23:30:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2334720. Throughput: 0: 229.5. Samples: 585244. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:30:39,809][04690] Avg episode reward: [(0, '18.864')] [2024-10-18 23:30:44,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2334720. Throughput: 0: 226.9. Samples: 586146. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:30:44,809][04690] Avg episode reward: [(0, '18.968')] [2024-10-18 23:30:49,807][04690] Fps is (10 sec: 409.5, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 2338816. Throughput: 0: 220.3. Samples: 587062. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:30:49,815][04690] Avg episode reward: [(0, '18.985')] [2024-10-18 23:30:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2342912. Throughput: 0: 233.7. Samples: 588366. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:30:54,815][04690] Avg episode reward: [(0, '18.699')] [2024-10-18 23:30:59,805][04690] Fps is (10 sec: 819.4, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2347008. Throughput: 0: 227.2. Samples: 589320. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:30:59,808][04690] Avg episode reward: [(0, '18.519')] [2024-10-18 23:31:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2351104. Throughput: 0: 206.9. Samples: 590344. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:31:04,809][04690] Avg episode reward: [(0, '18.312')] [2024-10-18 23:31:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2359296. Throughput: 0: 212.8. Samples: 590928. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:31:09,812][04690] Avg episode reward: [(0, '17.977')] [2024-10-18 23:31:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2363392. Throughput: 0: 221.4. Samples: 592468. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:31:14,808][04690] Avg episode reward: [(0, '17.874')] [2024-10-18 23:31:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2367488. Throughput: 0: 210.5. Samples: 593760. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:31:19,813][04690] Avg episode reward: [(0, '17.778')] [2024-10-18 23:31:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2371584. Throughput: 0: 219.0. Samples: 595098. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:31:24,814][04690] Avg episode reward: [(0, '17.439')] [2024-10-18 23:31:27,099][05478] Updated weights for policy 0, policy_version 580 (0.0043) [2024-10-18 23:31:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2375680. Throughput: 0: 214.2. Samples: 595784. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:31:29,814][04690] Avg episode reward: [(0, '17.998')] [2024-10-18 23:31:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2379776. Throughput: 0: 232.5. Samples: 597524. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:31:34,807][04690] Avg episode reward: [(0, '18.475')] [2024-10-18 23:31:39,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2383872. Throughput: 0: 225.3. Samples: 598506. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:31:39,816][04690] Avg episode reward: [(0, '18.741')] [2024-10-18 23:31:41,015][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000583_2387968.pth... [2024-10-18 23:31:41,126][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000530_2170880.pth [2024-10-18 23:31:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2392064. Throughput: 0: 220.8. Samples: 599256. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:31:44,815][04690] Avg episode reward: [(0, '19.092')] [2024-10-18 23:31:49,805][04690] Fps is (10 sec: 1228.9, 60 sec: 955.8, 300 sec: 902.5). Total num frames: 2396160. Throughput: 0: 227.1. Samples: 600564. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:31:49,807][04690] Avg episode reward: [(0, '19.429')] [2024-10-18 23:31:54,806][04690] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2400256. Throughput: 0: 234.3. Samples: 601472. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:31:54,809][04690] Avg episode reward: [(0, '19.429')] [2024-10-18 23:31:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2404352. Throughput: 0: 221.0. Samples: 602414. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:31:59,816][04690] Avg episode reward: [(0, '20.426')] [2024-10-18 23:32:02,939][05464] Saving new best policy, reward=20.426! [2024-10-18 23:32:04,805][04690] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2408448. Throughput: 0: 226.9. Samples: 603972. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:04,810][04690] Avg episode reward: [(0, '19.748')] [2024-10-18 23:32:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2412544. Throughput: 0: 233.1. Samples: 605588. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:32:09,810][04690] Avg episode reward: [(0, '19.267')] [2024-10-18 23:32:11,276][05478] Updated weights for policy 0, policy_version 590 (0.2150) [2024-10-18 23:32:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2416640. Throughput: 0: 225.9. Samples: 605950. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:32:14,810][04690] Avg episode reward: [(0, '19.614')] [2024-10-18 23:32:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2420736. Throughput: 0: 221.0. Samples: 607468. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:32:19,811][04690] Avg episode reward: [(0, '19.949')] [2024-10-18 23:32:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2428928. Throughput: 0: 214.5. Samples: 608158. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:24,817][04690] Avg episode reward: [(0, '19.926')] [2024-10-18 23:32:29,808][04690] Fps is (10 sec: 1228.4, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2433024. Throughput: 0: 231.2. Samples: 609662. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:29,811][04690] Avg episode reward: [(0, '19.601')] [2024-10-18 23:32:34,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2433024. Throughput: 0: 226.4. Samples: 610750. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:34,811][04690] Avg episode reward: [(0, '19.167')] [2024-10-18 23:32:39,805][04690] Fps is (10 sec: 819.5, 60 sec: 955.8, 300 sec: 902.5). Total num frames: 2441216. Throughput: 0: 219.6. Samples: 611352. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:39,813][04690] Avg episode reward: [(0, '19.422')] [2024-10-18 23:32:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2445312. Throughput: 0: 232.5. Samples: 612876. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:44,808][04690] Avg episode reward: [(0, '19.372')] [2024-10-18 23:32:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2449408. Throughput: 0: 226.3. Samples: 614156. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:49,811][04690] Avg episode reward: [(0, '19.251')] [2024-10-18 23:32:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2453504. Throughput: 0: 223.7. Samples: 615656. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:54,808][04690] Avg episode reward: [(0, '19.179')] [2024-10-18 23:32:56,730][05478] Updated weights for policy 0, policy_version 600 (0.1481) [2024-10-18 23:32:59,091][05464] Signal inference workers to stop experience collection... (600 times) [2024-10-18 23:32:59,139][05478] InferenceWorker_p0-w0: stopping experience collection (600 times) [2024-10-18 23:32:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2457600. Throughput: 0: 230.8. Samples: 616338. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:32:59,811][04690] Avg episode reward: [(0, '19.733')] [2024-10-18 23:33:00,577][05464] Signal inference workers to resume experience collection... (600 times) [2024-10-18 23:33:00,578][05478] InferenceWorker_p0-w0: resuming experience collection (600 times) [2024-10-18 23:33:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2461696. Throughput: 0: 232.4. Samples: 617928. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:33:04,808][04690] Avg episode reward: [(0, '20.373')] [2024-10-18 23:33:09,815][04690] Fps is (10 sec: 818.4, 60 sec: 887.3, 300 sec: 888.6). Total num frames: 2465792. Throughput: 0: 221.7. Samples: 618136. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:33:09,829][04690] Avg episode reward: [(0, '19.469')] [2024-10-18 23:33:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2469888. Throughput: 0: 206.4. Samples: 618948. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:14,816][04690] Avg episode reward: [(0, '19.613')] [2024-10-18 23:33:19,805][04690] Fps is (10 sec: 820.0, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2473984. Throughput: 0: 209.2. Samples: 620162. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:19,813][04690] Avg episode reward: [(0, '19.569')] [2024-10-18 23:33:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 902.5). Total num frames: 2478080. Throughput: 0: 236.1. Samples: 621978. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:24,813][04690] Avg episode reward: [(0, '19.210')] [2024-10-18 23:33:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2482176. Throughput: 0: 212.4. Samples: 622434. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:29,809][04690] Avg episode reward: [(0, '18.762')] [2024-10-18 23:33:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2486272. Throughput: 0: 213.6. Samples: 623768. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:34,808][04690] Avg episode reward: [(0, '18.031')] [2024-10-18 23:33:39,453][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000609_2494464.pth... [2024-10-18 23:33:39,555][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000556_2277376.pth [2024-10-18 23:33:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2494464. Throughput: 0: 192.8. Samples: 624330. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:39,812][04690] Avg episode reward: [(0, '17.349')] [2024-10-18 23:33:43,882][05478] Updated weights for policy 0, policy_version 610 (0.1056) [2024-10-18 23:33:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2498560. Throughput: 0: 216.4. Samples: 626078. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:44,815][04690] Avg episode reward: [(0, '18.011')] [2024-10-18 23:33:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2502656. Throughput: 0: 204.9. Samples: 627150. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:49,808][04690] Avg episode reward: [(0, '18.403')] [2024-10-18 23:33:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2506752. Throughput: 0: 219.0. Samples: 627990. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:33:54,807][04690] Avg episode reward: [(0, '18.308')] [2024-10-18 23:33:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2510848. Throughput: 0: 228.5. Samples: 629230. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:33:59,809][04690] Avg episode reward: [(0, '17.992')] [2024-10-18 23:34:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2514944. Throughput: 0: 232.9. Samples: 630642. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:34:04,817][04690] Avg episode reward: [(0, '17.900')] [2024-10-18 23:34:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.6, 300 sec: 902.5). Total num frames: 2519040. Throughput: 0: 225.4. Samples: 632120. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:34:09,810][04690] Avg episode reward: [(0, '18.442')] [2024-10-18 23:34:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2523136. Throughput: 0: 226.1. Samples: 632610. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:34:14,808][04690] Avg episode reward: [(0, '18.820')] [2024-10-18 23:34:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2527232. Throughput: 0: 235.9. Samples: 634384. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:34:19,810][04690] Avg episode reward: [(0, '19.281')] [2024-10-18 23:34:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2531328. Throughput: 0: 245.5. Samples: 635378. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:34:24,809][04690] Avg episode reward: [(0, '19.625')] [2024-10-18 23:34:29,440][05478] Updated weights for policy 0, policy_version 620 (0.0987) [2024-10-18 23:34:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2539520. Throughput: 0: 226.3. Samples: 636262. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:34:29,813][04690] Avg episode reward: [(0, '19.363')] [2024-10-18 23:34:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2543616. Throughput: 0: 232.0. Samples: 637592. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:34:34,812][04690] Avg episode reward: [(0, '19.478')] [2024-10-18 23:34:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2547712. Throughput: 0: 231.6. Samples: 638410. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:34:39,811][04690] Avg episode reward: [(0, '19.573')] [2024-10-18 23:34:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2551808. Throughput: 0: 226.6. Samples: 639428. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:34:44,807][04690] Avg episode reward: [(0, '20.041')] [2024-10-18 23:34:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2555904. Throughput: 0: 227.2. Samples: 640868. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:34:49,812][04690] Avg episode reward: [(0, '19.399')] [2024-10-18 23:34:54,807][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 2560000. Throughput: 0: 231.6. Samples: 642542. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:34:54,809][04690] Avg episode reward: [(0, '19.897')] [2024-10-18 23:34:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2564096. Throughput: 0: 228.8. Samples: 642906. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:34:59,813][04690] Avg episode reward: [(0, '20.322')] [2024-10-18 23:35:04,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2568192. Throughput: 0: 220.0. Samples: 644284. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:35:04,810][04690] Avg episode reward: [(0, '20.644')] [2024-10-18 23:35:09,217][05464] Saving new best policy, reward=20.644! [2024-10-18 23:35:09,809][04690] Fps is (10 sec: 1228.4, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2576384. Throughput: 0: 231.8. Samples: 645810. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:35:09,813][04690] Avg episode reward: [(0, '20.987')] [2024-10-18 23:35:14,600][05464] Saving new best policy, reward=20.987! [2024-10-18 23:35:14,612][05478] Updated weights for policy 0, policy_version 630 (0.1031) [2024-10-18 23:35:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2580480. Throughput: 0: 231.0. Samples: 646658. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:35:14,808][04690] Avg episode reward: [(0, '21.170')] [2024-10-18 23:35:19,747][05464] Saving new best policy, reward=21.170! [2024-10-18 23:35:19,805][04690] Fps is (10 sec: 819.5, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2584576. Throughput: 0: 224.2. Samples: 647680. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:35:19,810][04690] Avg episode reward: [(0, '21.429')] [2024-10-18 23:35:23,748][05464] Saving new best policy, reward=21.429! [2024-10-18 23:35:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 902.5). Total num frames: 2588672. Throughput: 0: 225.0. Samples: 648534. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:35:24,813][04690] Avg episode reward: [(0, '21.980')] [2024-10-18 23:35:29,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2588672. Throughput: 0: 227.1. Samples: 649646. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:35:29,808][04690] Avg episode reward: [(0, '21.844')] [2024-10-18 23:35:29,889][05464] Saving new best policy, reward=21.980! [2024-10-18 23:35:34,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2592768. Throughput: 0: 213.9. Samples: 650494. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:35:34,808][04690] Avg episode reward: [(0, '22.129')] [2024-10-18 23:35:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2596864. Throughput: 0: 201.9. Samples: 651626. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:35:39,810][04690] Avg episode reward: [(0, '21.734')] [2024-10-18 23:35:40,829][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000635_2600960.pth... [2024-10-18 23:35:40,934][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000583_2387968.pth [2024-10-18 23:35:40,950][05464] Saving new best policy, reward=22.129! [2024-10-18 23:35:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2605056. Throughput: 0: 211.3. Samples: 652416. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:35:44,814][04690] Avg episode reward: [(0, '21.719')] [2024-10-18 23:35:49,807][04690] Fps is (10 sec: 1228.6, 60 sec: 887.4, 300 sec: 902.5). Total num frames: 2609152. Throughput: 0: 209.7. Samples: 653720. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:35:49,814][04690] Avg episode reward: [(0, '21.946')] [2024-10-18 23:35:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2613248. Throughput: 0: 189.0. Samples: 654314. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:35:54,812][04690] Avg episode reward: [(0, '22.274')] [2024-10-18 23:35:58,910][05464] Saving new best policy, reward=22.274! [2024-10-18 23:35:59,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 902.5). Total num frames: 2617344. Throughput: 0: 201.5. Samples: 655724. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:35:59,813][04690] Avg episode reward: [(0, '22.228')] [2024-10-18 23:36:02,834][05478] Updated weights for policy 0, policy_version 640 (0.0706) [2024-10-18 23:36:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2621440. Throughput: 0: 209.6. Samples: 657110. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:36:04,808][04690] Avg episode reward: [(0, '22.696')] [2024-10-18 23:36:06,740][05464] Saving new best policy, reward=22.696! [2024-10-18 23:36:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2625536. Throughput: 0: 205.7. Samples: 657792. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:36:09,808][04690] Avg episode reward: [(0, '22.985')] [2024-10-18 23:36:12,788][05464] Saving new best policy, reward=22.985! [2024-10-18 23:36:14,806][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 2629632. Throughput: 0: 202.9. Samples: 658776. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:36:14,844][04690] Avg episode reward: [(0, '22.896')] [2024-10-18 23:36:19,807][04690] Fps is (10 sec: 409.5, 60 sec: 750.9, 300 sec: 874.7). Total num frames: 2629632. Throughput: 0: 207.5. Samples: 659832. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:36:19,819][04690] Avg episode reward: [(0, '22.896')] [2024-10-18 23:36:24,805][04690] Fps is (10 sec: 409.6, 60 sec: 750.9, 300 sec: 874.7). Total num frames: 2633728. Throughput: 0: 190.3. Samples: 660188. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:36:24,808][04690] Avg episode reward: [(0, '22.876')] [2024-10-18 23:36:29,805][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2637824. Throughput: 0: 191.1. Samples: 661016. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:36:29,809][04690] Avg episode reward: [(0, '23.153')] [2024-10-18 23:36:33,697][05464] Saving new best policy, reward=23.153! [2024-10-18 23:36:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2641920. Throughput: 0: 185.7. Samples: 662078. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:36:34,808][04690] Avg episode reward: [(0, '23.891')] [2024-10-18 23:36:37,462][05464] Saving new best policy, reward=23.891! [2024-10-18 23:36:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2646016. Throughput: 0: 215.9. Samples: 664028. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:36:39,808][04690] Avg episode reward: [(0, '23.672')] [2024-10-18 23:36:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 860.9). Total num frames: 2650112. Throughput: 0: 195.6. Samples: 664526. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:36:44,813][04690] Avg episode reward: [(0, '24.108')] [2024-10-18 23:36:49,808][04690] Fps is (10 sec: 818.9, 60 sec: 750.9, 300 sec: 860.8). Total num frames: 2654208. Throughput: 0: 189.2. Samples: 665626. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:36:49,811][04690] Avg episode reward: [(0, '23.553')] [2024-10-18 23:36:51,816][05464] Saving new best policy, reward=24.108! [2024-10-18 23:36:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 750.9, 300 sec: 860.9). Total num frames: 2658304. Throughput: 0: 208.4. Samples: 667168. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:36:54,814][04690] Avg episode reward: [(0, '22.713')] [2024-10-18 23:36:55,659][05478] Updated weights for policy 0, policy_version 650 (0.0570) [2024-10-18 23:36:57,984][05464] Signal inference workers to stop experience collection... (650 times) [2024-10-18 23:36:58,010][05478] InferenceWorker_p0-w0: stopping experience collection (650 times) [2024-10-18 23:36:59,442][05464] Signal inference workers to resume experience collection... (650 times) [2024-10-18 23:36:59,444][05478] InferenceWorker_p0-w0: resuming experience collection (650 times) [2024-10-18 23:36:59,805][04690] Fps is (10 sec: 1229.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2666496. Throughput: 0: 207.0. Samples: 668090. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:36:59,816][04690] Avg episode reward: [(0, '22.117')] [2024-10-18 23:37:04,805][04690] Fps is (10 sec: 1228.8, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2670592. Throughput: 0: 209.0. Samples: 669238. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:04,814][04690] Avg episode reward: [(0, '21.995')] [2024-10-18 23:37:09,805][04690] Fps is (10 sec: 409.6, 60 sec: 750.9, 300 sec: 860.9). Total num frames: 2670592. Throughput: 0: 223.4. Samples: 670242. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:09,816][04690] Avg episode reward: [(0, '21.679')] [2024-10-18 23:37:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2678784. Throughput: 0: 227.2. Samples: 671242. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:37:14,814][04690] Avg episode reward: [(0, '21.791')] [2024-10-18 23:37:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2682880. Throughput: 0: 237.8. Samples: 672778. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:37:19,815][04690] Avg episode reward: [(0, '21.977')] [2024-10-18 23:37:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2686976. Throughput: 0: 206.4. Samples: 673318. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:37:24,808][04690] Avg episode reward: [(0, '21.463')] [2024-10-18 23:37:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2691072. Throughput: 0: 217.2. Samples: 674298. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:37:29,812][04690] Avg episode reward: [(0, '21.485')] [2024-10-18 23:37:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2695168. Throughput: 0: 231.8. Samples: 676056. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:34,810][04690] Avg episode reward: [(0, '20.864')] [2024-10-18 23:37:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2699264. Throughput: 0: 226.3. Samples: 677350. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:39,810][04690] Avg episode reward: [(0, '20.995')] [2024-10-18 23:37:40,925][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000660_2703360.pth... [2024-10-18 23:37:40,914][05478] Updated weights for policy 0, policy_version 660 (0.1480) [2024-10-18 23:37:41,100][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000609_2494464.pth [2024-10-18 23:37:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2703360. Throughput: 0: 216.4. Samples: 677828. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:44,808][04690] Avg episode reward: [(0, '20.600')] [2024-10-18 23:37:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2707456. Throughput: 0: 211.5. Samples: 678756. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:49,815][04690] Avg episode reward: [(0, '20.366')] [2024-10-18 23:37:54,810][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 860.8). Total num frames: 2711552. Throughput: 0: 204.5. Samples: 679446. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:54,812][04690] Avg episode reward: [(0, '20.063')] [2024-10-18 23:37:59,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2715648. Throughput: 0: 208.6. Samples: 680628. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:37:59,809][04690] Avg episode reward: [(0, '19.758')] [2024-10-18 23:38:04,805][04690] Fps is (10 sec: 819.6, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2719744. Throughput: 0: 198.0. Samples: 681690. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:04,809][04690] Avg episode reward: [(0, '19.203')] [2024-10-18 23:38:09,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2723840. Throughput: 0: 203.9. Samples: 682494. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:09,807][04690] Avg episode reward: [(0, '18.644')] [2024-10-18 23:38:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2727936. Throughput: 0: 219.2. Samples: 684162. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:14,807][04690] Avg episode reward: [(0, '18.449')] [2024-10-18 23:38:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2732032. Throughput: 0: 208.8. Samples: 685452. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:19,812][04690] Avg episode reward: [(0, '18.171')] [2024-10-18 23:38:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2740224. Throughput: 0: 204.5. Samples: 686554. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:38:24,811][04690] Avg episode reward: [(0, '17.850')] [2024-10-18 23:38:28,555][05478] Updated weights for policy 0, policy_version 670 (0.1450) [2024-10-18 23:38:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2744320. Throughput: 0: 216.8. Samples: 687586. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:29,807][04690] Avg episode reward: [(0, '17.584')] [2024-10-18 23:38:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2748416. Throughput: 0: 223.9. Samples: 688832. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:34,808][04690] Avg episode reward: [(0, '17.609')] [2024-10-18 23:38:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2752512. Throughput: 0: 234.2. Samples: 689982. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:38:39,809][04690] Avg episode reward: [(0, '17.611')] [2024-10-18 23:38:44,807][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2756608. Throughput: 0: 224.7. Samples: 690738. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:38:44,813][04690] Avg episode reward: [(0, '18.184')] [2024-10-18 23:38:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2760704. Throughput: 0: 244.8. Samples: 692704. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:38:49,808][04690] Avg episode reward: [(0, '17.980')] [2024-10-18 23:38:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2764800. Throughput: 0: 249.7. Samples: 693730. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:38:54,811][04690] Avg episode reward: [(0, '17.618')] [2024-10-18 23:38:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2768896. Throughput: 0: 222.3. Samples: 694164. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:38:59,817][04690] Avg episode reward: [(0, '18.034')] [2024-10-18 23:39:04,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2777088. Throughput: 0: 229.3. Samples: 695772. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:39:04,808][04690] Avg episode reward: [(0, '18.453')] [2024-10-18 23:39:09,806][04690] Fps is (10 sec: 1228.7, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2781184. Throughput: 0: 228.2. Samples: 696822. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:39:09,812][04690] Avg episode reward: [(0, '17.560')] [2024-10-18 23:39:14,347][05478] Updated weights for policy 0, policy_version 680 (0.0052) [2024-10-18 23:39:14,806][04690] Fps is (10 sec: 819.1, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2785280. Throughput: 0: 227.9. Samples: 697842. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:39:14,811][04690] Avg episode reward: [(0, '17.575')] [2024-10-18 23:39:19,805][04690] Fps is (10 sec: 819.3, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2789376. Throughput: 0: 223.7. Samples: 698898. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:39:19,810][04690] Avg episode reward: [(0, '17.619')] [2024-10-18 23:39:24,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2793472. Throughput: 0: 241.3. Samples: 700842. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:39:24,816][04690] Avg episode reward: [(0, '17.556')] [2024-10-18 23:39:29,807][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 860.8). Total num frames: 2797568. Throughput: 0: 232.8. Samples: 701214. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:39:29,810][04690] Avg episode reward: [(0, '17.668')] [2024-10-18 23:39:34,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 860.8). Total num frames: 2801664. Throughput: 0: 216.1. Samples: 702430. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:39:34,811][04690] Avg episode reward: [(0, '18.009')] [2024-10-18 23:39:36,444][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000685_2805760.pth... [2024-10-18 23:39:36,552][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000635_2600960.pth [2024-10-18 23:39:39,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2805760. Throughput: 0: 228.8. Samples: 704028. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:39:39,816][04690] Avg episode reward: [(0, '18.154')] [2024-10-18 23:39:44,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2809856. Throughput: 0: 238.2. Samples: 704884. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:39:44,809][04690] Avg episode reward: [(0, '18.471')] [2024-10-18 23:39:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2813952. Throughput: 0: 227.2. Samples: 705998. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:39:49,808][04690] Avg episode reward: [(0, '18.310')] [2024-10-18 23:39:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2822144. Throughput: 0: 215.9. Samples: 706536. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:39:54,818][04690] Avg episode reward: [(0, '18.653')] [2024-10-18 23:39:58,489][05478] Updated weights for policy 0, policy_version 690 (0.1493) [2024-10-18 23:39:59,808][04690] Fps is (10 sec: 1228.5, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 2826240. Throughput: 0: 230.6. Samples: 708218. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:39:59,815][04690] Avg episode reward: [(0, '19.438')] [2024-10-18 23:40:04,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 860.9). Total num frames: 2830336. Throughput: 0: 233.9. Samples: 709424. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:04,810][04690] Avg episode reward: [(0, '20.304')] [2024-10-18 23:40:09,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 2830336. Throughput: 0: 210.0. Samples: 710290. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:09,809][04690] Avg episode reward: [(0, '21.088')] [2024-10-18 23:40:14,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 847.0). Total num frames: 2834432. Throughput: 0: 215.4. Samples: 710908. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:14,808][04690] Avg episode reward: [(0, '20.974')] [2024-10-18 23:40:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2842624. Throughput: 0: 218.2. Samples: 712250. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:19,808][04690] Avg episode reward: [(0, '21.070')] [2024-10-18 23:40:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2846720. Throughput: 0: 203.1. Samples: 713166. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:24,808][04690] Avg episode reward: [(0, '21.119')] [2024-10-18 23:40:29,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2846720. Throughput: 0: 205.2. Samples: 714116. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:29,813][04690] Avg episode reward: [(0, '21.566')] [2024-10-18 23:40:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2854912. Throughput: 0: 208.1. Samples: 715362. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:34,814][04690] Avg episode reward: [(0, '22.010')] [2024-10-18 23:40:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2859008. Throughput: 0: 219.3. Samples: 716404. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:39,810][04690] Avg episode reward: [(0, '21.991')] [2024-10-18 23:40:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2863104. Throughput: 0: 204.4. Samples: 717416. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:44,816][04690] Avg episode reward: [(0, '21.959')] [2024-10-18 23:40:48,919][05478] Updated weights for policy 0, policy_version 700 (0.1029) [2024-10-18 23:40:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2867200. Throughput: 0: 201.4. Samples: 718488. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:49,817][04690] Avg episode reward: [(0, '22.464')] [2024-10-18 23:40:51,282][05464] Signal inference workers to stop experience collection... (700 times) [2024-10-18 23:40:51,353][05478] InferenceWorker_p0-w0: stopping experience collection (700 times) [2024-10-18 23:40:52,320][05464] Signal inference workers to resume experience collection... (700 times) [2024-10-18 23:40:52,321][05478] InferenceWorker_p0-w0: resuming experience collection (700 times) [2024-10-18 23:40:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2871296. Throughput: 0: 226.6. Samples: 720488. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:54,812][04690] Avg episode reward: [(0, '22.515')] [2024-10-18 23:40:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 2875392. Throughput: 0: 227.7. Samples: 721156. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:40:59,810][04690] Avg episode reward: [(0, '22.788')] [2024-10-18 23:41:04,809][04690] Fps is (10 sec: 818.9, 60 sec: 819.2, 300 sec: 860.8). Total num frames: 2879488. Throughput: 0: 220.4. Samples: 722168. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:41:04,812][04690] Avg episode reward: [(0, '22.747')] [2024-10-18 23:41:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 860.9). Total num frames: 2883584. Throughput: 0: 230.5. Samples: 723540. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:41:09,816][04690] Avg episode reward: [(0, '22.654')] [2024-10-18 23:41:14,805][04690] Fps is (10 sec: 1229.3, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2891776. Throughput: 0: 227.8. Samples: 724366. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:41:14,808][04690] Avg episode reward: [(0, '23.077')] [2024-10-18 23:41:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2895872. Throughput: 0: 227.6. Samples: 725602. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:41:19,807][04690] Avg episode reward: [(0, '23.956')] [2024-10-18 23:41:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2899968. Throughput: 0: 216.6. Samples: 726152. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:41:24,817][04690] Avg episode reward: [(0, '24.354')] [2024-10-18 23:41:28,629][05464] Saving new best policy, reward=24.354! [2024-10-18 23:41:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2904064. Throughput: 0: 226.9. Samples: 727626. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:41:29,811][04690] Avg episode reward: [(0, '24.238')] [2024-10-18 23:41:32,501][05478] Updated weights for policy 0, policy_version 710 (0.1023) [2024-10-18 23:41:34,811][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 2908160. Throughput: 0: 236.1. Samples: 729114. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:41:34,813][04690] Avg episode reward: [(0, '24.529')] [2024-10-18 23:41:37,991][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000711_2912256.pth... [2024-10-18 23:41:38,145][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000660_2703360.pth [2024-10-18 23:41:38,160][05464] Saving new best policy, reward=24.529! [2024-10-18 23:41:39,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 2912256. Throughput: 0: 204.0. Samples: 729670. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:41:39,814][04690] Avg episode reward: [(0, '24.488')] [2024-10-18 23:41:44,805][04690] Fps is (10 sec: 819.6, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2916352. Throughput: 0: 211.8. Samples: 730686. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:41:44,808][04690] Avg episode reward: [(0, '24.573')] [2024-10-18 23:41:47,114][05464] Saving new best policy, reward=24.573! [2024-10-18 23:41:49,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2920448. Throughput: 0: 227.5. Samples: 732406. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:41:49,812][04690] Avg episode reward: [(0, '24.369')] [2024-10-18 23:41:54,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2924544. Throughput: 0: 207.0. Samples: 732856. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:41:54,818][04690] Avg episode reward: [(0, '24.062')] [2024-10-18 23:41:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 2928640. Throughput: 0: 215.4. Samples: 734058. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:41:59,808][04690] Avg episode reward: [(0, '24.062')] [2024-10-18 23:42:04,807][04690] Fps is (10 sec: 819.1, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2932736. Throughput: 0: 224.3. Samples: 735694. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:42:04,814][04690] Avg episode reward: [(0, '23.090')] [2024-10-18 23:42:09,808][04690] Fps is (10 sec: 1228.5, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 2940928. Throughput: 0: 225.5. Samples: 736300. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:42:09,817][04690] Avg episode reward: [(0, '22.734')] [2024-10-18 23:42:14,805][04690] Fps is (10 sec: 819.4, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2940928. Throughput: 0: 219.0. Samples: 737480. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:42:14,808][04690] Avg episode reward: [(0, '22.911')] [2024-10-18 23:42:19,598][05478] Updated weights for policy 0, policy_version 720 (0.1014) [2024-10-18 23:42:19,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2949120. Throughput: 0: 214.3. Samples: 738758. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:42:19,808][04690] Avg episode reward: [(0, '22.505')] [2024-10-18 23:42:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2953216. Throughput: 0: 233.7. Samples: 740186. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:42:24,809][04690] Avg episode reward: [(0, '22.354')] [2024-10-18 23:42:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2957312. Throughput: 0: 223.9. Samples: 740762. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:42:29,809][04690] Avg episode reward: [(0, '21.719')] [2024-10-18 23:42:34,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.3, 300 sec: 874.7). Total num frames: 2957312. Throughput: 0: 208.3. Samples: 741780. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:42:34,808][04690] Avg episode reward: [(0, '22.241')] [2024-10-18 23:42:39,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2961408. Throughput: 0: 206.2. Samples: 742134. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:42:39,809][04690] Avg episode reward: [(0, '22.590')] [2024-10-18 23:42:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2969600. Throughput: 0: 213.1. Samples: 743648. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:42:44,813][04690] Avg episode reward: [(0, '22.802')] [2024-10-18 23:42:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2973696. Throughput: 0: 204.9. Samples: 744916. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:42:49,808][04690] Avg episode reward: [(0, '22.679')] [2024-10-18 23:42:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2977792. Throughput: 0: 211.4. Samples: 745812. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:42:54,811][04690] Avg episode reward: [(0, '22.685')] [2024-10-18 23:42:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2981888. Throughput: 0: 208.3. Samples: 746852. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:42:59,807][04690] Avg episode reward: [(0, '22.980')] [2024-10-18 23:43:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2985984. Throughput: 0: 213.5. Samples: 748364. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:43:04,809][04690] Avg episode reward: [(0, '22.639')] [2024-10-18 23:43:06,497][05478] Updated weights for policy 0, policy_version 730 (0.0993) [2024-10-18 23:43:09,814][04690] Fps is (10 sec: 818.5, 60 sec: 819.1, 300 sec: 888.6). Total num frames: 2990080. Throughput: 0: 215.5. Samples: 749886. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:09,820][04690] Avg episode reward: [(0, '22.527')] [2024-10-18 23:43:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 2994176. Throughput: 0: 208.9. Samples: 750164. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:14,813][04690] Avg episode reward: [(0, '22.527')] [2024-10-18 23:43:19,805][04690] Fps is (10 sec: 819.9, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 2998272. Throughput: 0: 216.6. Samples: 751528. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:19,808][04690] Avg episode reward: [(0, '21.668')] [2024-10-18 23:43:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3006464. Throughput: 0: 223.4. Samples: 752188. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:24,818][04690] Avg episode reward: [(0, '21.880')] [2024-10-18 23:43:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3006464. Throughput: 0: 227.0. Samples: 753862. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:29,809][04690] Avg episode reward: [(0, '22.353')] [2024-10-18 23:43:34,805][04690] Fps is (10 sec: 409.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3010560. Throughput: 0: 224.1. Samples: 755000. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:34,816][04690] Avg episode reward: [(0, '21.746')] [2024-10-18 23:43:39,003][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000737_3018752.pth... [2024-10-18 23:43:39,116][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000685_2805760.pth [2024-10-18 23:43:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3018752. Throughput: 0: 235.1. Samples: 756392. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:43:39,807][04690] Avg episode reward: [(0, '21.350')] [2024-10-18 23:43:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3022848. Throughput: 0: 228.1. Samples: 757116. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:43:44,809][04690] Avg episode reward: [(0, '21.834')] [2024-10-18 23:43:49,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 3026944. Throughput: 0: 218.1. Samples: 758180. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:43:49,809][04690] Avg episode reward: [(0, '22.592')] [2024-10-18 23:43:53,253][05478] Updated weights for policy 0, policy_version 740 (0.1503) [2024-10-18 23:43:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3031040. Throughput: 0: 220.4. Samples: 759802. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:43:54,816][04690] Avg episode reward: [(0, '22.749')] [2024-10-18 23:43:59,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3035136. Throughput: 0: 226.4. Samples: 760354. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:43:59,815][04690] Avg episode reward: [(0, '23.370')] [2024-10-18 23:44:04,807][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3039232. Throughput: 0: 231.8. Samples: 761960. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:04,811][04690] Avg episode reward: [(0, '22.283')] [2024-10-18 23:44:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.6, 300 sec: 874.7). Total num frames: 3043328. Throughput: 0: 225.4. Samples: 762332. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:09,808][04690] Avg episode reward: [(0, '21.949')] [2024-10-18 23:44:14,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3047424. Throughput: 0: 223.8. Samples: 763932. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:14,808][04690] Avg episode reward: [(0, '22.100')] [2024-10-18 23:44:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3055616. Throughput: 0: 230.1. Samples: 765354. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:19,813][04690] Avg episode reward: [(0, '23.197')] [2024-10-18 23:44:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3059712. Throughput: 0: 216.9. Samples: 766154. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:24,814][04690] Avg episode reward: [(0, '22.725')] [2024-10-18 23:44:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3063808. Throughput: 0: 225.5. Samples: 767264. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:29,809][04690] Avg episode reward: [(0, '21.633')] [2024-10-18 23:44:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3067904. Throughput: 0: 236.9. Samples: 768838. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:34,812][04690] Avg episode reward: [(0, '21.308')] [2024-10-18 23:44:36,412][05478] Updated weights for policy 0, policy_version 750 (0.0534) [2024-10-18 23:44:39,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3072000. Throughput: 0: 216.2. Samples: 769530. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:39,810][04690] Avg episode reward: [(0, '21.390')] [2024-10-18 23:44:40,363][05464] Signal inference workers to stop experience collection... (750 times) [2024-10-18 23:44:40,440][05478] InferenceWorker_p0-w0: stopping experience collection (750 times) [2024-10-18 23:44:42,110][05464] Signal inference workers to resume experience collection... (750 times) [2024-10-18 23:44:42,113][05478] InferenceWorker_p0-w0: resuming experience collection (750 times) [2024-10-18 23:44:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3076096. Throughput: 0: 225.2. Samples: 770486. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:44,808][04690] Avg episode reward: [(0, '20.954')] [2024-10-18 23:44:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3080192. Throughput: 0: 213.8. Samples: 771580. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:49,812][04690] Avg episode reward: [(0, '21.059')] [2024-10-18 23:44:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3084288. Throughput: 0: 221.2. Samples: 772284. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:44:54,812][04690] Avg episode reward: [(0, '21.193')] [2024-10-18 23:44:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3088384. Throughput: 0: 213.5. Samples: 773538. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:44:59,813][04690] Avg episode reward: [(0, '21.075')] [2024-10-18 23:45:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3092480. Throughput: 0: 202.2. Samples: 774452. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:45:04,808][04690] Avg episode reward: [(0, '20.908')] [2024-10-18 23:45:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3096576. Throughput: 0: 207.2. Samples: 775476. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:45:09,809][04690] Avg episode reward: [(0, '21.852')] [2024-10-18 23:45:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3100672. Throughput: 0: 215.0. Samples: 776940. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:45:14,813][04690] Avg episode reward: [(0, '21.827')] [2024-10-18 23:45:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3104768. Throughput: 0: 202.8. Samples: 777962. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:45:19,811][04690] Avg episode reward: [(0, '21.717')] [2024-10-18 23:45:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 888.6). Total num frames: 3108864. Throughput: 0: 202.3. Samples: 778634. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:45:24,813][04690] Avg episode reward: [(0, '21.542')] [2024-10-18 23:45:26,372][05478] Updated weights for policy 0, policy_version 760 (0.2138) [2024-10-18 23:45:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3112960. Throughput: 0: 216.8. Samples: 780244. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:45:29,808][04690] Avg episode reward: [(0, '21.725')] [2024-10-18 23:45:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3117056. Throughput: 0: 222.9. Samples: 781610. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:45:34,808][04690] Avg episode reward: [(0, '22.172')] [2024-10-18 23:45:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3121152. Throughput: 0: 213.7. Samples: 781902. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:45:39,807][04690] Avg episode reward: [(0, '22.760')] [2024-10-18 23:45:40,876][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000763_3125248.pth... [2024-10-18 23:45:40,986][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000711_2912256.pth [2024-10-18 23:45:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3129344. Throughput: 0: 221.2. Samples: 783492. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:45:44,808][04690] Avg episode reward: [(0, '22.760')] [2024-10-18 23:45:49,806][04690] Fps is (10 sec: 1228.7, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3133440. Throughput: 0: 230.3. Samples: 784816. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:45:49,816][04690] Avg episode reward: [(0, '22.571')] [2024-10-18 23:45:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3137536. Throughput: 0: 224.6. Samples: 785584. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:45:54,812][04690] Avg episode reward: [(0, '23.234')] [2024-10-18 23:45:59,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3141632. Throughput: 0: 217.4. Samples: 786724. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:45:59,814][04690] Avg episode reward: [(0, '22.824')] [2024-10-18 23:46:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3145728. Throughput: 0: 225.2. Samples: 788096. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:46:04,808][04690] Avg episode reward: [(0, '24.089')] [2024-10-18 23:46:09,808][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3149824. Throughput: 0: 226.0. Samples: 788804. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:46:09,814][04690] Avg episode reward: [(0, '24.451')] [2024-10-18 23:46:12,497][05478] Updated weights for policy 0, policy_version 770 (0.0063) [2024-10-18 23:46:14,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3153920. Throughput: 0: 215.0. Samples: 789920. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2024-10-18 23:46:14,810][04690] Avg episode reward: [(0, '24.674')] [2024-10-18 23:46:17,213][05464] Saving new best policy, reward=24.674! [2024-10-18 23:46:19,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3158016. Throughput: 0: 218.9. Samples: 791460. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2024-10-18 23:46:19,818][04690] Avg episode reward: [(0, '24.724')] [2024-10-18 23:46:24,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3162112. Throughput: 0: 245.3. Samples: 792940. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:46:24,817][04690] Avg episode reward: [(0, '24.819')] [2024-10-18 23:46:25,011][05464] Saving new best policy, reward=24.724! [2024-10-18 23:46:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 3166208. Throughput: 0: 225.7. Samples: 793650. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:46:29,808][04690] Avg episode reward: [(0, '24.549')] [2024-10-18 23:46:30,892][05464] Saving new best policy, reward=24.819! [2024-10-18 23:46:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3170304. Throughput: 0: 220.7. Samples: 794748. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:46:34,808][04690] Avg episode reward: [(0, '24.225')] [2024-10-18 23:46:39,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3178496. Throughput: 0: 220.4. Samples: 795502. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:46:39,808][04690] Avg episode reward: [(0, '24.463')] [2024-10-18 23:46:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3182592. Throughput: 0: 229.7. Samples: 797060. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:46:44,808][04690] Avg episode reward: [(0, '24.719')] [2024-10-18 23:46:49,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3186688. Throughput: 0: 219.4. Samples: 797970. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:46:49,814][04690] Avg episode reward: [(0, '24.657')] [2024-10-18 23:46:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3190784. Throughput: 0: 237.5. Samples: 799492. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:46:54,810][04690] Avg episode reward: [(0, '25.809')] [2024-10-18 23:46:57,439][05464] Saving new best policy, reward=25.809! [2024-10-18 23:46:57,454][05478] Updated weights for policy 0, policy_version 780 (0.0062) [2024-10-18 23:46:59,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3194880. Throughput: 0: 227.7. Samples: 800168. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:46:59,814][04690] Avg episode reward: [(0, '26.095')] [2024-10-18 23:47:04,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3198976. Throughput: 0: 224.3. Samples: 801556. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:04,816][04690] Avg episode reward: [(0, '26.215')] [2024-10-18 23:47:07,729][05464] Saving new best policy, reward=26.095! [2024-10-18 23:47:07,914][05464] Saving new best policy, reward=26.215! [2024-10-18 23:47:09,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3203072. Throughput: 0: 202.7. Samples: 802062. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:09,811][04690] Avg episode reward: [(0, '25.702')] [2024-10-18 23:47:14,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3207168. Throughput: 0: 209.4. Samples: 803074. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:14,814][04690] Avg episode reward: [(0, '25.292')] [2024-10-18 23:47:19,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3211264. Throughput: 0: 207.5. Samples: 804084. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:19,817][04690] Avg episode reward: [(0, '25.343')] [2024-10-18 23:47:24,811][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3215360. Throughput: 0: 222.7. Samples: 805524. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:24,816][04690] Avg episode reward: [(0, '25.628')] [2024-10-18 23:47:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3219456. Throughput: 0: 201.6. Samples: 806134. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2024-10-18 23:47:29,812][04690] Avg episode reward: [(0, '26.005')] [2024-10-18 23:47:34,805][04690] Fps is (10 sec: 819.7, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3223552. Throughput: 0: 208.7. Samples: 807360. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2024-10-18 23:47:34,808][04690] Avg episode reward: [(0, '25.689')] [2024-10-18 23:47:37,157][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000788_3227648.pth... [2024-10-18 23:47:37,269][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000737_3018752.pth [2024-10-18 23:47:39,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3227648. Throughput: 0: 216.1. Samples: 809218. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2024-10-18 23:47:39,818][04690] Avg episode reward: [(0, '25.505')] [2024-10-18 23:47:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3231744. Throughput: 0: 208.2. Samples: 809538. Policy #0 lag: (min: 1.0, avg: 1.2, max: 2.0) [2024-10-18 23:47:44,808][04690] Avg episode reward: [(0, '25.190')] [2024-10-18 23:47:47,287][05478] Updated weights for policy 0, policy_version 790 (0.0549) [2024-10-18 23:47:49,806][04690] Fps is (10 sec: 819.3, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3235840. Throughput: 0: 202.1. Samples: 810650. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:49,808][04690] Avg episode reward: [(0, '25.700')] [2024-10-18 23:47:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3239936. Throughput: 0: 229.1. Samples: 812370. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:54,817][04690] Avg episode reward: [(0, '25.128')] [2024-10-18 23:47:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3248128. Throughput: 0: 228.9. Samples: 813376. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:47:59,814][04690] Avg episode reward: [(0, '25.732')] [2024-10-18 23:48:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.8). Total num frames: 3248128. Throughput: 0: 229.4. Samples: 814406. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:04,808][04690] Avg episode reward: [(0, '25.523')] [2024-10-18 23:48:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3256320. Throughput: 0: 206.3. Samples: 814808. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:09,811][04690] Avg episode reward: [(0, '24.849')] [2024-10-18 23:48:14,806][04690] Fps is (10 sec: 1228.7, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3260416. Throughput: 0: 230.4. Samples: 816504. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:14,808][04690] Avg episode reward: [(0, '24.141')] [2024-10-18 23:48:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3264512. Throughput: 0: 231.6. Samples: 817782. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:19,812][04690] Avg episode reward: [(0, '23.970')] [2024-10-18 23:48:24,814][04690] Fps is (10 sec: 818.5, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 3268608. Throughput: 0: 215.9. Samples: 818934. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:48:24,817][04690] Avg episode reward: [(0, '23.662')] [2024-10-18 23:48:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3272704. Throughput: 0: 224.9. Samples: 819658. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:48:29,807][04690] Avg episode reward: [(0, '23.218')] [2024-10-18 23:48:31,421][05478] Updated weights for policy 0, policy_version 800 (0.0531) [2024-10-18 23:48:33,762][05464] Signal inference workers to stop experience collection... (800 times) [2024-10-18 23:48:33,816][05478] InferenceWorker_p0-w0: stopping experience collection (800 times) [2024-10-18 23:48:34,805][04690] Fps is (10 sec: 819.9, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3276800. Throughput: 0: 243.2. Samples: 821594. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:34,808][04690] Avg episode reward: [(0, '22.847')] [2024-10-18 23:48:35,738][05464] Signal inference workers to resume experience collection... (800 times) [2024-10-18 23:48:35,748][05478] InferenceWorker_p0-w0: resuming experience collection (800 times) [2024-10-18 23:48:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3280896. Throughput: 0: 207.5. Samples: 821706. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:39,808][04690] Avg episode reward: [(0, '22.847')] [2024-10-18 23:48:44,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3284992. Throughput: 0: 214.5. Samples: 823030. Policy #0 lag: (min: 1.0, avg: 1.3, max: 2.0) [2024-10-18 23:48:44,811][04690] Avg episode reward: [(0, '22.120')] [2024-10-18 23:48:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3293184. Throughput: 0: 229.5. Samples: 824732. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:48:49,815][04690] Avg episode reward: [(0, '21.626')] [2024-10-18 23:48:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3297280. Throughput: 0: 235.7. Samples: 825414. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:48:54,808][04690] Avg episode reward: [(0, '21.392')] [2024-10-18 23:48:59,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3297280. Throughput: 0: 220.1. Samples: 826410. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:48:59,808][04690] Avg episode reward: [(0, '21.050')] [2024-10-18 23:49:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3305472. Throughput: 0: 222.9. Samples: 827812. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:49:04,808][04690] Avg episode reward: [(0, '21.547')] [2024-10-18 23:49:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3309568. Throughput: 0: 220.0. Samples: 828834. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:49:09,808][04690] Avg episode reward: [(0, '20.463')] [2024-10-18 23:49:14,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3313664. Throughput: 0: 227.8. Samples: 829908. Policy #0 lag: (min: 1.0, avg: 1.4, max: 3.0) [2024-10-18 23:49:14,814][04690] Avg episode reward: [(0, '20.500')] [2024-10-18 23:49:18,806][05478] Updated weights for policy 0, policy_version 810 (0.3465) [2024-10-18 23:49:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3317760. Throughput: 0: 208.3. Samples: 830968. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:49:19,810][04690] Avg episode reward: [(0, '20.453')] [2024-10-18 23:49:24,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.6, 300 sec: 874.7). Total num frames: 3321856. Throughput: 0: 227.6. Samples: 831946. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:49:24,816][04690] Avg episode reward: [(0, '20.457')] [2024-10-18 23:49:29,826][04690] Fps is (10 sec: 817.5, 60 sec: 887.2, 300 sec: 874.7). Total num frames: 3325952. Throughput: 0: 224.0. Samples: 833116. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:49:29,834][04690] Avg episode reward: [(0, '20.736')] [2024-10-18 23:49:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3330048. Throughput: 0: 206.6. Samples: 834028. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:49:34,812][04690] Avg episode reward: [(0, '21.071')] [2024-10-18 23:49:39,623][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000814_3334144.pth... [2024-10-18 23:49:39,732][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000763_3125248.pth [2024-10-18 23:49:39,805][04690] Fps is (10 sec: 820.9, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3334144. Throughput: 0: 204.4. Samples: 834614. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:49:39,808][04690] Avg episode reward: [(0, '20.299')] [2024-10-18 23:49:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3338240. Throughput: 0: 216.9. Samples: 836172. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:49:44,815][04690] Avg episode reward: [(0, '20.198')] [2024-10-18 23:49:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3342336. Throughput: 0: 215.6. Samples: 837514. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:49:49,819][04690] Avg episode reward: [(0, '19.174')] [2024-10-18 23:49:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3346432. Throughput: 0: 208.4. Samples: 838210. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:49:54,812][04690] Avg episode reward: [(0, '19.447')] [2024-10-18 23:49:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3350528. Throughput: 0: 206.4. Samples: 839194. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:49:59,808][04690] Avg episode reward: [(0, '19.147')] [2024-10-18 23:50:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3354624. Throughput: 0: 221.5. Samples: 840934. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:50:04,810][04690] Avg episode reward: [(0, '20.191')] [2024-10-18 23:50:05,886][05478] Updated weights for policy 0, policy_version 820 (0.1010) [2024-10-18 23:50:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3358720. Throughput: 0: 230.2. Samples: 842306. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:50:09,813][04690] Avg episode reward: [(0, '20.414')] [2024-10-18 23:50:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3362816. Throughput: 0: 211.3. Samples: 842620. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:50:14,813][04690] Avg episode reward: [(0, '20.661')] [2024-10-18 23:50:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3366912. Throughput: 0: 229.3. Samples: 844348. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:50:19,816][04690] Avg episode reward: [(0, '21.435')] [2024-10-18 23:50:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3375104. Throughput: 0: 230.9. Samples: 845006. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:50:24,810][04690] Avg episode reward: [(0, '22.384')] [2024-10-18 23:50:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.8, 300 sec: 888.6). Total num frames: 3379200. Throughput: 0: 222.3. Samples: 846174. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:50:29,807][04690] Avg episode reward: [(0, '22.433')] [2024-10-18 23:50:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3383296. Throughput: 0: 219.0. Samples: 847368. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:50:34,814][04690] Avg episode reward: [(0, '22.639')] [2024-10-18 23:50:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3387392. Throughput: 0: 220.7. Samples: 848142. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:50:39,808][04690] Avg episode reward: [(0, '23.854')] [2024-10-18 23:50:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3391488. Throughput: 0: 232.0. Samples: 849632. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:50:44,811][04690] Avg episode reward: [(0, '23.567')] [2024-10-18 23:50:49,809][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3395584. Throughput: 0: 215.0. Samples: 850608. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:50:49,813][04690] Avg episode reward: [(0, '23.431')] [2024-10-18 23:50:52,472][05478] Updated weights for policy 0, policy_version 830 (0.1643) [2024-10-18 23:50:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3399680. Throughput: 0: 201.6. Samples: 851380. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:50:54,813][04690] Avg episode reward: [(0, '24.221')] [2024-10-18 23:50:59,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3403776. Throughput: 0: 228.3. Samples: 852892. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:50:59,808][04690] Avg episode reward: [(0, '24.116')] [2024-10-18 23:51:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3407872. Throughput: 0: 222.0. Samples: 854340. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:04,809][04690] Avg episode reward: [(0, '24.654')] [2024-10-18 23:51:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3411968. Throughput: 0: 232.4. Samples: 855462. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:09,815][04690] Avg episode reward: [(0, '24.401')] [2024-10-18 23:51:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3420160. Throughput: 0: 221.1. Samples: 856122. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:51:14,811][04690] Avg episode reward: [(0, '24.689')] [2024-10-18 23:51:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3424256. Throughput: 0: 228.4. Samples: 857644. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:19,809][04690] Avg episode reward: [(0, '24.883')] [2024-10-18 23:51:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3428352. Throughput: 0: 226.4. Samples: 858332. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:24,814][04690] Avg episode reward: [(0, '25.194')] [2024-10-18 23:51:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3432448. Throughput: 0: 221.6. Samples: 859606. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:29,811][04690] Avg episode reward: [(0, '25.012')] [2024-10-18 23:51:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3436544. Throughput: 0: 227.4. Samples: 860838. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:34,808][04690] Avg episode reward: [(0, '24.907')] [2024-10-18 23:51:36,675][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000840_3440640.pth... [2024-10-18 23:51:36,691][05478] Updated weights for policy 0, policy_version 840 (0.2458) [2024-10-18 23:51:36,807][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000788_3227648.pth [2024-10-18 23:51:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3440640. Throughput: 0: 243.1. Samples: 862318. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:51:39,811][04690] Avg episode reward: [(0, '25.032')] [2024-10-18 23:51:44,811][04690] Fps is (10 sec: 818.7, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3444736. Throughput: 0: 216.9. Samples: 862654. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:51:44,814][04690] Avg episode reward: [(0, '25.722')] [2024-10-18 23:51:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3448832. Throughput: 0: 206.1. Samples: 863616. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:49,811][04690] Avg episode reward: [(0, '25.437')] [2024-10-18 23:51:54,805][04690] Fps is (10 sec: 819.7, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3452928. Throughput: 0: 209.8. Samples: 864902. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:54,808][04690] Avg episode reward: [(0, '25.850')] [2024-10-18 23:51:59,809][04690] Fps is (10 sec: 818.9, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3457024. Throughput: 0: 213.0. Samples: 865708. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:51:59,813][04690] Avg episode reward: [(0, '25.714')] [2024-10-18 23:52:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3461120. Throughput: 0: 202.2. Samples: 866744. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:52:04,815][04690] Avg episode reward: [(0, '26.084')] [2024-10-18 23:52:09,805][04690] Fps is (10 sec: 819.5, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3465216. Throughput: 0: 203.6. Samples: 867494. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:52:09,808][04690] Avg episode reward: [(0, '25.515')] [2024-10-18 23:52:14,807][04690] Fps is (10 sec: 819.0, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3469312. Throughput: 0: 208.1. Samples: 868970. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:52:14,811][04690] Avg episode reward: [(0, '24.534')] [2024-10-18 23:52:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.8). Total num frames: 3473408. Throughput: 0: 208.7. Samples: 870230. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:52:19,814][04690] Avg episode reward: [(0, '24.397')] [2024-10-18 23:52:24,805][04690] Fps is (10 sec: 819.4, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3477504. Throughput: 0: 191.6. Samples: 870942. Policy #0 lag: (min: 1.0, avg: 1.4, max: 2.0) [2024-10-18 23:52:24,808][04690] Avg episode reward: [(0, '23.570')] [2024-10-18 23:52:26,899][05478] Updated weights for policy 0, policy_version 850 (0.1103) [2024-10-18 23:52:29,235][05464] Signal inference workers to stop experience collection... (850 times) [2024-10-18 23:52:29,307][05478] InferenceWorker_p0-w0: stopping experience collection (850 times) [2024-10-18 23:52:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3481600. Throughput: 0: 214.8. Samples: 872320. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:29,813][04690] Avg episode reward: [(0, '23.570')] [2024-10-18 23:52:30,691][05464] Signal inference workers to resume experience collection... (850 times) [2024-10-18 23:52:30,693][05478] InferenceWorker_p0-w0: resuming experience collection (850 times) [2024-10-18 23:52:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3485696. Throughput: 0: 227.6. Samples: 873860. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:34,808][04690] Avg episode reward: [(0, '24.323')] [2024-10-18 23:52:39,809][04690] Fps is (10 sec: 818.9, 60 sec: 819.1, 300 sec: 874.7). Total num frames: 3489792. Throughput: 0: 207.0. Samples: 874220. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:39,812][04690] Avg episode reward: [(0, '23.603')] [2024-10-18 23:52:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.3, 300 sec: 874.7). Total num frames: 3493888. Throughput: 0: 220.4. Samples: 875624. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:44,818][04690] Avg episode reward: [(0, '23.603')] [2024-10-18 23:52:49,805][04690] Fps is (10 sec: 1229.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3502080. Throughput: 0: 228.8. Samples: 877040. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:49,808][04690] Avg episode reward: [(0, '24.378')] [2024-10-18 23:52:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3506176. Throughput: 0: 232.4. Samples: 877954. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:54,813][04690] Avg episode reward: [(0, '24.576')] [2024-10-18 23:52:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3510272. Throughput: 0: 219.9. Samples: 878864. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:52:59,813][04690] Avg episode reward: [(0, '24.042')] [2024-10-18 23:53:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3514368. Throughput: 0: 222.6. Samples: 880248. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:53:04,808][04690] Avg episode reward: [(0, '24.646')] [2024-10-18 23:53:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3518464. Throughput: 0: 228.6. Samples: 881228. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:53:09,815][04690] Avg episode reward: [(0, '24.676')] [2024-10-18 23:53:13,105][05478] Updated weights for policy 0, policy_version 860 (0.1029) [2024-10-18 23:53:14,807][04690] Fps is (10 sec: 819.0, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3522560. Throughput: 0: 220.7. Samples: 882250. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:53:14,813][04690] Avg episode reward: [(0, '24.597')] [2024-10-18 23:53:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 3526656. Throughput: 0: 216.7. Samples: 883610. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:53:19,811][04690] Avg episode reward: [(0, '24.419')] [2024-10-18 23:53:24,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3530752. Throughput: 0: 246.1. Samples: 885292. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:53:24,812][04690] Avg episode reward: [(0, '24.731')] [2024-10-18 23:53:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3534848. Throughput: 0: 228.3. Samples: 885898. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:53:29,808][04690] Avg episode reward: [(0, '25.226')] [2024-10-18 23:53:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3538944. Throughput: 0: 218.2. Samples: 886858. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:53:34,812][04690] Avg episode reward: [(0, '25.429')] [2024-10-18 23:53:36,078][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000865_3543040.pth... [2024-10-18 23:53:36,189][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000814_3334144.pth [2024-10-18 23:53:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3543040. Throughput: 0: 232.9. Samples: 888434. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:53:39,812][04690] Avg episode reward: [(0, '24.919')] [2024-10-18 23:53:44,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 3551232. Throughput: 0: 234.4. Samples: 889414. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:53:44,815][04690] Avg episode reward: [(0, '24.081')] [2024-10-18 23:53:49,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3555328. Throughput: 0: 226.4. Samples: 890434. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:53:49,813][04690] Avg episode reward: [(0, '24.203')] [2024-10-18 23:53:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3559424. Throughput: 0: 217.2. Samples: 891000. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:53:54,815][04690] Avg episode reward: [(0, '24.669')] [2024-10-18 23:53:58,245][05478] Updated weights for policy 0, policy_version 870 (0.2139) [2024-10-18 23:53:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3563520. Throughput: 0: 227.3. Samples: 892478. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:53:59,808][04690] Avg episode reward: [(0, '24.421')] [2024-10-18 23:54:04,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3567616. Throughput: 0: 229.4. Samples: 893932. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:54:04,816][04690] Avg episode reward: [(0, '24.439')] [2024-10-18 23:54:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3571712. Throughput: 0: 203.6. Samples: 894454. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:54:09,811][04690] Avg episode reward: [(0, '24.096')] [2024-10-18 23:54:14,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 860.9). Total num frames: 3571712. Throughput: 0: 212.0. Samples: 895436. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:54:14,808][04690] Avg episode reward: [(0, '24.076')] [2024-10-18 23:54:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3579904. Throughput: 0: 214.8. Samples: 896522. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:54:19,817][04690] Avg episode reward: [(0, '24.294')] [2024-10-18 23:54:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 3584000. Throughput: 0: 198.8. Samples: 897380. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:54:24,811][04690] Avg episode reward: [(0, '24.813')] [2024-10-18 23:54:29,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3588096. Throughput: 0: 204.1. Samples: 898600. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:54:29,808][04690] Avg episode reward: [(0, '24.561')] [2024-10-18 23:54:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3592192. Throughput: 0: 204.8. Samples: 899650. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:54:34,811][04690] Avg episode reward: [(0, '24.415')] [2024-10-18 23:54:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3596288. Throughput: 0: 215.0. Samples: 900674. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:54:39,812][04690] Avg episode reward: [(0, '24.027')] [2024-10-18 23:54:44,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3600384. Throughput: 0: 214.3. Samples: 902122. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:54:44,813][04690] Avg episode reward: [(0, '23.598')] [2024-10-18 23:54:47,030][05478] Updated weights for policy 0, policy_version 880 (0.1037) [2024-10-18 23:54:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3604480. Throughput: 0: 204.9. Samples: 903150. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:54:49,808][04690] Avg episode reward: [(0, '23.318')] [2024-10-18 23:54:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3608576. Throughput: 0: 230.3. Samples: 904816. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:54:54,808][04690] Avg episode reward: [(0, '23.349')] [2024-10-18 23:54:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3616768. Throughput: 0: 225.5. Samples: 905582. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:54:59,808][04690] Avg episode reward: [(0, '23.383')] [2024-10-18 23:55:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3616768. Throughput: 0: 229.5. Samples: 906848. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:04,808][04690] Avg episode reward: [(0, '23.009')] [2024-10-18 23:55:09,805][04690] Fps is (10 sec: 409.6, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3620864. Throughput: 0: 233.1. Samples: 907868. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:09,808][04690] Avg episode reward: [(0, '22.923')] [2024-10-18 23:55:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3629056. Throughput: 0: 228.4. Samples: 908878. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:14,808][04690] Avg episode reward: [(0, '22.827')] [2024-10-18 23:55:19,806][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3633152. Throughput: 0: 232.4. Samples: 910110. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:19,811][04690] Avg episode reward: [(0, '22.902')] [2024-10-18 23:55:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3637248. Throughput: 0: 225.7. Samples: 910832. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:24,814][04690] Avg episode reward: [(0, '22.622')] [2024-10-18 23:55:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3641344. Throughput: 0: 218.7. Samples: 911962. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:29,815][04690] Avg episode reward: [(0, '22.370')] [2024-10-18 23:55:31,991][05478] Updated weights for policy 0, policy_version 890 (0.1988) [2024-10-18 23:55:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3645440. Throughput: 0: 234.1. Samples: 913684. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:34,816][04690] Avg episode reward: [(0, '21.771')] [2024-10-18 23:55:39,811][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3649536. Throughput: 0: 206.8. Samples: 914122. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:39,819][04690] Avg episode reward: [(0, '21.868')] [2024-10-18 23:55:41,991][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000892_3653632.pth... [2024-10-18 23:55:42,178][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000840_3440640.pth [2024-10-18 23:55:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3653632. Throughput: 0: 214.0. Samples: 915214. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:55:44,808][04690] Avg episode reward: [(0, '22.566')] [2024-10-18 23:55:49,805][04690] Fps is (10 sec: 819.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3657728. Throughput: 0: 225.6. Samples: 917002. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:55:49,813][04690] Avg episode reward: [(0, '23.143')] [2024-10-18 23:55:54,808][04690] Fps is (10 sec: 1228.5, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3665920. Throughput: 0: 217.8. Samples: 917670. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:54,811][04690] Avg episode reward: [(0, '22.989')] [2024-10-18 23:55:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3665920. Throughput: 0: 222.4. Samples: 918884. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:55:59,811][04690] Avg episode reward: [(0, '23.230')] [2024-10-18 23:56:04,805][04690] Fps is (10 sec: 819.4, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3674112. Throughput: 0: 222.5. Samples: 920122. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:56:04,808][04690] Avg episode reward: [(0, '23.442')] [2024-10-18 23:56:09,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 874.7). Total num frames: 3678208. Throughput: 0: 242.6. Samples: 921748. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:56:09,808][04690] Avg episode reward: [(0, '23.297')] [2024-10-18 23:56:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3682304. Throughput: 0: 226.8. Samples: 922170. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:56:14,808][04690] Avg episode reward: [(0, '23.150')] [2024-10-18 23:56:18,653][05478] Updated weights for policy 0, policy_version 900 (0.0735) [2024-10-18 23:56:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3686400. Throughput: 0: 211.3. Samples: 923194. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:19,813][04690] Avg episode reward: [(0, '22.544')] [2024-10-18 23:56:21,075][05464] Signal inference workers to stop experience collection... (900 times) [2024-10-18 23:56:21,112][05478] InferenceWorker_p0-w0: stopping experience collection (900 times) [2024-10-18 23:56:22,481][05464] Signal inference workers to resume experience collection... (900 times) [2024-10-18 23:56:22,482][05478] InferenceWorker_p0-w0: resuming experience collection (900 times) [2024-10-18 23:56:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3690496. Throughput: 0: 223.2. Samples: 924166. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:24,813][04690] Avg episode reward: [(0, '22.731')] [2024-10-18 23:56:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3694592. Throughput: 0: 222.4. Samples: 925220. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:29,819][04690] Avg episode reward: [(0, '22.745')] [2024-10-18 23:56:34,810][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3698688. Throughput: 0: 205.6. Samples: 926256. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:34,814][04690] Avg episode reward: [(0, '22.780')] [2024-10-18 23:56:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.8). Total num frames: 3702784. Throughput: 0: 205.4. Samples: 926914. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:56:39,808][04690] Avg episode reward: [(0, '22.571')] [2024-10-18 23:56:44,805][04690] Fps is (10 sec: 819.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3706880. Throughput: 0: 208.6. Samples: 928270. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:56:44,808][04690] Avg episode reward: [(0, '22.747')] [2024-10-18 23:56:49,810][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3710976. Throughput: 0: 212.8. Samples: 929700. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:49,814][04690] Avg episode reward: [(0, '22.860')] [2024-10-18 23:56:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3715072. Throughput: 0: 191.2. Samples: 930354. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:54,808][04690] Avg episode reward: [(0, '22.609')] [2024-10-18 23:56:59,805][04690] Fps is (10 sec: 819.6, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3719168. Throughput: 0: 205.0. Samples: 931394. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:56:59,813][04690] Avg episode reward: [(0, '23.282')] [2024-10-18 23:57:04,806][04690] Fps is (10 sec: 819.1, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3723264. Throughput: 0: 219.1. Samples: 933052. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:04,816][04690] Avg episode reward: [(0, '23.273')] [2024-10-18 23:57:05,763][05478] Updated weights for policy 0, policy_version 910 (0.2643) [2024-10-18 23:57:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3727360. Throughput: 0: 228.8. Samples: 934460. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:57:09,808][04690] Avg episode reward: [(0, '22.839')] [2024-10-18 23:57:14,809][04690] Fps is (10 sec: 819.0, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3731456. Throughput: 0: 215.0. Samples: 934896. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:57:14,815][04690] Avg episode reward: [(0, '22.446')] [2024-10-18 23:57:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3735552. Throughput: 0: 227.4. Samples: 936486. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:57:19,813][04690] Avg episode reward: [(0, '22.306')] [2024-10-18 23:57:24,805][04690] Fps is (10 sec: 1229.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3743744. Throughput: 0: 229.3. Samples: 937234. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:24,808][04690] Avg episode reward: [(0, '22.033')] [2024-10-18 23:57:29,806][04690] Fps is (10 sec: 1228.7, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3747840. Throughput: 0: 228.9. Samples: 938572. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:29,813][04690] Avg episode reward: [(0, '21.939')] [2024-10-18 23:57:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3751936. Throughput: 0: 219.0. Samples: 939552. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:34,808][04690] Avg episode reward: [(0, '22.060')] [2024-10-18 23:57:37,917][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000917_3756032.pth... [2024-10-18 23:57:38,030][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000865_3543040.pth [2024-10-18 23:57:39,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3756032. Throughput: 0: 245.3. Samples: 941392. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:39,816][04690] Avg episode reward: [(0, '22.172')] [2024-10-18 23:57:44,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3760128. Throughput: 0: 236.2. Samples: 942024. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:44,812][04690] Avg episode reward: [(0, '21.116')] [2024-10-18 23:57:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3764224. Throughput: 0: 222.6. Samples: 943070. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:57:49,809][04690] Avg episode reward: [(0, '21.445')] [2024-10-18 23:57:52,328][05478] Updated weights for policy 0, policy_version 920 (0.0518) [2024-10-18 23:57:54,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3768320. Throughput: 0: 228.8. Samples: 944756. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:57:54,814][04690] Avg episode reward: [(0, '21.699')] [2024-10-18 23:57:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3772416. Throughput: 0: 234.4. Samples: 945444. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:57:59,812][04690] Avg episode reward: [(0, '21.494')] [2024-10-18 23:58:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3776512. Throughput: 0: 228.5. Samples: 946770. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:58:04,808][04690] Avg episode reward: [(0, '21.720')] [2024-10-18 23:58:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3780608. Throughput: 0: 234.1. Samples: 947770. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:58:09,815][04690] Avg episode reward: [(0, '22.737')] [2024-10-18 23:58:14,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.8, 300 sec: 888.6). Total num frames: 3788800. Throughput: 0: 227.5. Samples: 948808. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:58:14,808][04690] Avg episode reward: [(0, '23.159')] [2024-10-18 23:58:19,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3792896. Throughput: 0: 231.2. Samples: 949954. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:58:19,813][04690] Avg episode reward: [(0, '22.523')] [2024-10-18 23:58:24,808][04690] Fps is (10 sec: 819.0, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 3796992. Throughput: 0: 205.4. Samples: 950634. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:58:24,810][04690] Avg episode reward: [(0, '22.888')] [2024-10-18 23:58:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3801088. Throughput: 0: 217.1. Samples: 951792. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:58:29,808][04690] Avg episode reward: [(0, '23.208')] [2024-10-18 23:58:34,805][04690] Fps is (10 sec: 819.4, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3805184. Throughput: 0: 228.0. Samples: 953330. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:58:34,807][04690] Avg episode reward: [(0, '23.714')] [2024-10-18 23:58:36,179][05478] Updated weights for policy 0, policy_version 930 (0.1480) [2024-10-18 23:58:39,806][04690] Fps is (10 sec: 819.1, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 3809280. Throughput: 0: 209.3. Samples: 954176. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:58:39,817][04690] Avg episode reward: [(0, '24.175')] [2024-10-18 23:58:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3813376. Throughput: 0: 216.6. Samples: 955192. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:58:44,813][04690] Avg episode reward: [(0, '24.162')] [2024-10-18 23:58:49,805][04690] Fps is (10 sec: 819.3, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3817472. Throughput: 0: 207.3. Samples: 956098. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:58:49,812][04690] Avg episode reward: [(0, '24.060')] [2024-10-18 23:58:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3821568. Throughput: 0: 200.6. Samples: 956798. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-18 23:58:54,810][04690] Avg episode reward: [(0, '23.957')] [2024-10-18 23:58:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3825664. Throughput: 0: 202.9. Samples: 957940. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:58:59,810][04690] Avg episode reward: [(0, '24.097')] [2024-10-18 23:59:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3829760. Throughput: 0: 202.5. Samples: 959066. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:59:04,813][04690] Avg episode reward: [(0, '24.989')] [2024-10-18 23:59:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3833856. Throughput: 0: 208.7. Samples: 960024. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:59:09,809][04690] Avg episode reward: [(0, '24.449')] [2024-10-18 23:59:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3837952. Throughput: 0: 220.2. Samples: 961700. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:59:14,813][04690] Avg episode reward: [(0, '24.921')] [2024-10-18 23:59:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3842048. Throughput: 0: 205.3. Samples: 962570. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:59:19,808][04690] Avg episode reward: [(0, '24.817')] [2024-10-18 23:59:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3846144. Throughput: 0: 221.5. Samples: 964144. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:59:24,808][04690] Avg episode reward: [(0, '24.584')] [2024-10-18 23:59:25,525][05478] Updated weights for policy 0, policy_version 940 (0.1222) [2024-10-18 23:59:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3854336. Throughput: 0: 220.2. Samples: 965102. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-18 23:59:29,809][04690] Avg episode reward: [(0, '25.067')] [2024-10-18 23:59:34,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3858432. Throughput: 0: 224.5. Samples: 966202. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:59:34,808][04690] Avg episode reward: [(0, '24.826')] [2024-10-18 23:59:39,723][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000943_3862528.pth... [2024-10-18 23:59:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3862528. Throughput: 0: 222.7. Samples: 966820. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:59:39,809][04690] Avg episode reward: [(0, '25.855')] [2024-10-18 23:59:39,860][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000892_3653632.pth [2024-10-18 23:59:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3866624. Throughput: 0: 228.9. Samples: 968240. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:59:44,813][04690] Avg episode reward: [(0, '26.711')] [2024-10-18 23:59:47,430][05464] Saving new best policy, reward=26.711! [2024-10-18 23:59:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3870720. Throughput: 0: 238.9. Samples: 969816. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-18 23:59:49,808][04690] Avg episode reward: [(0, '26.457')] [2024-10-18 23:59:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3874816. Throughput: 0: 228.9. Samples: 970326. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:59:54,814][04690] Avg episode reward: [(0, '25.526')] [2024-10-18 23:59:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3878912. Throughput: 0: 214.4. Samples: 971346. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-18 23:59:59,808][04690] Avg episode reward: [(0, '25.401')] [2024-10-19 00:00:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3883008. Throughput: 0: 236.1. Samples: 973196. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:00:04,808][04690] Avg episode reward: [(0, '23.976')] [2024-10-19 00:00:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3887104. Throughput: 0: 227.2. Samples: 974368. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:00:09,808][04690] Avg episode reward: [(0, '23.934')] [2024-10-19 00:00:10,984][05478] Updated weights for policy 0, policy_version 950 (0.0982) [2024-10-19 00:00:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3891200. Throughput: 0: 215.1. Samples: 974780. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:00:14,808][04690] Avg episode reward: [(0, '24.598')] [2024-10-19 00:00:14,978][05464] Signal inference workers to stop experience collection... (950 times) [2024-10-19 00:00:15,026][05478] InferenceWorker_p0-w0: stopping experience collection (950 times) [2024-10-19 00:00:16,098][05464] Signal inference workers to resume experience collection... (950 times) [2024-10-19 00:00:16,102][05478] InferenceWorker_p0-w0: resuming experience collection (950 times) [2024-10-19 00:00:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3895296. Throughput: 0: 222.7. Samples: 976222. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:00:19,808][04690] Avg episode reward: [(0, '23.829')] [2024-10-19 00:00:24,805][04690] Fps is (10 sec: 1228.8, 60 sec: 955.7, 300 sec: 888.6). Total num frames: 3903488. Throughput: 0: 228.8. Samples: 977114. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:00:24,809][04690] Avg episode reward: [(0, '23.480')] [2024-10-19 00:00:29,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3907584. Throughput: 0: 227.3. Samples: 978468. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-19 00:00:29,817][04690] Avg episode reward: [(0, '23.535')] [2024-10-19 00:00:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3911680. Throughput: 0: 215.0. Samples: 979490. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-19 00:00:34,808][04690] Avg episode reward: [(0, '22.732')] [2024-10-19 00:00:39,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3915776. Throughput: 0: 240.9. Samples: 981166. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-19 00:00:39,807][04690] Avg episode reward: [(0, '22.552')] [2024-10-19 00:00:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3919872. Throughput: 0: 233.9. Samples: 981870. Policy #0 lag: (min: 1.0, avg: 1.5, max: 2.0) [2024-10-19 00:00:44,809][04690] Avg episode reward: [(0, '22.680')] [2024-10-19 00:00:49,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3923968. Throughput: 0: 214.4. Samples: 982842. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-19 00:00:49,814][04690] Avg episode reward: [(0, '22.893')] [2024-10-19 00:00:54,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3928064. Throughput: 0: 225.9. Samples: 984532. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-19 00:00:54,813][04690] Avg episode reward: [(0, '24.070')] [2024-10-19 00:00:56,819][05478] Updated weights for policy 0, policy_version 960 (0.1003) [2024-10-19 00:00:59,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3932160. Throughput: 0: 227.7. Samples: 985026. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:00:59,817][04690] Avg episode reward: [(0, '24.344')] [2024-10-19 00:01:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3936256. Throughput: 0: 232.2. Samples: 986670. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:01:04,811][04690] Avg episode reward: [(0, '23.979')] [2024-10-19 00:01:09,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3940352. Throughput: 0: 216.6. Samples: 986860. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:01:09,808][04690] Avg episode reward: [(0, '24.548')] [2024-10-19 00:01:14,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3944448. Throughput: 0: 205.3. Samples: 987706. Policy #0 lag: (min: 1.0, avg: 1.5, max: 3.0) [2024-10-19 00:01:14,813][04690] Avg episode reward: [(0, '24.548')] [2024-10-19 00:01:19,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 874.7). Total num frames: 3948544. Throughput: 0: 216.6. Samples: 989236. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:01:19,817][04690] Avg episode reward: [(0, '24.516')] [2024-10-19 00:01:24,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3952640. Throughput: 0: 193.6. Samples: 989880. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:01:24,813][04690] Avg episode reward: [(0, '25.064')] [2024-10-19 00:01:29,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.8). Total num frames: 3956736. Throughput: 0: 200.3. Samples: 990882. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-19 00:01:29,819][04690] Avg episode reward: [(0, '24.965')] [2024-10-19 00:01:34,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3960832. Throughput: 0: 216.9. Samples: 992604. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-19 00:01:34,808][04690] Avg episode reward: [(0, '25.597')] [2024-10-19 00:01:39,589][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000969_3969024.pth... [2024-10-19 00:01:39,709][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000917_3756032.pth [2024-10-19 00:01:39,808][04690] Fps is (10 sec: 1228.5, 60 sec: 887.4, 300 sec: 888.6). Total num frames: 3969024. Throughput: 0: 194.4. Samples: 993280. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-19 00:01:39,810][04690] Avg episode reward: [(0, '24.254')] [2024-10-19 00:01:44,805][04690] Fps is (10 sec: 819.2, 60 sec: 819.2, 300 sec: 874.8). Total num frames: 3969024. Throughput: 0: 214.1. Samples: 994660. Policy #0 lag: (min: 1.0, avg: 1.6, max: 2.0) [2024-10-19 00:01:44,810][04690] Avg episode reward: [(0, '24.211')] [2024-10-19 00:01:46,213][05478] Updated weights for policy 0, policy_version 970 (0.0653) [2024-10-19 00:01:49,805][04690] Fps is (10 sec: 409.7, 60 sec: 819.2, 300 sec: 874.7). Total num frames: 3973120. Throughput: 0: 201.8. Samples: 995750. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:01:49,816][04690] Avg episode reward: [(0, '24.857')] [2024-10-19 00:01:54,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3981312. Throughput: 0: 228.4. Samples: 997138. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:01:54,810][04690] Avg episode reward: [(0, '24.976')] [2024-10-19 00:01:59,805][04690] Fps is (10 sec: 1228.8, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3985408. Throughput: 0: 229.0. Samples: 998012. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:01:59,811][04690] Avg episode reward: [(0, '24.890')] [2024-10-19 00:02:04,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3989504. Throughput: 0: 217.7. Samples: 999034. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:02:04,811][04690] Avg episode reward: [(0, '25.316')] [2024-10-19 00:02:09,806][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3993600. Throughput: 0: 241.0. Samples: 1000726. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:02:09,809][04690] Avg episode reward: [(0, '25.683')] [2024-10-19 00:02:14,805][04690] Fps is (10 sec: 819.2, 60 sec: 887.5, 300 sec: 888.6). Total num frames: 3997696. Throughput: 0: 234.8. Samples: 1001448. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:02:14,818][04690] Avg episode reward: [(0, '25.750')] [2024-10-19 00:02:19,810][04690] Fps is (10 sec: 818.8, 60 sec: 887.4, 300 sec: 874.7). Total num frames: 4001792. Throughput: 0: 222.9. Samples: 1002634. Policy #0 lag: (min: 1.0, avg: 1.6, max: 3.0) [2024-10-19 00:02:19,816][04690] Avg episode reward: [(0, '25.441')] [2024-10-19 00:02:21,706][05464] Stopping Batcher_0... [2024-10-19 00:02:21,710][05464] Loop batcher_evt_loop terminating... [2024-10-19 00:02:21,710][04690] Component Batcher_0 stopped! [2024-10-19 00:02:22,160][05478] Weights refcount: 2 0 [2024-10-19 00:02:22,165][05478] Stopping InferenceWorker_p0-w0... [2024-10-19 00:02:22,169][05478] Loop inference_proc0-0_evt_loop terminating... [2024-10-19 00:02:22,169][04690] Component InferenceWorker_p0-w0 stopped! [2024-10-19 00:02:22,457][04690] Component RolloutWorker_w7 stopped! [2024-10-19 00:02:22,458][05485] Stopping RolloutWorker_w7... [2024-10-19 00:02:22,513][04690] Component RolloutWorker_w1 stopped! [2024-10-19 00:02:22,532][05479] Stopping RolloutWorker_w1... [2024-10-19 00:02:22,542][05485] Loop rollout_proc7_evt_loop terminating... [2024-10-19 00:02:22,555][05479] Loop rollout_proc1_evt_loop terminating... [2024-10-19 00:02:22,629][04690] Component RolloutWorker_w3 stopped! [2024-10-19 00:02:22,666][04690] Component RolloutWorker_w5 stopped! [2024-10-19 00:02:22,638][05480] Stopping RolloutWorker_w3... [2024-10-19 00:02:22,667][05483] Stopping RolloutWorker_w5... [2024-10-19 00:02:22,704][05483] Loop rollout_proc5_evt_loop terminating... [2024-10-19 00:02:22,703][05480] Loop rollout_proc3_evt_loop terminating... [2024-10-19 00:02:22,859][04690] Component RolloutWorker_w2 stopped! [2024-10-19 00:02:22,880][04690] Component RolloutWorker_w6 stopped! [2024-10-19 00:02:22,869][05481] Stopping RolloutWorker_w2... [2024-10-19 00:02:22,933][05481] Loop rollout_proc2_evt_loop terminating... [2024-10-19 00:02:22,935][04690] Component RolloutWorker_w0 stopped! [2024-10-19 00:02:22,886][05484] Stopping RolloutWorker_w6... [2024-10-19 00:02:22,948][05484] Loop rollout_proc6_evt_loop terminating... [2024-10-19 00:02:22,967][04690] Component RolloutWorker_w4 stopped! [2024-10-19 00:02:22,941][05477] Stopping RolloutWorker_w0... [2024-10-19 00:02:22,978][05482] Stopping RolloutWorker_w4... [2024-10-19 00:02:23,012][05482] Loop rollout_proc4_evt_loop terminating... [2024-10-19 00:02:23,000][05477] Loop rollout_proc0_evt_loop terminating... [2024-10-19 00:02:27,271][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2024-10-19 00:02:27,382][05464] Removing /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000943_3862528.pth [2024-10-19 00:02:27,399][05464] Saving /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2024-10-19 00:02:27,593][05464] Stopping LearnerWorker_p0... [2024-10-19 00:02:27,594][05464] Loop learner_proc0_evt_loop terminating... [2024-10-19 00:02:27,593][04690] Component LearnerWorker_p0 stopped! [2024-10-19 00:02:27,598][04690] Waiting for process learner_proc0 to stop... [2024-10-19 00:02:28,380][04690] Waiting for process inference_proc0-0 to join... [2024-10-19 00:02:28,384][04690] Waiting for process rollout_proc0 to join... [2024-10-19 00:02:28,389][04690] Waiting for process rollout_proc1 to join... [2024-10-19 00:02:28,396][04690] Waiting for process rollout_proc2 to join... [2024-10-19 00:02:28,402][04690] Waiting for process rollout_proc3 to join... [2024-10-19 00:02:28,406][04690] Waiting for process rollout_proc4 to join... [2024-10-19 00:02:28,410][04690] Waiting for process rollout_proc5 to join... [2024-10-19 00:02:28,416][04690] Waiting for process rollout_proc6 to join... [2024-10-19 00:02:28,419][04690] Waiting for process rollout_proc7 to join... [2024-10-19 00:02:28,424][04690] Batcher 0 profile tree view: batching: 21.1572, releasing_batches: 0.3506 [2024-10-19 00:02:28,426][04690] InferenceWorker_p0-w0 profile tree view: wait_policy: 0.0052 wait_policy_total: 47.7508 update_model: 129.9132 weight_update: 0.1769 one_step: 0.1966 handle_policy_step: 2982.8958 deserialize: 94.4265, stack: 15.5977, obs_to_device_normalize: 485.0916, forward: 2193.1101, send_messages: 69.8722 prepare_outputs: 36.0803 to_cpu: 3.5450 [2024-10-19 00:02:28,429][04690] Learner 0 profile tree view: misc: 0.0158, prepare_batch: 1274.6872 train: 3263.4361 epoch_init: 0.0076, minibatch_init: 0.0110, losses_postprocess: 0.1571, kl_divergence: 0.5553, after_optimizer: 2.6213 calculate_losses: 1594.1997 losses_init: 0.0044, forward_head: 1419.8722, bptt_initial: 4.5054, tail: 3.3777, advantages_returns: 0.2165, losses: 1.5991 bptt: 163.9603 bptt_forward_core: 162.8808 update: 1665.1861 clip: 3.9528 [2024-10-19 00:02:28,430][04690] RolloutWorker_w0 profile tree view: wait_for_trajectories: 0.7474, enqueue_policy_requests: 58.5719, env_step: 1718.8782, overhead: 40.3656, complete_rollouts: 16.5092 save_policy_outputs: 49.0667 split_output_tensors: 16.5794 [2024-10-19 00:02:28,432][04690] RolloutWorker_w7 profile tree view: wait_for_trajectories: 0.8212, enqueue_policy_requests: 61.0612, env_step: 1680.7220, overhead: 44.7416, complete_rollouts: 17.6057 save_policy_outputs: 49.2472 split_output_tensors: 16.3751 [2024-10-19 00:02:28,436][04690] Loop Runner_EvtLoop terminating... [2024-10-19 00:02:28,439][04690] Runner profile tree view: main_loop: 4611.2620 [2024-10-19 00:02:28,444][04690] Collected {0: 4009984}, FPS: 869.6 [2024-10-19 00:02:30,640][04690] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-10-19 00:02:30,644][04690] Overriding arg 'num_workers' with value 1 passed from command line [2024-10-19 00:02:30,651][04690] Adding new argument 'no_render'=True that is not in the saved config file! [2024-10-19 00:02:30,653][04690] Adding new argument 'save_video'=True that is not in the saved config file! [2024-10-19 00:02:30,656][04690] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-10-19 00:02:30,658][04690] Adding new argument 'video_name'=None that is not in the saved config file! [2024-10-19 00:02:30,660][04690] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! [2024-10-19 00:02:30,665][04690] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-10-19 00:02:30,666][04690] Adding new argument 'push_to_hub'=False that is not in the saved config file! [2024-10-19 00:02:30,668][04690] Adding new argument 'hf_repository'=None that is not in the saved config file! [2024-10-19 00:02:30,670][04690] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-10-19 00:02:30,671][04690] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-10-19 00:02:30,673][04690] Adding new argument 'train_script'=None that is not in the saved config file! [2024-10-19 00:02:30,675][04690] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-10-19 00:02:30,676][04690] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-10-19 00:02:30,703][04690] Doom resolution: 160x120, resize resolution: (128, 72) [2024-10-19 00:02:30,710][04690] RunningMeanStd input shape: (3, 72, 128) [2024-10-19 00:02:30,716][04690] RunningMeanStd input shape: (1,) [2024-10-19 00:02:30,758][04690] ConvEncoder: input_channels=3 [2024-10-19 00:02:30,917][04690] Conv encoder output size: 512 [2024-10-19 00:02:30,919][04690] Policy head output size: 512 [2024-10-19 00:02:30,945][04690] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2024-10-19 00:02:31,731][04690] Num frames 100... [2024-10-19 00:02:31,929][04690] Num frames 200... [2024-10-19 00:02:32,128][04690] Num frames 300... [2024-10-19 00:02:32,327][04690] Num frames 400... [2024-10-19 00:02:32,531][04690] Num frames 500... [2024-10-19 00:02:32,742][04690] Num frames 600... [2024-10-19 00:02:32,941][04690] Num frames 700... [2024-10-19 00:02:33,137][04690] Num frames 800... [2024-10-19 00:02:33,349][04690] Num frames 900... [2024-10-19 00:02:33,657][04690] Num frames 1000... [2024-10-19 00:02:33,933][04690] Num frames 1100... [2024-10-19 00:02:34,193][04690] Num frames 1200... [2024-10-19 00:02:34,464][04690] Num frames 1300... [2024-10-19 00:02:34,745][04690] Num frames 1400... [2024-10-19 00:02:35,024][04690] Num frames 1500... [2024-10-19 00:02:35,319][04690] Num frames 1600... [2024-10-19 00:02:35,637][04690] Num frames 1700... [2024-10-19 00:02:35,935][04690] Num frames 1800... [2024-10-19 00:02:36,224][04690] Num frames 1900... [2024-10-19 00:02:36,511][04690] Num frames 2000... [2024-10-19 00:02:36,732][04690] Num frames 2100... [2024-10-19 00:02:36,785][04690] Avg episode rewards: #0: 55.999, true rewards: #0: 21.000 [2024-10-19 00:02:36,787][04690] Avg episode reward: 55.999, avg true_objective: 21.000 [2024-10-19 00:02:36,995][04690] Num frames 2200... [2024-10-19 00:02:37,202][04690] Num frames 2300... [2024-10-19 00:02:37,407][04690] Num frames 2400... [2024-10-19 00:02:37,621][04690] Num frames 2500... [2024-10-19 00:02:37,842][04690] Num frames 2600... [2024-10-19 00:02:38,045][04690] Num frames 2700... [2024-10-19 00:02:38,246][04690] Num frames 2800... [2024-10-19 00:02:38,468][04690] Num frames 2900... [2024-10-19 00:02:38,677][04690] Num frames 3000... [2024-10-19 00:02:38,904][04690] Num frames 3100... [2024-10-19 00:02:39,110][04690] Num frames 3200... [2024-10-19 00:02:39,315][04690] Num frames 3300... [2024-10-19 00:02:39,519][04690] Num frames 3400... [2024-10-19 00:02:39,728][04690] Num frames 3500... [2024-10-19 00:02:39,940][04690] Avg episode rewards: #0: 47.859, true rewards: #0: 17.860 [2024-10-19 00:02:39,943][04690] Avg episode reward: 47.859, avg true_objective: 17.860 [2024-10-19 00:02:40,002][04690] Num frames 3600... [2024-10-19 00:02:40,205][04690] Num frames 3700... [2024-10-19 00:02:40,405][04690] Num frames 3800... [2024-10-19 00:02:40,608][04690] Num frames 3900... [2024-10-19 00:02:40,811][04690] Avg episode rewards: #0: 33.566, true rewards: #0: 13.233 [2024-10-19 00:02:40,813][04690] Avg episode reward: 33.566, avg true_objective: 13.233 [2024-10-19 00:02:40,884][04690] Num frames 4000... [2024-10-19 00:02:41,083][04690] Num frames 4100... [2024-10-19 00:02:41,282][04690] Num frames 4200... [2024-10-19 00:02:41,483][04690] Num frames 4300... [2024-10-19 00:02:41,686][04690] Num frames 4400... [2024-10-19 00:02:41,900][04690] Num frames 4500... [2024-10-19 00:02:42,101][04690] Num frames 4600... [2024-10-19 00:02:42,302][04690] Num frames 4700... [2024-10-19 00:02:42,434][04690] Avg episode rewards: #0: 29.095, true rewards: #0: 11.845 [2024-10-19 00:02:42,436][04690] Avg episode reward: 29.095, avg true_objective: 11.845 [2024-10-19 00:02:42,565][04690] Num frames 4800... [2024-10-19 00:02:42,770][04690] Num frames 4900... [2024-10-19 00:02:42,981][04690] Num frames 5000... [2024-10-19 00:02:43,175][04690] Num frames 5100... [2024-10-19 00:02:43,332][04690] Avg episode rewards: #0: 24.500, true rewards: #0: 10.300 [2024-10-19 00:02:43,335][04690] Avg episode reward: 24.500, avg true_objective: 10.300 [2024-10-19 00:02:43,432][04690] Num frames 5200... [2024-10-19 00:02:43,649][04690] Num frames 5300... [2024-10-19 00:02:43,872][04690] Num frames 5400... [2024-10-19 00:02:44,081][04690] Num frames 5500... [2024-10-19 00:02:44,284][04690] Num frames 5600... [2024-10-19 00:02:44,489][04690] Num frames 5700... [2024-10-19 00:02:44,690][04690] Num frames 5800... [2024-10-19 00:02:44,898][04690] Num frames 5900... [2024-10-19 00:02:45,113][04690] Num frames 6000... [2024-10-19 00:02:45,308][04690] Avg episode rewards: #0: 23.103, true rewards: #0: 10.103 [2024-10-19 00:02:45,310][04690] Avg episode reward: 23.103, avg true_objective: 10.103 [2024-10-19 00:02:45,393][04690] Num frames 6100... [2024-10-19 00:02:45,611][04690] Num frames 6200... [2024-10-19 00:02:45,837][04690] Num frames 6300... [2024-10-19 00:02:46,068][04690] Num frames 6400... [2024-10-19 00:02:46,280][04690] Num frames 6500... [2024-10-19 00:02:46,510][04690] Num frames 6600... [2024-10-19 00:02:46,795][04690] Num frames 6700... [2024-10-19 00:02:47,085][04690] Num frames 6800... [2024-10-19 00:02:47,364][04690] Num frames 6900... [2024-10-19 00:02:47,647][04690] Num frames 7000... [2024-10-19 00:02:47,938][04690] Num frames 7100... [2024-10-19 00:02:48,239][04690] Num frames 7200... [2024-10-19 00:02:48,551][04690] Num frames 7300... [2024-10-19 00:02:48,893][04690] Num frames 7400... [2024-10-19 00:02:49,190][04690] Num frames 7500... [2024-10-19 00:02:49,484][04690] Num frames 7600... [2024-10-19 00:02:49,773][04690] Num frames 7700... [2024-10-19 00:02:50,011][04690] Num frames 7800... [2024-10-19 00:02:50,235][04690] Num frames 7900... [2024-10-19 00:02:50,455][04690] Num frames 8000... [2024-10-19 00:02:50,673][04690] Num frames 8100... [2024-10-19 00:02:50,869][04690] Avg episode rewards: #0: 26.660, true rewards: #0: 11.660 [2024-10-19 00:02:50,871][04690] Avg episode reward: 26.660, avg true_objective: 11.660 [2024-10-19 00:02:50,952][04690] Num frames 8200... [2024-10-19 00:02:51,159][04690] Num frames 8300... [2024-10-19 00:02:51,371][04690] Num frames 8400... [2024-10-19 00:02:51,590][04690] Avg episode rewards: #0: 24.093, true rewards: #0: 10.594 [2024-10-19 00:02:51,593][04690] Avg episode reward: 24.093, avg true_objective: 10.594 [2024-10-19 00:02:51,647][04690] Num frames 8500... [2024-10-19 00:02:51,862][04690] Num frames 8600... [2024-10-19 00:02:52,076][04690] Num frames 8700... [2024-10-19 00:02:52,296][04690] Num frames 8800... [2024-10-19 00:02:52,506][04690] Num frames 8900... [2024-10-19 00:02:52,720][04690] Num frames 9000... [2024-10-19 00:02:52,930][04690] Num frames 9100... [2024-10-19 00:02:53,137][04690] Num frames 9200... [2024-10-19 00:02:53,365][04690] Num frames 9300... [2024-10-19 00:02:53,583][04690] Num frames 9400... [2024-10-19 00:02:53,798][04690] Num frames 9500... [2024-10-19 00:02:54,002][04690] Num frames 9600... [2024-10-19 00:02:54,195][04690] Num frames 9700... [2024-10-19 00:02:54,422][04690] Num frames 9800... [2024-10-19 00:02:54,641][04690] Num frames 9900... [2024-10-19 00:02:54,851][04690] Num frames 10000... [2024-10-19 00:02:55,053][04690] Num frames 10100... [2024-10-19 00:02:55,249][04690] Num frames 10200... [2024-10-19 00:02:55,315][04690] Avg episode rewards: #0: 26.336, true rewards: #0: 11.337 [2024-10-19 00:02:55,318][04690] Avg episode reward: 26.336, avg true_objective: 11.337 [2024-10-19 00:02:55,513][04690] Num frames 10300... [2024-10-19 00:02:55,729][04690] Num frames 10400... [2024-10-19 00:02:55,939][04690] Num frames 10500... [2024-10-19 00:02:56,142][04690] Num frames 10600... [2024-10-19 00:02:56,357][04690] Num frames 10700... [2024-10-19 00:02:56,565][04690] Num frames 10800... [2024-10-19 00:02:56,778][04690] Num frames 10900... [2024-10-19 00:02:56,988][04690] Num frames 11000... [2024-10-19 00:02:57,118][04690] Avg episode rewards: #0: 25.135, true rewards: #0: 11.035 [2024-10-19 00:02:57,121][04690] Avg episode reward: 25.135, avg true_objective: 11.035 [2024-10-19 00:04:19,970][04690] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2024-10-19 00:04:20,854][04690] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-10-19 00:04:20,857][04690] Overriding arg 'num_workers' with value 1 passed from command line [2024-10-19 00:04:20,859][04690] Adding new argument 'no_render'=True that is not in the saved config file! [2024-10-19 00:04:20,862][04690] Adding new argument 'save_video'=True that is not in the saved config file! [2024-10-19 00:04:20,864][04690] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-10-19 00:04:20,866][04690] Adding new argument 'video_name'=None that is not in the saved config file! [2024-10-19 00:04:20,868][04690] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-10-19 00:04:20,869][04690] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-10-19 00:04:20,870][04690] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-10-19 00:04:20,871][04690] Adding new argument 'hf_repository'='ThomasSimonini/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-10-19 00:04:20,873][04690] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-10-19 00:04:20,874][04690] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-10-19 00:04:20,875][04690] Adding new argument 'train_script'=None that is not in the saved config file! [2024-10-19 00:04:20,876][04690] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-10-19 00:04:20,878][04690] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-10-19 00:04:20,889][04690] RunningMeanStd input shape: (3, 72, 128) [2024-10-19 00:04:20,899][04690] RunningMeanStd input shape: (1,) [2024-10-19 00:04:20,922][04690] ConvEncoder: input_channels=3 [2024-10-19 00:04:21,000][04690] Conv encoder output size: 512 [2024-10-19 00:04:21,003][04690] Policy head output size: 512 [2024-10-19 00:04:21,029][04690] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2024-10-19 00:04:21,874][04690] Num frames 100... [2024-10-19 00:04:22,129][04690] Num frames 200... [2024-10-19 00:04:22,383][04690] Num frames 300... [2024-10-19 00:04:22,661][04690] Num frames 400... [2024-10-19 00:04:22,912][04690] Num frames 500... [2024-10-19 00:04:23,151][04690] Num frames 600... [2024-10-19 00:04:23,394][04690] Num frames 700... [2024-10-19 00:04:23,651][04690] Num frames 800... [2024-10-19 00:04:23,912][04690] Num frames 900... [2024-10-19 00:04:24,183][04690] Num frames 1000... [2024-10-19 00:04:24,474][04690] Num frames 1100... [2024-10-19 00:04:24,589][04690] Avg episode rewards: #0: 24.200, true rewards: #0: 11.200 [2024-10-19 00:04:24,591][04690] Avg episode reward: 24.200, avg true_objective: 11.200 [2024-10-19 00:04:24,841][04690] Num frames 1200... [2024-10-19 00:04:25,154][04690] Num frames 1300... [2024-10-19 00:04:25,488][04690] Num frames 1400... [2024-10-19 00:04:25,774][04690] Num frames 1500... [2024-10-19 00:04:26,106][04690] Num frames 1600... [2024-10-19 00:04:26,452][04690] Num frames 1700... [2024-10-19 00:04:26,795][04690] Num frames 1800... [2024-10-19 00:04:27,135][04690] Num frames 1900... [2024-10-19 00:04:27,419][04690] Num frames 2000... [2024-10-19 00:04:27,733][04690] Num frames 2100... [2024-10-19 00:04:28,043][04690] Num frames 2200... [2024-10-19 00:04:28,385][04690] Num frames 2300... [2024-10-19 00:04:28,660][04690] Avg episode rewards: #0: 26.340, true rewards: #0: 11.840 [2024-10-19 00:04:28,663][04690] Avg episode reward: 26.340, avg true_objective: 11.840 [2024-10-19 00:04:28,786][04690] Num frames 2400... [2024-10-19 00:04:29,125][04690] Num frames 2500... [2024-10-19 00:04:29,479][04690] Num frames 2600... [2024-10-19 00:04:29,851][04690] Num frames 2700... [2024-10-19 00:04:30,231][04690] Num frames 2800... [2024-10-19 00:04:30,592][04690] Num frames 2900... [2024-10-19 00:04:30,895][04690] Num frames 3000... [2024-10-19 00:04:31,186][04690] Num frames 3100... [2024-10-19 00:04:31,477][04690] Num frames 3200... [2024-10-19 00:04:31,743][04690] Num frames 3300... [2024-10-19 00:04:32,005][04690] Num frames 3400... [2024-10-19 00:04:32,214][04690] Avg episode rewards: #0: 24.187, true rewards: #0: 11.520 [2024-10-19 00:04:32,217][04690] Avg episode reward: 24.187, avg true_objective: 11.520 [2024-10-19 00:04:32,354][04690] Num frames 3500... [2024-10-19 00:04:32,613][04690] Num frames 3600... [2024-10-19 00:04:32,822][04690] Num frames 3700... [2024-10-19 00:04:33,024][04690] Num frames 3800... [2024-10-19 00:04:33,212][04690] Num frames 3900... [2024-10-19 00:04:33,422][04690] Num frames 4000... [2024-10-19 00:04:33,634][04690] Num frames 4100... [2024-10-19 00:04:33,840][04690] Num frames 4200... [2024-10-19 00:04:34,039][04690] Num frames 4300... [2024-10-19 00:04:34,233][04690] Num frames 4400... [2024-10-19 00:04:34,388][04690] Avg episode rewards: #0: 24.370, true rewards: #0: 11.120 [2024-10-19 00:04:34,391][04690] Avg episode reward: 24.370, avg true_objective: 11.120 [2024-10-19 00:04:34,494][04690] Num frames 4500... [2024-10-19 00:04:34,683][04690] Num frames 4600... [2024-10-19 00:04:34,878][04690] Num frames 4700... [2024-10-19 00:04:35,083][04690] Num frames 4800... [2024-10-19 00:04:35,291][04690] Num frames 4900... [2024-10-19 00:04:35,497][04690] Num frames 5000... [2024-10-19 00:04:35,687][04690] Num frames 5100... [2024-10-19 00:04:35,879][04690] Num frames 5200... [2024-10-19 00:04:36,079][04690] Num frames 5300... [2024-10-19 00:04:36,280][04690] Num frames 5400... [2024-10-19 00:04:36,503][04690] Num frames 5500... [2024-10-19 00:04:36,702][04690] Num frames 5600... [2024-10-19 00:04:36,898][04690] Num frames 5700... [2024-10-19 00:04:37,099][04690] Num frames 5800... [2024-10-19 00:04:37,299][04690] Num frames 5900... [2024-10-19 00:04:37,549][04690] Avg episode rewards: #0: 26.792, true rewards: #0: 11.992 [2024-10-19 00:04:37,554][04690] Avg episode reward: 26.792, avg true_objective: 11.992 [2024-10-19 00:04:37,568][04690] Num frames 6000... [2024-10-19 00:04:37,755][04690] Num frames 6100... [2024-10-19 00:04:37,958][04690] Num frames 6200... [2024-10-19 00:04:38,170][04690] Num frames 6300... [2024-10-19 00:04:38,383][04690] Num frames 6400... [2024-10-19 00:04:38,600][04690] Num frames 6500... [2024-10-19 00:04:38,792][04690] Num frames 6600... [2024-10-19 00:04:38,990][04690] Num frames 6700... [2024-10-19 00:04:39,187][04690] Num frames 6800... [2024-10-19 00:04:39,391][04690] Num frames 6900... [2024-10-19 00:04:39,610][04690] Num frames 7000... [2024-10-19 00:04:39,832][04690] Num frames 7100... [2024-10-19 00:04:40,035][04690] Num frames 7200... [2024-10-19 00:04:40,230][04690] Num frames 7300... [2024-10-19 00:04:40,441][04690] Num frames 7400... [2024-10-19 00:04:40,692][04690] Num frames 7500... [2024-10-19 00:04:41,018][04690] Avg episode rewards: #0: 28.827, true rewards: #0: 12.660 [2024-10-19 00:04:41,022][04690] Avg episode reward: 28.827, avg true_objective: 12.660 [2024-10-19 00:04:41,041][04690] Num frames 7600... [2024-10-19 00:04:41,306][04690] Num frames 7700... [2024-10-19 00:04:41,595][04690] Num frames 7800... [2024-10-19 00:04:41,857][04690] Num frames 7900... [2024-10-19 00:04:42,123][04690] Num frames 8000... [2024-10-19 00:04:42,415][04690] Num frames 8100... [2024-10-19 00:04:42,706][04690] Num frames 8200... [2024-10-19 00:04:42,994][04690] Num frames 8300... [2024-10-19 00:04:43,267][04690] Num frames 8400... [2024-10-19 00:04:43,554][04690] Num frames 8500... [2024-10-19 00:04:43,805][04690] Num frames 8600... [2024-10-19 00:04:44,007][04690] Num frames 8700... [2024-10-19 00:04:44,213][04690] Num frames 8800... [2024-10-19 00:04:44,429][04690] Num frames 8900... [2024-10-19 00:04:44,638][04690] Num frames 9000... [2024-10-19 00:04:44,847][04690] Num frames 9100... [2024-10-19 00:04:45,058][04690] Num frames 9200... [2024-10-19 00:04:45,260][04690] Num frames 9300... [2024-10-19 00:04:45,468][04690] Num frames 9400... [2024-10-19 00:04:45,679][04690] Num frames 9500... [2024-10-19 00:04:45,845][04690] Avg episode rewards: #0: 31.211, true rewards: #0: 13.640 [2024-10-19 00:04:45,846][04690] Avg episode reward: 31.211, avg true_objective: 13.640 [2024-10-19 00:04:45,956][04690] Num frames 9600... [2024-10-19 00:04:46,160][04690] Num frames 9700... [2024-10-19 00:04:46,378][04690] Num frames 9800... [2024-10-19 00:04:46,445][04690] Avg episode rewards: #0: 28.005, true rewards: #0: 12.255 [2024-10-19 00:04:46,446][04690] Avg episode reward: 28.005, avg true_objective: 12.255 [2024-10-19 00:04:46,663][04690] Num frames 9900... [2024-10-19 00:04:46,880][04690] Num frames 10000... [2024-10-19 00:04:47,083][04690] Num frames 10100... [2024-10-19 00:04:47,290][04690] Num frames 10200... [2024-10-19 00:04:47,485][04690] Num frames 10300... [2024-10-19 00:04:47,687][04690] Num frames 10400... [2024-10-19 00:04:47,897][04690] Num frames 10500... [2024-10-19 00:04:48,101][04690] Num frames 10600... [2024-10-19 00:04:48,168][04690] Avg episode rewards: #0: 26.671, true rewards: #0: 11.782 [2024-10-19 00:04:48,170][04690] Avg episode reward: 26.671, avg true_objective: 11.782 [2024-10-19 00:04:48,365][04690] Num frames 10700... [2024-10-19 00:04:48,570][04690] Num frames 10800... [2024-10-19 00:04:48,783][04690] Num frames 10900... [2024-10-19 00:04:48,996][04690] Num frames 11000... [2024-10-19 00:04:49,192][04690] Num frames 11100... [2024-10-19 00:04:49,395][04690] Num frames 11200... [2024-10-19 00:04:49,606][04690] Avg episode rewards: #0: 25.176, true rewards: #0: 11.276 [2024-10-19 00:04:49,609][04690] Avg episode reward: 25.176, avg true_objective: 11.276 [2024-10-19 00:06:08,112][04690] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2024-10-19 00:06:15,483][04690] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-10-19 00:06:15,487][04690] Overriding arg 'num_workers' with value 1 passed from command line [2024-10-19 00:06:15,489][04690] Adding new argument 'no_render'=True that is not in the saved config file! [2024-10-19 00:06:15,492][04690] Adding new argument 'save_video'=True that is not in the saved config file! [2024-10-19 00:06:15,495][04690] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-10-19 00:06:15,497][04690] Adding new argument 'video_name'=None that is not in the saved config file! [2024-10-19 00:06:15,498][04690] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-10-19 00:06:15,500][04690] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-10-19 00:06:15,502][04690] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-10-19 00:06:15,503][04690] Adding new argument 'hf_repository'='SwordAndTea/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-10-19 00:06:15,504][04690] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-10-19 00:06:15,505][04690] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-10-19 00:06:15,507][04690] Adding new argument 'train_script'=None that is not in the saved config file! [2024-10-19 00:06:15,511][04690] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-10-19 00:06:15,512][04690] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-10-19 00:06:15,550][04690] RunningMeanStd input shape: (3, 72, 128) [2024-10-19 00:06:15,554][04690] RunningMeanStd input shape: (1,) [2024-10-19 00:06:15,577][04690] ConvEncoder: input_channels=3 [2024-10-19 00:06:15,655][04690] Conv encoder output size: 512 [2024-10-19 00:06:15,658][04690] Policy head output size: 512 [2024-10-19 00:06:15,686][04690] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2024-10-19 00:06:16,515][04690] Num frames 100... [2024-10-19 00:06:16,815][04690] Num frames 200... [2024-10-19 00:06:17,101][04690] Num frames 300... [2024-10-19 00:06:17,427][04690] Num frames 400... [2024-10-19 00:06:17,565][04690] Avg episode rewards: #0: 10.280, true rewards: #0: 4.280 [2024-10-19 00:06:17,568][04690] Avg episode reward: 10.280, avg true_objective: 4.280 [2024-10-19 00:06:17,794][04690] Num frames 500... [2024-10-19 00:06:18,093][04690] Num frames 600... [2024-10-19 00:06:18,392][04690] Num frames 700... [2024-10-19 00:06:18,720][04690] Num frames 800... [2024-10-19 00:06:18,944][04690] Num frames 900... [2024-10-19 00:06:19,161][04690] Num frames 1000... [2024-10-19 00:06:19,366][04690] Num frames 1100... [2024-10-19 00:06:19,585][04690] Num frames 1200... [2024-10-19 00:06:19,814][04690] Num frames 1300... [2024-10-19 00:06:20,034][04690] Num frames 1400... [2024-10-19 00:06:20,250][04690] Num frames 1500... [2024-10-19 00:06:20,476][04690] Num frames 1600... [2024-10-19 00:06:20,756][04690] Num frames 1700... [2024-10-19 00:06:20,985][04690] Num frames 1800... [2024-10-19 00:06:21,212][04690] Num frames 1900... [2024-10-19 00:06:21,457][04690] Num frames 2000... [2024-10-19 00:06:21,678][04690] Num frames 2100... [2024-10-19 00:06:21,910][04690] Num frames 2200... [2024-10-19 00:06:22,144][04690] Num frames 2300... [2024-10-19 00:06:22,387][04690] Avg episode rewards: #0: 28.900, true rewards: #0: 11.900 [2024-10-19 00:06:22,389][04690] Avg episode reward: 28.900, avg true_objective: 11.900 [2024-10-19 00:06:22,438][04690] Num frames 2400... [2024-10-19 00:06:22,663][04690] Num frames 2500... [2024-10-19 00:06:22,895][04690] Num frames 2600... [2024-10-19 00:06:23,121][04690] Num frames 2700... [2024-10-19 00:06:23,350][04690] Num frames 2800... [2024-10-19 00:06:23,580][04690] Num frames 2900... [2024-10-19 00:06:23,797][04690] Num frames 3000... [2024-10-19 00:06:24,034][04690] Num frames 3100... [2024-10-19 00:06:24,254][04690] Num frames 3200... [2024-10-19 00:06:24,477][04690] Num frames 3300... [2024-10-19 00:06:24,698][04690] Num frames 3400... [2024-10-19 00:06:24,927][04690] Num frames 3500... [2024-10-19 00:06:25,148][04690] Num frames 3600... [2024-10-19 00:06:25,372][04690] Num frames 3700... [2024-10-19 00:06:25,593][04690] Num frames 3800... [2024-10-19 00:06:25,808][04690] Num frames 3900... [2024-10-19 00:06:25,962][04690] Avg episode rewards: #0: 31.483, true rewards: #0: 13.150 [2024-10-19 00:06:25,965][04690] Avg episode reward: 31.483, avg true_objective: 13.150 [2024-10-19 00:06:26,075][04690] Num frames 4000... [2024-10-19 00:06:26,274][04690] Num frames 4100... [2024-10-19 00:06:26,487][04690] Num frames 4200... [2024-10-19 00:06:26,706][04690] Num frames 4300... [2024-10-19 00:06:26,928][04690] Num frames 4400... [2024-10-19 00:06:27,146][04690] Num frames 4500... [2024-10-19 00:06:27,364][04690] Num frames 4600... [2024-10-19 00:06:27,589][04690] Avg episode rewards: #0: 26.702, true rewards: #0: 11.702 [2024-10-19 00:06:27,591][04690] Avg episode reward: 26.702, avg true_objective: 11.702 [2024-10-19 00:06:27,635][04690] Num frames 4700... [2024-10-19 00:06:27,862][04690] Num frames 4800... [2024-10-19 00:06:28,090][04690] Num frames 4900... [2024-10-19 00:06:28,310][04690] Num frames 5000... [2024-10-19 00:06:28,522][04690] Num frames 5100... [2024-10-19 00:06:28,749][04690] Num frames 5200... [2024-10-19 00:06:29,050][04690] Num frames 5300... [2024-10-19 00:06:29,345][04690] Num frames 5400... [2024-10-19 00:06:29,627][04690] Num frames 5500... [2024-10-19 00:06:29,895][04690] Num frames 5600... [2024-10-19 00:06:30,185][04690] Num frames 5700... [2024-10-19 00:06:30,488][04690] Num frames 5800... [2024-10-19 00:06:30,784][04690] Num frames 5900... [2024-10-19 00:06:31,084][04690] Num frames 6000... [2024-10-19 00:06:31,392][04690] Num frames 6100... [2024-10-19 00:06:31,676][04690] Num frames 6200... [2024-10-19 00:06:31,982][04690] Num frames 6300... [2024-10-19 00:06:32,231][04690] Num frames 6400... [2024-10-19 00:06:32,450][04690] Num frames 6500... [2024-10-19 00:06:32,675][04690] Num frames 6600... [2024-10-19 00:06:32,895][04690] Num frames 6700... [2024-10-19 00:06:33,122][04690] Avg episode rewards: #0: 32.362, true rewards: #0: 13.562 [2024-10-19 00:06:33,124][04690] Avg episode reward: 32.362, avg true_objective: 13.562 [2024-10-19 00:06:33,167][04690] Num frames 6800... [2024-10-19 00:06:33,406][04690] Num frames 6900... [2024-10-19 00:06:33,627][04690] Num frames 7000... [2024-10-19 00:06:33,836][04690] Num frames 7100... [2024-10-19 00:06:34,054][04690] Num frames 7200... [2024-10-19 00:06:34,267][04690] Num frames 7300... [2024-10-19 00:06:34,506][04690] Num frames 7400... [2024-10-19 00:06:34,729][04690] Num frames 7500... [2024-10-19 00:06:34,944][04690] Num frames 7600... [2024-10-19 00:06:35,160][04690] Num frames 7700... [2024-10-19 00:06:35,396][04690] Num frames 7800... [2024-10-19 00:06:35,541][04690] Avg episode rewards: #0: 31.395, true rewards: #0: 13.062 [2024-10-19 00:06:35,544][04690] Avg episode reward: 31.395, avg true_objective: 13.062 [2024-10-19 00:06:35,692][04690] Num frames 7900... [2024-10-19 00:06:35,914][04690] Num frames 8000... [2024-10-19 00:06:36,148][04690] Num frames 8100... [2024-10-19 00:06:36,396][04690] Num frames 8200... [2024-10-19 00:06:36,629][04690] Num frames 8300... [2024-10-19 00:06:36,727][04690] Avg episode rewards: #0: 27.738, true rewards: #0: 11.881 [2024-10-19 00:06:36,729][04690] Avg episode reward: 27.738, avg true_objective: 11.881 [2024-10-19 00:06:36,908][04690] Num frames 8400... [2024-10-19 00:06:37,125][04690] Num frames 8500... [2024-10-19 00:06:37,344][04690] Num frames 8600... [2024-10-19 00:06:37,574][04690] Num frames 8700... [2024-10-19 00:06:37,796][04690] Num frames 8800... [2024-10-19 00:06:38,015][04690] Num frames 8900... [2024-10-19 00:06:38,235][04690] Num frames 9000... [2024-10-19 00:06:38,453][04690] Avg episode rewards: #0: 26.714, true rewards: #0: 11.339 [2024-10-19 00:06:38,456][04690] Avg episode reward: 26.714, avg true_objective: 11.339 [2024-10-19 00:06:38,522][04690] Num frames 9100... [2024-10-19 00:06:38,744][04690] Num frames 9200... [2024-10-19 00:06:38,969][04690] Num frames 9300... [2024-10-19 00:06:39,182][04690] Num frames 9400... [2024-10-19 00:06:39,400][04690] Num frames 9500... [2024-10-19 00:06:39,632][04690] Num frames 9600... [2024-10-19 00:06:39,910][04690] Avg episode rewards: #0: 24.997, true rewards: #0: 10.774 [2024-10-19 00:06:39,912][04690] Avg episode reward: 24.997, avg true_objective: 10.774 [2024-10-19 00:06:39,922][04690] Num frames 9700... [2024-10-19 00:06:40,140][04690] Num frames 9800... [2024-10-19 00:06:40,349][04690] Num frames 9900... [2024-10-19 00:06:40,579][04690] Num frames 10000... [2024-10-19 00:06:40,800][04690] Num frames 10100... [2024-10-19 00:06:41,036][04690] Num frames 10200... [2024-10-19 00:06:41,246][04690] Num frames 10300... [2024-10-19 00:06:41,477][04690] Num frames 10400... [2024-10-19 00:06:41,541][04690] Avg episode rewards: #0: 23.701, true rewards: #0: 10.401 [2024-10-19 00:06:41,544][04690] Avg episode reward: 23.701, avg true_objective: 10.401 [2024-10-19 00:07:54,053][04690] Replay video saved to /content/train_dir/default_experiment/replay.mp4! [2024-10-19 00:09:22,408][04690] Loading existing experiment configuration from /content/train_dir/default_experiment/config.json [2024-10-19 00:09:22,411][04690] Overriding arg 'num_workers' with value 1 passed from command line [2024-10-19 00:09:22,413][04690] Adding new argument 'no_render'=True that is not in the saved config file! [2024-10-19 00:09:22,416][04690] Adding new argument 'save_video'=True that is not in the saved config file! [2024-10-19 00:09:22,419][04690] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! [2024-10-19 00:09:22,421][04690] Adding new argument 'video_name'=None that is not in the saved config file! [2024-10-19 00:09:22,422][04690] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! [2024-10-19 00:09:22,424][04690] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! [2024-10-19 00:09:22,425][04690] Adding new argument 'push_to_hub'=True that is not in the saved config file! [2024-10-19 00:09:22,426][04690] Adding new argument 'hf_repository'='SwordAndTea/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! [2024-10-19 00:09:22,428][04690] Adding new argument 'policy_index'=0 that is not in the saved config file! [2024-10-19 00:09:22,429][04690] Adding new argument 'eval_deterministic'=False that is not in the saved config file! [2024-10-19 00:09:22,431][04690] Adding new argument 'train_script'=None that is not in the saved config file! [2024-10-19 00:09:22,432][04690] Adding new argument 'enjoy_script'=None that is not in the saved config file! [2024-10-19 00:09:22,433][04690] Using frameskip 1 and render_action_repeat=4 for evaluation [2024-10-19 00:09:22,476][04690] RunningMeanStd input shape: (3, 72, 128) [2024-10-19 00:09:22,481][04690] RunningMeanStd input shape: (1,) [2024-10-19 00:09:22,501][04690] ConvEncoder: input_channels=3 [2024-10-19 00:09:22,574][04690] Conv encoder output size: 512 [2024-10-19 00:09:22,579][04690] Policy head output size: 512 [2024-10-19 00:09:22,599][04690] Loading state from checkpoint /content/train_dir/default_experiment/checkpoint_p0/checkpoint_000000979_4009984.pth... [2024-10-19 00:09:23,193][04690] Num frames 100... [2024-10-19 00:09:23,437][04690] Num frames 200... [2024-10-19 00:09:23,648][04690] Num frames 300... [2024-10-19 00:09:23,852][04690] Num frames 400... [2024-10-19 00:09:24,074][04690] Avg episode rewards: #0: 8.800, true rewards: #0: 4.800 [2024-10-19 00:09:24,076][04690] Avg episode reward: 8.800, avg true_objective: 4.800 [2024-10-19 00:09:24,123][04690] Num frames 500... [2024-10-19 00:09:24,329][04690] Num frames 600... [2024-10-19 00:09:24,541][04690] Num frames 700... [2024-10-19 00:09:24,759][04690] Num frames 800... [2024-10-19 00:09:24,975][04690] Num frames 900... [2024-10-19 00:09:25,190][04690] Num frames 1000... [2024-10-19 00:09:25,417][04690] Num frames 1100... [2024-10-19 00:09:25,652][04690] Num frames 1200... [2024-10-19 00:09:25,867][04690] Num frames 1300... [2024-10-19 00:09:26,078][04690] Num frames 1400... [2024-10-19 00:09:26,291][04690] Num frames 1500... [2024-10-19 00:09:26,358][04690] Avg episode rewards: #0: 13.520, true rewards: #0: 7.520 [2024-10-19 00:09:26,360][04690] Avg episode reward: 13.520, avg true_objective: 7.520 [2024-10-19 00:09:26,580][04690] Num frames 1600... [2024-10-19 00:09:26,821][04690] Num frames 1700... [2024-10-19 00:09:27,036][04690] Num frames 1800... [2024-10-19 00:09:27,312][04690] Num frames 1900... [2024-10-19 00:09:27,610][04690] Num frames 2000... [2024-10-19 00:09:27,925][04690] Num frames 2100... [2024-10-19 00:09:28,208][04690] Num frames 2200... [2024-10-19 00:09:28,492][04690] Num frames 2300... [2024-10-19 00:09:28,804][04690] Num frames 2400... [2024-10-19 00:09:28,981][04690] Avg episode rewards: #0: 16.453, true rewards: #0: 8.120 [2024-10-19 00:09:28,984][04690] Avg episode reward: 16.453, avg true_objective: 8.120 [2024-10-19 00:09:29,170][04690] Num frames 2500... [2024-10-19 00:09:29,476][04690] Num frames 2600... [2024-10-19 00:09:29,768][04690] Num frames 2700... [2024-10-19 00:09:30,062][04690] Num frames 2800... [2024-10-19 00:09:30,356][04690] Num frames 2900... [2024-10-19 00:09:30,558][04690] Num frames 3000... [2024-10-19 00:09:30,776][04690] Num frames 3100... [2024-10-19 00:09:31,009][04690] Num frames 3200... [2024-10-19 00:09:31,144][04690] Avg episode rewards: #0: 15.840, true rewards: #0: 8.090 [2024-10-19 00:09:31,146][04690] Avg episode reward: 15.840, avg true_objective: 8.090 [2024-10-19 00:09:31,286][04690] Num frames 3300... [2024-10-19 00:09:31,504][04690] Num frames 3400... [2024-10-19 00:09:31,720][04690] Num frames 3500... [2024-10-19 00:09:31,931][04690] Num frames 3600... [2024-10-19 00:09:32,145][04690] Num frames 3700... [2024-10-19 00:09:32,372][04690] Num frames 3800... [2024-10-19 00:09:32,602][04690] Num frames 3900... [2024-10-19 00:09:32,813][04690] Num frames 4000... [2024-10-19 00:09:33,036][04690] Num frames 4100... [2024-10-19 00:09:33,249][04690] Num frames 4200... [2024-10-19 00:09:33,489][04690] Num frames 4300... [2024-10-19 00:09:33,717][04690] Num frames 4400... [2024-10-19 00:09:33,937][04690] Num frames 4500... [2024-10-19 00:09:34,159][04690] Num frames 4600... [2024-10-19 00:09:34,374][04690] Num frames 4700... [2024-10-19 00:09:34,596][04690] Num frames 4800... [2024-10-19 00:09:34,811][04690] Num frames 4900... [2024-10-19 00:09:35,034][04690] Num frames 5000... [2024-10-19 00:09:35,247][04690] Num frames 5100... [2024-10-19 00:09:35,516][04690] Avg episode rewards: #0: 22.386, true rewards: #0: 10.386 [2024-10-19 00:09:35,518][04690] Avg episode reward: 22.386, avg true_objective: 10.386 [2024-10-19 00:09:35,538][04690] Num frames 5200... [2024-10-19 00:09:35,752][04690] Num frames 5300... [2024-10-19 00:09:35,960][04690] Num frames 5400... [2024-10-19 00:09:36,188][04690] Num frames 5500... [2024-10-19 00:09:36,415][04690] Num frames 5600... [2024-10-19 00:09:36,634][04690] Num frames 5700... [2024-10-19 00:09:36,850][04690] Num frames 5800... [2024-10-19 00:09:37,067][04690] Num frames 5900... [2024-10-19 00:09:37,304][04690] Num frames 6000... [2024-10-19 00:09:37,516][04690] Num frames 6100... [2024-10-19 00:09:37,736][04690] Num frames 6200... [2024-10-19 00:09:37,960][04690] Num frames 6300... [2024-10-19 00:09:38,187][04690] Num frames 6400... [2024-10-19 00:09:38,412][04690] Num frames 6500... [2024-10-19 00:09:38,638][04690] Num frames 6600... [2024-10-19 00:09:38,865][04690] Num frames 6700... [2024-10-19 00:09:39,079][04690] Num frames 6800... [2024-10-19 00:09:39,307][04690] Num frames 6900... [2024-10-19 00:09:39,557][04690] Avg episode rewards: #0: 27.141, true rewards: #0: 11.642 [2024-10-19 00:09:39,559][04690] Avg episode reward: 27.141, avg true_objective: 11.642 [2024-10-19 00:09:39,596][04690] Num frames 7000... [2024-10-19 00:09:39,814][04690] Num frames 7100... [2024-10-19 00:09:40,037][04690] Num frames 7200... [2024-10-19 00:09:40,255][04690] Num frames 7300... [2024-10-19 00:09:40,513][04690] Num frames 7400... [2024-10-19 00:09:40,811][04690] Num frames 7500... [2024-10-19 00:09:41,132][04690] Avg episode rewards: #0: 24.990, true rewards: #0: 10.847 [2024-10-19 00:09:41,135][04690] Avg episode reward: 24.990, avg true_objective: 10.847 [2024-10-19 00:09:41,160][04690] Num frames 7600... [2024-10-19 00:09:41,464][04690] Num frames 7700... [2024-10-19 00:09:41,762][04690] Num frames 7800... [2024-10-19 00:09:42,055][04690] Num frames 7900... [2024-10-19 00:09:42,343][04690] Num frames 8000... [2024-10-19 00:09:42,638][04690] Num frames 8100... [2024-10-19 00:09:42,938][04690] Num frames 8200... [2024-10-19 00:09:43,216][04690] Num frames 8300... [2024-10-19 00:09:43,575][04690] Num frames 8400... [2024-10-19 00:09:43,830][04690] Num frames 8500... [2024-10-19 00:09:44,053][04690] Num frames 8600... [2024-10-19 00:09:44,279][04690] Num frames 8700... [2024-10-19 00:09:44,543][04690] Avg episode rewards: #0: 25.360, true rewards: #0: 10.985 [2024-10-19 00:09:44,546][04690] Avg episode reward: 25.360, avg true_objective: 10.985 [2024-10-19 00:09:44,577][04690] Num frames 8800... [2024-10-19 00:09:44,797][04690] Num frames 8900... [2024-10-19 00:09:45,018][04690] Num frames 9000... [2024-10-19 00:09:45,234][04690] Num frames 9100... [2024-10-19 00:09:45,468][04690] Num frames 9200... [2024-10-19 00:09:45,694][04690] Num frames 9300... [2024-10-19 00:09:45,920][04690] Num frames 9400... [2024-10-19 00:09:46,018][04690] Avg episode rewards: #0: 23.687, true rewards: #0: 10.464 [2024-10-19 00:09:46,021][04690] Avg episode reward: 23.687, avg true_objective: 10.464 [2024-10-19 00:09:46,201][04690] Num frames 9500... [2024-10-19 00:09:46,428][04690] Num frames 9600... [2024-10-19 00:09:46,660][04690] Num frames 9700... [2024-10-19 00:09:46,879][04690] Num frames 9800... [2024-10-19 00:09:47,084][04690] Num frames 9900... [2024-10-19 00:09:47,343][04690] Avg episode rewards: #0: 22.394, true rewards: #0: 9.994 [2024-10-19 00:09:47,347][04690] Avg episode reward: 22.394, avg true_objective: 9.994 [2024-10-19 00:10:57,372][04690] Replay video saved to /content/train_dir/default_experiment/replay.mp4!