Hugging Face's logo

Femboyuwu2000
/
bloomz-1b1-vn-chat

PEFT
TensorBoard
Safetensors
trl
sft
Generated from Trainer
Model card Files Metrics Community
Femboyuwu2000 commited on
Commit
84dc2bf
•
1 Parent(s): 6b2e29a

Training in progress, step 8220

Browse files
Files changed (7) hide show
  1. adapter_model.safetensors +1 -1
  2. runs/Apr12_07-40-41_e5a48bec8248/events.out.tfevents.1712907642.e5a48bec8248.443.0 +2 -2
  3. wandb/debug-internal.log +692 -0
  4. wandb/run-20240412_074044-widovcn3/files/output.log +84 -0
  5. wandb/run-20240412_074044-widovcn3/files/wandb-summary.json +1 -1
  6. wandb/run-20240412_074044-widovcn3/logs/debug-internal.log +692 -0
  7. wandb/run-20240412_074044-widovcn3/run-widovcn3.wandb +0 -0
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0c291b58c13f93a64f1b6f82d56e113a222f9938bd3dcf1a4e14b09d3ad485a2
3
  size 13982248
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5527ef85ce2afb6e70c189ef1af9531e23a0c8724fdea09fe2496cd74e87351f
3
  size 13982248
runs/Apr12_07-40-41_e5a48bec8248/events.out.tfevents.1712907642.e5a48bec8248.443.0 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:815382879fd03330b42542c6a1a2fd32be100c1c6332b124ad3e27f6e4850eb9
3
- size 83117
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a086f1cee555a7d151d92d603211fc73fb043c8b05ca46a81d064f2d71e67e1
3
+ size 91979
wandb/debug-internal.log CHANGED
@@ -6102,3 +6102,695 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
6102
  2024-04-12 10:11:22,655 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6103
  2024-04-12 10:11:22,655 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6104
  2024-04-12 10:11:22,710 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6102
  2024-04-12 10:11:22,655 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6103
  2024-04-12 10:11:22,655 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6104
  2024-04-12 10:11:22,710 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6105
+ 2024-04-12 10:11:24,711 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6106
+ 2024-04-12 10:11:25,428 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6107
+ 2024-04-12 10:11:30,429 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6108
+ 2024-04-12 10:11:32,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6109
+ 2024-04-12 10:11:32,724 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6110
+ 2024-04-12 10:11:32,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6111
+ 2024-04-12 10:11:36,031 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6112
+ 2024-04-12 10:11:41,032 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6113
+ 2024-04-12 10:11:45,339 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6114
+ 2024-04-12 10:11:46,340 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6115
+ 2024-04-12 10:11:47,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6116
+ 2024-04-12 10:11:47,681 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6117
+ 2024-04-12 10:11:47,682 DEBUG SenderThread:485 [sender.py:send():379] send: history
6118
+ 2024-04-12 10:11:47,682 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6119
+ 2024-04-12 10:11:47,683 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6120
+ 2024-04-12 10:11:47,720 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6121
+ 2024-04-12 10:11:47,724 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6122
+ 2024-04-12 10:11:47,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6123
+ 2024-04-12 10:11:50,721 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6124
+ 2024-04-12 10:11:51,443 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6125
+ 2024-04-12 10:11:56,443 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6126
+ 2024-04-12 10:12:01,444 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6127
+ 2024-04-12 10:12:02,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6128
+ 2024-04-12 10:12:02,724 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6129
+ 2024-04-12 10:12:02,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6130
+ 2024-04-12 10:12:06,857 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6131
+ 2024-04-12 10:12:10,756 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6132
+ 2024-04-12 10:12:10,757 DEBUG SenderThread:485 [sender.py:send():379] send: history
6133
+ 2024-04-12 10:12:10,757 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6134
+ 2024-04-12 10:12:10,758 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6135
+ 2024-04-12 10:12:11,731 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6136
+ 2024-04-12 10:12:12,679 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6137
+ 2024-04-12 10:12:14,733 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6138
+ 2024-04-12 10:12:15,340 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6139
+ 2024-04-12 10:12:17,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6140
+ 2024-04-12 10:12:17,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6141
+ 2024-04-12 10:12:17,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6142
+ 2024-04-12 10:12:17,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6143
+ 2024-04-12 10:12:22,844 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6144
+ 2024-04-12 10:12:27,845 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6145
+ 2024-04-12 10:12:32,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6146
+ 2024-04-12 10:12:32,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6147
+ 2024-04-12 10:12:32,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6148
+ 2024-04-12 10:12:32,856 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6149
+ 2024-04-12 10:12:37,857 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6150
+ 2024-04-12 10:12:37,993 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6151
+ 2024-04-12 10:12:37,994 DEBUG SenderThread:485 [sender.py:send():379] send: history
6152
+ 2024-04-12 10:12:37,995 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6153
+ 2024-04-12 10:12:37,995 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6154
+ 2024-04-12 10:12:38,742 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6155
+ 2024-04-12 10:12:40,742 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6156
+ 2024-04-12 10:12:43,814 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6157
+ 2024-04-12 10:12:45,341 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6158
+ 2024-04-12 10:12:47,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6159
+ 2024-04-12 10:12:47,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6160
+ 2024-04-12 10:12:47,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6161
+ 2024-04-12 10:12:48,827 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6162
+ 2024-04-12 10:12:53,828 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6163
+ 2024-04-12 10:12:58,829 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6164
+ 2024-04-12 10:13:02,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6165
+ 2024-04-12 10:13:02,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6166
+ 2024-04-12 10:13:02,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6167
+ 2024-04-12 10:13:03,983 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6168
+ 2024-04-12 10:13:03,984 DEBUG SenderThread:485 [sender.py:send():379] send: history
6169
+ 2024-04-12 10:13:03,984 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6170
+ 2024-04-12 10:13:03,985 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6171
+ 2024-04-12 10:13:03,985 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6172
+ 2024-04-12 10:13:04,751 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6173
+ 2024-04-12 10:13:06,752 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6174
+ 2024-04-12 10:13:09,357 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6175
+ 2024-04-12 10:13:14,359 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6176
+ 2024-04-12 10:13:15,342 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6177
+ 2024-04-12 10:13:17,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6178
+ 2024-04-12 10:13:17,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6179
+ 2024-04-12 10:13:17,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6180
+ 2024-04-12 10:13:19,819 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6181
+ 2024-04-12 10:13:24,820 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6182
+ 2024-04-12 10:13:29,401 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6183
+ 2024-04-12 10:13:29,402 DEBUG SenderThread:485 [sender.py:send():379] send: history
6184
+ 2024-04-12 10:13:29,403 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6185
+ 2024-04-12 10:13:29,403 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6186
+ 2024-04-12 10:13:29,760 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6187
+ 2024-04-12 10:13:30,159 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6188
+ 2024-04-12 10:13:32,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6189
+ 2024-04-12 10:13:32,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6190
+ 2024-04-12 10:13:32,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6191
+ 2024-04-12 10:13:32,761 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6192
+ 2024-04-12 10:13:36,138 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6193
+ 2024-04-12 10:13:41,139 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6194
+ 2024-04-12 10:13:45,343 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6195
+ 2024-04-12 10:13:46,344 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6196
+ 2024-04-12 10:13:47,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6197
+ 2024-04-12 10:13:47,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6198
+ 2024-04-12 10:13:47,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6199
+ 2024-04-12 10:13:51,774 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6200
+ 2024-04-12 10:13:54,945 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6201
+ 2024-04-12 10:13:54,946 DEBUG SenderThread:485 [sender.py:send():379] send: history
6202
+ 2024-04-12 10:13:54,946 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6203
+ 2024-04-12 10:13:54,947 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6204
+ 2024-04-12 10:13:55,771 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6205
+ 2024-04-12 10:13:57,740 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6206
+ 2024-04-12 10:13:58,772 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6207
+ 2024-04-12 10:14:02,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6208
+ 2024-04-12 10:14:02,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6209
+ 2024-04-12 10:14:02,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6210
+ 2024-04-12 10:14:02,775 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6211
+ 2024-04-12 10:14:07,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6212
+ 2024-04-12 10:14:12,777 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6213
+ 2024-04-12 10:14:15,344 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6214
+ 2024-04-12 10:14:17,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6215
+ 2024-04-12 10:14:17,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6216
+ 2024-04-12 10:14:17,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6217
+ 2024-04-12 10:14:17,824 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6218
+ 2024-04-12 10:14:19,964 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6219
+ 2024-04-12 10:14:19,966 DEBUG SenderThread:485 [sender.py:send():379] send: history
6220
+ 2024-04-12 10:14:19,966 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6221
+ 2024-04-12 10:14:19,966 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6222
+ 2024-04-12 10:14:20,781 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6223
+ 2024-04-12 10:14:22,781 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6224
+ 2024-04-12 10:14:23,735 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6225
+ 2024-04-12 10:14:28,736 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6226
+ 2024-04-12 10:14:32,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6227
+ 2024-04-12 10:14:32,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6228
+ 2024-04-12 10:14:32,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6229
+ 2024-04-12 10:14:33,770 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6230
+ 2024-04-12 10:14:38,771 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6231
+ 2024-04-12 10:14:43,772 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6232
+ 2024-04-12 10:14:45,345 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6233
+ 2024-04-12 10:14:46,045 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6234
+ 2024-04-12 10:14:46,047 DEBUG SenderThread:485 [sender.py:send():379] send: history
6235
+ 2024-04-12 10:14:46,047 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6236
+ 2024-04-12 10:14:46,047 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6237
+ 2024-04-12 10:14:46,790 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6238
+ 2024-04-12 10:14:47,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6239
+ 2024-04-12 10:14:47,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6240
+ 2024-04-12 10:14:47,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6241
+ 2024-04-12 10:14:48,779 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6242
+ 2024-04-12 10:14:48,791 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6243
+ 2024-04-12 10:14:53,780 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6244
+ 2024-04-12 10:14:58,780 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6245
+ 2024-04-12 10:15:02,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6246
+ 2024-04-12 10:15:02,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6247
+ 2024-04-12 10:15:02,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6248
+ 2024-04-12 10:15:04,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6249
+ 2024-04-12 10:15:09,777 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6250
+ 2024-04-12 10:15:11,471 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6251
+ 2024-04-12 10:15:11,473 DEBUG SenderThread:485 [sender.py:send():379] send: history
6252
+ 2024-04-12 10:15:11,473 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6253
+ 2024-04-12 10:15:11,473 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6254
+ 2024-04-12 10:15:11,800 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6255
+ 2024-04-12 10:15:14,801 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6256
+ 2024-04-12 10:15:15,346 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6257
+ 2024-04-12 10:15:15,346 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6258
+ 2024-04-12 10:15:17,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6259
+ 2024-04-12 10:15:17,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6260
+ 2024-04-12 10:15:17,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6261
+ 2024-04-12 10:15:20,849 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6262
+ 2024-04-12 10:15:25,850 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6263
+ 2024-04-12 10:15:30,851 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6264
+ 2024-04-12 10:15:32,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6265
+ 2024-04-12 10:15:32,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6266
+ 2024-04-12 10:15:32,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6267
+ 2024-04-12 10:15:36,771 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6268
+ 2024-04-12 10:15:37,274 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6269
+ 2024-04-12 10:15:37,275 DEBUG SenderThread:485 [sender.py:send():379] send: history
6270
+ 2024-04-12 10:15:37,275 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6271
+ 2024-04-12 10:15:37,276 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6272
+ 2024-04-12 10:15:37,812 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6273
+ 2024-04-12 10:15:40,814 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6274
+ 2024-04-12 10:15:42,138 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6275
+ 2024-04-12 10:15:45,347 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6276
+ 2024-04-12 10:15:47,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6277
+ 2024-04-12 10:15:47,348 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6278
+ 2024-04-12 10:15:47,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6279
+ 2024-04-12 10:15:47,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6280
+ 2024-04-12 10:15:52,845 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6281
+ 2024-04-12 10:15:57,846 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6282
+ 2024-04-12 10:16:01,901 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6283
+ 2024-04-12 10:16:01,902 DEBUG SenderThread:485 [sender.py:send():379] send: history
6284
+ 2024-04-12 10:16:01,902 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6285
+ 2024-04-12 10:16:01,902 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6286
+ 2024-04-12 10:16:02,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6287
+ 2024-04-12 10:16:02,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6288
+ 2024-04-12 10:16:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6289
+ 2024-04-12 10:16:02,822 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6290
+ 2024-04-12 10:16:03,786 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6291
+ 2024-04-12 10:16:04,823 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6292
+ 2024-04-12 10:16:08,787 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6293
+ 2024-04-12 10:16:13,788 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6294
+ 2024-04-12 10:16:15,348 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6295
+ 2024-04-12 10:16:17,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6296
+ 2024-04-12 10:16:17,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6297
+ 2024-04-12 10:16:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6298
+ 2024-04-12 10:16:18,871 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6299
+ 2024-04-12 10:16:23,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6300
+ 2024-04-12 10:16:26,797 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6301
+ 2024-04-12 10:16:26,799 DEBUG SenderThread:485 [sender.py:send():379] send: history
6302
+ 2024-04-12 10:16:26,799 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6303
+ 2024-04-12 10:16:26,800 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6304
+ 2024-04-12 10:16:26,831 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6305
+ 2024-04-12 10:16:28,832 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6306
+ 2024-04-12 10:16:29,614 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6307
+ 2024-04-12 10:16:32,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6308
+ 2024-04-12 10:16:32,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6309
+ 2024-04-12 10:16:32,742 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6310
+ 2024-04-12 10:16:34,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6311
+ 2024-04-12 10:16:39,777 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6312
+ 2024-04-12 10:16:44,778 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6313
+ 2024-04-12 10:16:45,349 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6314
+ 2024-04-12 10:16:47,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6315
+ 2024-04-12 10:16:47,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6316
+ 2024-04-12 10:16:47,742 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6317
+ 2024-04-12 10:16:48,822 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6318
+ 2024-04-12 10:16:48,823 DEBUG SenderThread:485 [sender.py:send():379] send: history
6319
+ 2024-04-12 10:16:48,823 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6320
+ 2024-04-12 10:16:48,823 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6321
+ 2024-04-12 10:16:48,840 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6322
+ 2024-04-12 10:16:50,644 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6323
+ 2024-04-12 10:16:50,841 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6324
+ 2024-04-12 10:16:55,645 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6325
+ 2024-04-12 10:17:00,646 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6326
+ 2024-04-12 10:17:02,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6327
+ 2024-04-12 10:17:02,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6328
+ 2024-04-12 10:17:02,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6329
+ 2024-04-12 10:17:05,826 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6330
+ 2024-04-12 10:17:10,827 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6331
+ 2024-04-12 10:17:12,898 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6332
+ 2024-04-12 10:17:12,899 DEBUG SenderThread:485 [sender.py:send():379] send: history
6333
+ 2024-04-12 10:17:12,899 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6334
+ 2024-04-12 10:17:12,900 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6335
+ 2024-04-12 10:17:13,850 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6336
+ 2024-04-12 10:17:14,851 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6337
+ 2024-04-12 10:17:15,349 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6338
+ 2024-04-12 10:17:16,351 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6339
+ 2024-04-12 10:17:17,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6340
+ 2024-04-12 10:17:17,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6341
+ 2024-04-12 10:17:17,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6342
+ 2024-04-12 10:17:21,786 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6343
+ 2024-04-12 10:17:26,787 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6344
+ 2024-04-12 10:17:31,788 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6345
+ 2024-04-12 10:17:32,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6346
+ 2024-04-12 10:17:32,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6347
+ 2024-04-12 10:17:32,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6348
+ 2024-04-12 10:17:36,496 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6349
+ 2024-04-12 10:17:36,497 DEBUG SenderThread:485 [sender.py:send():379] send: history
6350
+ 2024-04-12 10:17:36,497 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6351
+ 2024-04-12 10:17:36,498 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6352
+ 2024-04-12 10:17:36,859 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6353
+ 2024-04-12 10:17:37,298 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6354
+ 2024-04-12 10:17:38,860 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6355
+ 2024-04-12 10:17:42,299 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6356
+ 2024-04-12 10:17:45,350 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6357
+ 2024-04-12 10:17:47,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6358
+ 2024-04-12 10:17:47,351 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6359
+ 2024-04-12 10:17:47,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6360
+ 2024-04-12 10:17:47,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6361
+ 2024-04-12 10:17:52,794 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6362
+ 2024-04-12 10:17:57,795 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6363
+ 2024-04-12 10:18:00,349 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6364
+ 2024-04-12 10:18:00,350 DEBUG SenderThread:485 [sender.py:send():379] send: history
6365
+ 2024-04-12 10:18:00,350 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6366
+ 2024-04-12 10:18:00,351 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6367
+ 2024-04-12 10:18:00,868 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6368
+ 2024-04-12 10:18:02,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6369
+ 2024-04-12 10:18:02,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6370
+ 2024-04-12 10:18:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6371
+ 2024-04-12 10:18:02,869 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6372
+ 2024-04-12 10:18:03,774 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6373
+ 2024-04-12 10:18:08,775 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6374
+ 2024-04-12 10:18:13,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6375
+ 2024-04-12 10:18:15,351 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6376
+ 2024-04-12 10:18:17,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6377
+ 2024-04-12 10:18:17,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6378
+ 2024-04-12 10:18:17,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6379
+ 2024-04-12 10:18:18,822 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6380
+ 2024-04-12 10:18:23,823 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6381
+ 2024-04-12 10:18:24,303 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6382
+ 2024-04-12 10:18:24,304 DEBUG SenderThread:485 [sender.py:send():379] send: history
6383
+ 2024-04-12 10:18:24,304 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6384
+ 2024-04-12 10:18:24,305 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6385
+ 2024-04-12 10:18:24,877 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6386
+ 2024-04-12 10:18:26,878 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6387
+ 2024-04-12 10:18:29,106 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6388
+ 2024-04-12 10:18:32,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6389
+ 2024-04-12 10:18:32,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6390
+ 2024-04-12 10:18:32,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6391
+ 2024-04-12 10:18:34,841 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6392
+ 2024-04-12 10:18:39,842 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6393
+ 2024-04-12 10:18:44,844 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6394
+ 2024-04-12 10:18:45,352 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6395
+ 2024-04-12 10:18:47,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6396
+ 2024-04-12 10:18:47,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6397
+ 2024-04-12 10:18:47,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6398
+ 2024-04-12 10:18:47,903 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6399
+ 2024-04-12 10:18:47,904 DEBUG SenderThread:485 [sender.py:send():379] send: history
6400
+ 2024-04-12 10:18:47,904 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6401
+ 2024-04-12 10:18:47,904 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6402
+ 2024-04-12 10:18:48,888 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6403
+ 2024-04-12 10:18:50,720 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6404
+ 2024-04-12 10:18:50,889 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6405
+ 2024-04-12 10:18:55,721 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6406
+ 2024-04-12 10:19:00,722 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6407
+ 2024-04-12 10:19:02,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6408
+ 2024-04-12 10:19:02,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6409
+ 2024-04-12 10:19:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6410
+ 2024-04-12 10:19:05,864 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6411
+ 2024-04-12 10:19:10,865 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6412
+ 2024-04-12 10:19:11,774 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6413
+ 2024-04-12 10:19:11,775 DEBUG SenderThread:485 [sender.py:send():379] send: history
6414
+ 2024-04-12 10:19:11,775 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6415
+ 2024-04-12 10:19:11,776 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6416
+ 2024-04-12 10:19:11,897 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6417
+ 2024-04-12 10:19:14,899 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6418
+ 2024-04-12 10:19:15,353 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6419
+ 2024-04-12 10:19:16,354 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6420
+ 2024-04-12 10:19:17,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6421
+ 2024-04-12 10:19:17,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6422
+ 2024-04-12 10:19:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6423
+ 2024-04-12 10:19:21,865 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6424
+ 2024-04-12 10:19:26,866 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6425
+ 2024-04-12 10:19:31,867 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6426
+ 2024-04-12 10:19:32,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6427
+ 2024-04-12 10:19:32,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6428
+ 2024-04-12 10:19:32,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6429
+ 2024-04-12 10:19:36,393 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6430
+ 2024-04-12 10:19:36,395 DEBUG SenderThread:485 [sender.py:send():379] send: history
6431
+ 2024-04-12 10:19:36,395 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6432
+ 2024-04-12 10:19:36,395 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6433
+ 2024-04-12 10:19:36,910 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6434
+ 2024-04-12 10:19:37,177 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6435
+ 2024-04-12 10:19:38,911 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6436
+ 2024-04-12 10:19:42,178 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6437
+ 2024-04-12 10:19:45,354 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6438
+ 2024-04-12 10:19:47,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6439
+ 2024-04-12 10:19:47,355 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6440
+ 2024-04-12 10:19:47,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6441
+ 2024-04-12 10:19:47,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6442
+ 2024-04-12 10:19:52,848 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6443
+ 2024-04-12 10:19:57,849 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6444
+ 2024-04-12 10:20:00,902 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6445
+ 2024-04-12 10:20:00,904 DEBUG SenderThread:485 [sender.py:send():379] send: history
6446
+ 2024-04-12 10:20:00,904 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6447
+ 2024-04-12 10:20:00,904 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6448
+ 2024-04-12 10:20:00,919 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6449
+ 2024-04-12 10:20:02,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6450
+ 2024-04-12 10:20:02,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6451
+ 2024-04-12 10:20:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6452
+ 2024-04-12 10:20:02,920 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6453
+ 2024-04-12 10:20:03,817 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6454
+ 2024-04-12 10:20:08,818 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6455
+ 2024-04-12 10:20:13,819 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6456
+ 2024-04-12 10:20:15,359 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6457
+ 2024-04-12 10:20:17,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6458
+ 2024-04-12 10:20:17,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6459
+ 2024-04-12 10:20:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6460
+ 2024-04-12 10:20:19,783 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6461
+ 2024-04-12 10:20:23,839 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6462
+ 2024-04-12 10:20:23,840 DEBUG SenderThread:485 [sender.py:send():379] send: history
6463
+ 2024-04-12 10:20:23,841 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6464
+ 2024-04-12 10:20:23,841 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6465
+ 2024-04-12 10:20:23,928 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6466
+ 2024-04-12 10:20:25,630 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6467
+ 2024-04-12 10:20:26,929 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6468
+ 2024-04-12 10:20:30,631 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6469
+ 2024-04-12 10:20:32,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6470
+ 2024-04-12 10:20:32,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6471
+ 2024-04-12 10:20:32,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6472
+ 2024-04-12 10:20:35,833 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6473
+ 2024-04-12 10:20:40,834 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6474
+ 2024-04-12 10:20:45,356 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6475
+ 2024-04-12 10:20:46,357 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6476
+ 2024-04-12 10:20:46,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6477
+ 2024-04-12 10:20:46,873 DEBUG SenderThread:485 [sender.py:send():379] send: history
6478
+ 2024-04-12 10:20:46,874 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6479
+ 2024-04-12 10:20:46,876 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6480
+ 2024-04-12 10:20:46,937 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6481
+ 2024-04-12 10:20:47,379 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6482
+ 2024-04-12 10:20:47,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6483
+ 2024-04-12 10:20:47,750 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6484
+ 2024-04-12 10:20:48,938 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6485
+ 2024-04-12 10:20:51,821 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6486
+ 2024-04-12 10:20:56,822 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6487
+ 2024-04-12 10:21:01,823 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6488
+ 2024-04-12 10:21:02,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6489
+ 2024-04-12 10:21:02,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6490
+ 2024-04-12 10:21:02,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6491
+ 2024-04-12 10:21:06,842 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6492
+ 2024-04-12 10:21:11,843 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6493
+ 2024-04-12 10:21:13,261 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6494
+ 2024-04-12 10:21:13,262 DEBUG SenderThread:485 [sender.py:send():379] send: history
6495
+ 2024-04-12 10:21:13,263 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6496
+ 2024-04-12 10:21:13,263 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6497
+ 2024-04-12 10:21:13,949 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6498
+ 2024-04-12 10:21:14,950 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6499
+ 2024-04-12 10:21:15,357 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6500
+ 2024-04-12 10:21:17,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6501
+ 2024-04-12 10:21:17,358 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6502
+ 2024-04-12 10:21:17,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6503
+ 2024-04-12 10:21:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6504
+ 2024-04-12 10:21:22,834 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6505
+ 2024-04-12 10:21:27,835 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6506
+ 2024-04-12 10:21:32,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6507
+ 2024-04-12 10:21:32,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6508
+ 2024-04-12 10:21:32,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6509
+ 2024-04-12 10:21:32,868 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6510
+ 2024-04-12 10:21:37,404 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6511
+ 2024-04-12 10:21:37,405 DEBUG SenderThread:485 [sender.py:send():379] send: history
6512
+ 2024-04-12 10:21:37,405 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6513
+ 2024-04-12 10:21:37,408 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6514
+ 2024-04-12 10:21:37,958 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6515
+ 2024-04-12 10:21:38,270 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6516
+ 2024-04-12 10:21:38,959 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6517
+ 2024-04-12 10:21:43,271 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6518
+ 2024-04-12 10:21:45,358 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6519
+ 2024-04-12 10:21:47,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6520
+ 2024-04-12 10:21:47,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6521
+ 2024-04-12 10:21:47,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6522
+ 2024-04-12 10:21:48,882 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6523
+ 2024-04-12 10:21:53,883 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6524
+ 2024-04-12 10:21:58,884 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6525
+ 2024-04-12 10:22:01,505 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6526
+ 2024-04-12 10:22:01,506 DEBUG SenderThread:485 [sender.py:send():379] send: history
6527
+ 2024-04-12 10:22:01,506 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6528
+ 2024-04-12 10:22:01,507 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6529
+ 2024-04-12 10:22:01,968 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6530
+ 2024-04-12 10:22:02,388 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6531
+ 2024-04-12 10:22:02,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6532
+ 2024-04-12 10:22:02,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6533
+ 2024-04-12 10:22:04,867 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6534
+ 2024-04-12 10:22:04,970 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6535
+ 2024-04-12 10:22:09,868 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6536
+ 2024-04-12 10:22:14,869 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6537
+ 2024-04-12 10:22:15,359 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6538
+ 2024-04-12 10:22:17,388 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6539
+ 2024-04-12 10:22:17,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6540
+ 2024-04-12 10:22:17,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6541
+ 2024-04-12 10:22:20,815 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6542
+ 2024-04-12 10:22:25,816 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6543
+ 2024-04-12 10:22:26,384 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6544
+ 2024-04-12 10:22:26,385 DEBUG SenderThread:485 [sender.py:send():379] send: history
6545
+ 2024-04-12 10:22:26,386 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6546
+ 2024-04-12 10:22:26,386 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6547
+ 2024-04-12 10:22:26,978 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6548
+ 2024-04-12 10:22:28,979 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6549
+ 2024-04-12 10:22:31,190 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6550
+ 2024-04-12 10:22:32,388 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6551
+ 2024-04-12 10:22:32,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6552
+ 2024-04-12 10:22:32,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6553
+ 2024-04-12 10:22:36,860 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6554
+ 2024-04-12 10:22:41,861 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6555
+ 2024-04-12 10:22:45,359 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6556
+ 2024-04-12 10:22:47,360 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6557
+ 2024-04-12 10:22:47,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6558
+ 2024-04-12 10:22:47,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6559
+ 2024-04-12 10:22:47,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6560
+ 2024-04-12 10:22:52,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6561
+ 2024-04-12 10:22:53,042 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6562
+ 2024-04-12 10:22:53,043 DEBUG SenderThread:485 [sender.py:send():379] send: history
6563
+ 2024-04-12 10:22:53,043 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6564
+ 2024-04-12 10:22:53,044 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6565
+ 2024-04-12 10:22:53,990 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6566
+ 2024-04-12 10:22:54,989 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6567
+ 2024-04-12 10:22:57,830 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6568
+ 2024-04-12 10:23:02,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6569
+ 2024-04-12 10:23:02,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6570
+ 2024-04-12 10:23:02,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6571
+ 2024-04-12 10:23:02,855 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6572
+ 2024-04-12 10:23:07,856 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6573
+ 2024-04-12 10:23:12,857 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6574
+ 2024-04-12 10:23:15,360 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6575
+ 2024-04-12 10:23:17,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6576
+ 2024-04-12 10:23:17,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6577
+ 2024-04-12 10:23:17,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6578
+ 2024-04-12 10:23:17,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6579
+ 2024-04-12 10:23:18,306 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6580
+ 2024-04-12 10:23:18,307 DEBUG SenderThread:485 [sender.py:send():379] send: history
6581
+ 2024-04-12 10:23:18,308 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6582
+ 2024-04-12 10:23:18,308 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6583
+ 2024-04-12 10:23:18,999 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6584
+ 2024-04-12 10:23:21,000 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6585
+ 2024-04-12 10:23:23,270 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6586
+ 2024-04-12 10:23:28,271 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6587
+ 2024-04-12 10:23:32,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6588
+ 2024-04-12 10:23:32,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6589
+ 2024-04-12 10:23:32,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6590
+ 2024-04-12 10:23:33,825 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6591
+ 2024-04-12 10:23:38,826 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6592
+ 2024-04-12 10:23:41,868 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6593
+ 2024-04-12 10:23:41,869 DEBUG SenderThread:485 [sender.py:send():379] send: history
6594
+ 2024-04-12 10:23:41,869 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6595
+ 2024-04-12 10:23:41,870 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6596
+ 2024-04-12 10:23:42,008 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6597
+ 2024-04-12 10:23:44,635 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6598
+ 2024-04-12 10:23:45,009 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6599
+ 2024-04-12 10:23:45,361 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6600
+ 2024-04-12 10:23:47,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6601
+ 2024-04-12 10:23:47,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6602
+ 2024-04-12 10:23:47,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6603
+ 2024-04-12 10:23:49,819 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6604
+ 2024-04-12 10:23:54,820 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6605
+ 2024-04-12 10:23:59,821 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6606
+ 2024-04-12 10:24:02,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6607
+ 2024-04-12 10:24:02,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6608
+ 2024-04-12 10:24:02,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6609
+ 2024-04-12 10:24:05,812 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6610
+ 2024-04-12 10:24:06,104 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6611
+ 2024-04-12 10:24:06,105 DEBUG SenderThread:485 [sender.py:send():379] send: history
6612
+ 2024-04-12 10:24:06,105 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6613
+ 2024-04-12 10:24:06,106 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6614
+ 2024-04-12 10:24:07,018 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6615
+ 2024-04-12 10:24:09,019 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6616
+ 2024-04-12 10:24:10,925 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6617
+ 2024-04-12 10:24:15,362 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6618
+ 2024-04-12 10:24:16,363 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6619
+ 2024-04-12 10:24:17,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6620
+ 2024-04-12 10:24:17,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6621
+ 2024-04-12 10:24:17,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6622
+ 2024-04-12 10:24:21,814 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6623
+ 2024-04-12 10:24:26,815 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6624
+ 2024-04-12 10:24:30,705 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6625
+ 2024-04-12 10:24:30,706 DEBUG SenderThread:485 [sender.py:send():379] send: history
6626
+ 2024-04-12 10:24:30,706 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6627
+ 2024-04-12 10:24:30,707 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6628
+ 2024-04-12 10:24:31,028 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6629
+ 2024-04-12 10:24:31,956 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6630
+ 2024-04-12 10:24:32,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6631
+ 2024-04-12 10:24:32,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6632
+ 2024-04-12 10:24:32,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6633
+ 2024-04-12 10:24:33,029 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6634
+ 2024-04-12 10:24:37,788 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6635
+ 2024-04-12 10:24:42,789 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6636
+ 2024-04-12 10:24:45,363 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6637
+ 2024-04-12 10:24:47,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6638
+ 2024-04-12 10:24:47,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6639
+ 2024-04-12 10:24:47,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6640
+ 2024-04-12 10:24:47,840 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6641
+ 2024-04-12 10:24:52,839 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6642
+ 2024-04-12 10:24:54,851 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6643
+ 2024-04-12 10:24:54,852 DEBUG SenderThread:485 [sender.py:send():379] send: history
6644
+ 2024-04-12 10:24:54,853 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6645
+ 2024-04-12 10:24:54,853 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6646
+ 2024-04-12 10:24:55,037 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6647
+ 2024-04-12 10:24:57,038 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6648
+ 2024-04-12 10:24:58,630 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6649
+ 2024-04-12 10:25:02,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6650
+ 2024-04-12 10:25:02,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6651
+ 2024-04-12 10:25:02,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6652
+ 2024-04-12 10:25:03,797 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6653
+ 2024-04-12 10:25:08,798 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6654
+ 2024-04-12 10:25:13,799 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6655
+ 2024-04-12 10:25:15,364 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6656
+ 2024-04-12 10:25:17,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6657
+ 2024-04-12 10:25:17,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6658
+ 2024-04-12 10:25:17,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6659
+ 2024-04-12 10:25:18,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6660
+ 2024-04-12 10:25:23,282 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6661
+ 2024-04-12 10:25:23,283 DEBUG SenderThread:485 [sender.py:send():379] send: history
6662
+ 2024-04-12 10:25:23,283 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6663
+ 2024-04-12 10:25:23,284 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6664
+ 2024-04-12 10:25:24,051 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6665
+ 2024-04-12 10:25:24,095 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6666
+ 2024-04-12 10:25:25,052 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6667
+ 2024-04-12 10:25:29,096 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6668
+ 2024-04-12 10:25:32,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6669
+ 2024-04-12 10:25:32,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6670
+ 2024-04-12 10:25:32,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6671
+ 2024-04-12 10:25:34,942 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6672
+ 2024-04-12 10:25:39,944 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6673
+ 2024-04-12 10:25:44,945 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6674
+ 2024-04-12 10:25:45,365 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6675
+ 2024-04-12 10:25:47,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6676
+ 2024-04-12 10:25:47,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6677
+ 2024-04-12 10:25:47,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6678
+ 2024-04-12 10:25:48,570 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6679
+ 2024-04-12 10:25:48,572 DEBUG SenderThread:485 [sender.py:send():379] send: history
6680
+ 2024-04-12 10:25:48,572 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6681
+ 2024-04-12 10:25:48,572 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6682
+ 2024-04-12 10:25:49,061 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6683
+ 2024-04-12 10:25:50,512 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6684
+ 2024-04-12 10:25:51,062 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6685
+ 2024-04-12 10:25:55,513 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6686
+ 2024-04-12 10:26:00,514 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6687
+ 2024-04-12 10:26:02,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6688
+ 2024-04-12 10:26:02,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6689
+ 2024-04-12 10:26:02,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6690
+ 2024-04-12 10:26:05,869 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6691
+ 2024-04-12 10:26:10,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6692
+ 2024-04-12 10:26:14,272 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6693
+ 2024-04-12 10:26:14,273 DEBUG SenderThread:485 [sender.py:send():379] send: history
6694
+ 2024-04-12 10:26:14,273 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6695
+ 2024-04-12 10:26:14,274 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6696
+ 2024-04-12 10:26:15,071 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6697
+ 2024-04-12 10:26:15,366 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6698
+ 2024-04-12 10:26:16,367 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6699
+ 2024-04-12 10:26:17,072 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6700
+ 2024-04-12 10:26:17,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6701
+ 2024-04-12 10:26:17,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6702
+ 2024-04-12 10:26:17,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6703
+ 2024-04-12 10:26:21,972 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6704
+ 2024-04-12 10:26:26,973 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6705
+ 2024-04-12 10:26:31,974 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6706
+ 2024-04-12 10:26:32,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6707
+ 2024-04-12 10:26:32,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6708
+ 2024-04-12 10:26:32,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6709
+ 2024-04-12 10:26:37,824 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6710
+ 2024-04-12 10:26:40,162 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6711
+ 2024-04-12 10:26:40,164 DEBUG SenderThread:485 [sender.py:send():379] send: history
6712
+ 2024-04-12 10:26:40,164 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6713
+ 2024-04-12 10:26:40,164 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6714
+ 2024-04-12 10:26:41,082 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6715
+ 2024-04-12 10:26:42,990 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6716
+ 2024-04-12 10:26:43,083 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6717
+ 2024-04-12 10:26:45,367 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6718
+ 2024-04-12 10:26:47,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6719
+ 2024-04-12 10:26:47,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6720
+ 2024-04-12 10:26:47,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6721
+ 2024-04-12 10:26:48,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6722
+ 2024-04-12 10:26:53,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6723
+ 2024-04-12 10:26:58,873 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6724
+ 2024-04-12 10:27:02,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6725
+ 2024-04-12 10:27:02,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6726
+ 2024-04-12 10:27:02,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6727
+ 2024-04-12 10:27:02,907 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6728
+ 2024-04-12 10:27:02,908 DEBUG SenderThread:485 [sender.py:send():379] send: history
6729
+ 2024-04-12 10:27:02,908 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6730
+ 2024-04-12 10:27:02,909 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6731
+ 2024-04-12 10:27:03,092 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6732
+ 2024-04-12 10:27:04,718 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6733
+ 2024-04-12 10:27:05,093 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6734
+ 2024-04-12 10:27:09,719 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6735
+ 2024-04-12 10:27:14,720 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6736
+ 2024-04-12 10:27:15,368 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6737
+ 2024-04-12 10:27:17,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6738
+ 2024-04-12 10:27:17,749 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6739
+ 2024-04-12 10:27:17,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6740
+ 2024-04-12 10:27:19,785 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6741
+ 2024-04-12 10:27:24,786 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6742
+ 2024-04-12 10:27:26,672 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6743
+ 2024-04-12 10:27:26,673 DEBUG SenderThread:485 [sender.py:send():379] send: history
6744
+ 2024-04-12 10:27:26,673 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6745
+ 2024-04-12 10:27:26,673 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6746
+ 2024-04-12 10:27:27,102 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6747
+ 2024-04-12 10:27:29,103 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6748
+ 2024-04-12 10:27:30,472 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6749
+ 2024-04-12 10:27:32,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6750
+ 2024-04-12 10:27:32,749 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6751
+ 2024-04-12 10:27:32,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6752
+ 2024-04-12 10:27:35,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6753
+ 2024-04-12 10:27:40,871 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6754
+ 2024-04-12 10:27:45,369 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6755
+ 2024-04-12 10:27:46,370 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6756
+ 2024-04-12 10:27:47,392 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6757
+ 2024-04-12 10:27:47,751 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6758
+ 2024-04-12 10:27:47,751 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6759
+ 2024-04-12 10:27:50,577 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6760
+ 2024-04-12 10:27:50,579 DEBUG SenderThread:485 [sender.py:send():379] send: history
6761
+ 2024-04-12 10:27:50,579 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6762
+ 2024-04-12 10:27:50,579 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6763
+ 2024-04-12 10:27:51,112 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6764
+ 2024-04-12 10:27:51,378 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6765
+ 2024-04-12 10:27:53,113 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6766
+ 2024-04-12 10:27:56,380 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6767
+ 2024-04-12 10:28:01,381 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6768
+ 2024-04-12 10:28:02,396 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6769
+ 2024-04-12 10:28:02,750 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6770
+ 2024-04-12 10:28:02,750 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6771
+ 2024-04-12 10:28:06,807 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6772
+ 2024-04-12 10:28:11,808 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6773
+ 2024-04-12 10:28:15,370 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6774
+ 2024-04-12 10:28:15,841 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6775
+ 2024-04-12 10:28:15,843 DEBUG SenderThread:485 [sender.py:send():379] send: history
6776
+ 2024-04-12 10:28:15,843 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6777
+ 2024-04-12 10:28:15,844 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6778
+ 2024-04-12 10:28:16,122 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6779
+ 2024-04-12 10:28:17,123 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6780
+ 2024-04-12 10:28:17,395 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6781
+ 2024-04-12 10:28:17,636 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6782
+ 2024-04-12 10:28:17,750 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6783
+ 2024-04-12 10:28:17,750 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6784
+ 2024-04-12 10:28:22,869 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6785
+ 2024-04-12 10:28:27,871 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6786
+ 2024-04-12 10:28:32,402 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6787
+ 2024-04-12 10:28:32,750 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6788
+ 2024-04-12 10:28:32,751 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6789
+ 2024-04-12 10:28:32,883 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6790
+ 2024-04-12 10:28:37,353 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6791
+ 2024-04-12 10:28:37,355 DEBUG SenderThread:485 [sender.py:send():379] send: history
6792
+ 2024-04-12 10:28:37,355 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6793
+ 2024-04-12 10:28:37,355 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6794
+ 2024-04-12 10:28:38,132 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6795
+ 2024-04-12 10:28:38,197 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6796
+ 2024-04-12 10:28:39,132 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
wandb/run-20240412_074044-widovcn3/files/output.log CHANGED
@@ -736,4 +736,88 @@
736
  warnings.warn(
737
  /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
738
  warnings.warn(
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
739
  /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
 
736
  warnings.warn(
737
  /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
738
  warnings.warn(
739
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
740
+ warnings.warn(
741
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
742
+ warnings.warn(
743
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
744
+ warnings.warn(
745
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
746
+ warnings.warn(
747
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
748
+ warnings.warn(
749
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
750
+ warnings.warn(
751
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
752
+ warnings.warn(
753
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
754
+ warnings.warn(
755
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
756
+ warnings.warn(
757
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
758
+ warnings.warn(
759
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
760
+ warnings.warn(
761
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
762
+ warnings.warn(
763
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
764
+ warnings.warn(
765
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
766
+ warnings.warn(
767
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
768
+ warnings.warn(
769
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
770
+ warnings.warn(
771
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
772
+ warnings.warn(
773
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
774
+ warnings.warn(
775
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
776
+ warnings.warn(
777
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
778
+ warnings.warn(
779
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
780
+ warnings.warn(
781
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
782
+ warnings.warn(
783
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
784
+ warnings.warn(
785
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
786
+ warnings.warn(
787
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
788
+ warnings.warn(
789
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
790
+ warnings.warn(
791
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
792
+ warnings.warn(
793
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
794
+ warnings.warn(
795
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
796
+ warnings.warn(
797
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
798
+ warnings.warn(
799
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
800
+ warnings.warn(
801
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
802
+ warnings.warn(
803
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
804
+ warnings.warn(
805
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
806
+ warnings.warn(
807
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
808
+ warnings.warn(
809
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
810
+ warnings.warn(
811
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
812
+ warnings.warn(
813
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
814
+ warnings.warn(
815
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
816
+ warnings.warn(
817
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
818
+ warnings.warn(
819
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
820
+ warnings.warn(
821
+ /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
822
+ warnings.warn(
823
  /opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:429: UserWarning: torch.utils.checkpoint: please pass in use_reentrant=True or use_reentrant=False explicitly. The default value of use_reentrant will be updated to be False in the future. To maintain current behavior, pass use_reentrant=True. It is recommended that you use use_reentrant=False. Refer to docs for more details on the differences between the two variants.
wandb/run-20240412_074044-widovcn3/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 3.5714, "train/grad_norm": 30.72098159790039, "train/learning_rate": 2.1831491698460923e-05, "train/epoch": 0.59, "train/global_step": 7380, "_timestamp": 1712916682.6488101, "_runtime": 9038.141092061996, "_step": 368}
 
1
+ {"train/loss": 3.4812, "train/grad_norm": 41.20121383666992, "train/learning_rate": 1.995746116153363e-05, "train/epoch": 0.66, "train/global_step": 8220, "_timestamp": 1712917717.3531704, "_runtime": 10072.845452308655, "_step": 410}
wandb/run-20240412_074044-widovcn3/logs/debug-internal.log CHANGED
@@ -6102,3 +6102,695 @@ subprocess.TimeoutExpired: Command '['conda', 'env', 'export']' timed out after
6102
  2024-04-12 10:11:22,655 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6103
  2024-04-12 10:11:22,655 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6104
  2024-04-12 10:11:22,710 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
6102
  2024-04-12 10:11:22,655 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6103
  2024-04-12 10:11:22,655 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6104
  2024-04-12 10:11:22,710 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6105
+ 2024-04-12 10:11:24,711 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6106
+ 2024-04-12 10:11:25,428 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6107
+ 2024-04-12 10:11:30,429 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6108
+ 2024-04-12 10:11:32,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6109
+ 2024-04-12 10:11:32,724 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6110
+ 2024-04-12 10:11:32,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6111
+ 2024-04-12 10:11:36,031 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6112
+ 2024-04-12 10:11:41,032 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6113
+ 2024-04-12 10:11:45,339 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6114
+ 2024-04-12 10:11:46,340 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6115
+ 2024-04-12 10:11:47,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6116
+ 2024-04-12 10:11:47,681 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6117
+ 2024-04-12 10:11:47,682 DEBUG SenderThread:485 [sender.py:send():379] send: history
6118
+ 2024-04-12 10:11:47,682 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6119
+ 2024-04-12 10:11:47,683 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6120
+ 2024-04-12 10:11:47,720 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6121
+ 2024-04-12 10:11:47,724 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6122
+ 2024-04-12 10:11:47,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6123
+ 2024-04-12 10:11:50,721 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6124
+ 2024-04-12 10:11:51,443 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6125
+ 2024-04-12 10:11:56,443 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6126
+ 2024-04-12 10:12:01,444 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6127
+ 2024-04-12 10:12:02,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6128
+ 2024-04-12 10:12:02,724 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6129
+ 2024-04-12 10:12:02,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6130
+ 2024-04-12 10:12:06,857 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6131
+ 2024-04-12 10:12:10,756 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6132
+ 2024-04-12 10:12:10,757 DEBUG SenderThread:485 [sender.py:send():379] send: history
6133
+ 2024-04-12 10:12:10,757 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6134
+ 2024-04-12 10:12:10,758 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6135
+ 2024-04-12 10:12:11,731 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6136
+ 2024-04-12 10:12:12,679 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6137
+ 2024-04-12 10:12:14,733 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6138
+ 2024-04-12 10:12:15,340 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6139
+ 2024-04-12 10:12:17,148 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6140
+ 2024-04-12 10:12:17,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6141
+ 2024-04-12 10:12:17,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6142
+ 2024-04-12 10:12:17,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6143
+ 2024-04-12 10:12:22,844 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6144
+ 2024-04-12 10:12:27,845 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6145
+ 2024-04-12 10:12:32,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6146
+ 2024-04-12 10:12:32,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6147
+ 2024-04-12 10:12:32,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6148
+ 2024-04-12 10:12:32,856 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6149
+ 2024-04-12 10:12:37,857 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6150
+ 2024-04-12 10:12:37,993 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6151
+ 2024-04-12 10:12:37,994 DEBUG SenderThread:485 [sender.py:send():379] send: history
6152
+ 2024-04-12 10:12:37,995 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6153
+ 2024-04-12 10:12:37,995 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6154
+ 2024-04-12 10:12:38,742 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6155
+ 2024-04-12 10:12:40,742 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6156
+ 2024-04-12 10:12:43,814 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6157
+ 2024-04-12 10:12:45,341 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6158
+ 2024-04-12 10:12:47,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6159
+ 2024-04-12 10:12:47,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6160
+ 2024-04-12 10:12:47,725 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6161
+ 2024-04-12 10:12:48,827 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6162
+ 2024-04-12 10:12:53,828 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6163
+ 2024-04-12 10:12:58,829 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6164
+ 2024-04-12 10:13:02,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6165
+ 2024-04-12 10:13:02,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6166
+ 2024-04-12 10:13:02,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6167
+ 2024-04-12 10:13:03,983 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6168
+ 2024-04-12 10:13:03,984 DEBUG SenderThread:485 [sender.py:send():379] send: history
6169
+ 2024-04-12 10:13:03,984 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6170
+ 2024-04-12 10:13:03,985 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6171
+ 2024-04-12 10:13:03,985 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6172
+ 2024-04-12 10:13:04,751 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6173
+ 2024-04-12 10:13:06,752 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6174
+ 2024-04-12 10:13:09,357 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6175
+ 2024-04-12 10:13:14,359 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6176
+ 2024-04-12 10:13:15,342 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6177
+ 2024-04-12 10:13:17,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6178
+ 2024-04-12 10:13:17,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6179
+ 2024-04-12 10:13:17,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6180
+ 2024-04-12 10:13:19,819 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6181
+ 2024-04-12 10:13:24,820 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6182
+ 2024-04-12 10:13:29,401 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6183
+ 2024-04-12 10:13:29,402 DEBUG SenderThread:485 [sender.py:send():379] send: history
6184
+ 2024-04-12 10:13:29,403 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6185
+ 2024-04-12 10:13:29,403 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6186
+ 2024-04-12 10:13:29,760 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6187
+ 2024-04-12 10:13:30,159 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6188
+ 2024-04-12 10:13:32,149 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6189
+ 2024-04-12 10:13:32,725 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6190
+ 2024-04-12 10:13:32,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6191
+ 2024-04-12 10:13:32,761 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6192
+ 2024-04-12 10:13:36,138 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6193
+ 2024-04-12 10:13:41,139 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6194
+ 2024-04-12 10:13:45,343 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6195
+ 2024-04-12 10:13:46,344 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6196
+ 2024-04-12 10:13:47,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6197
+ 2024-04-12 10:13:47,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6198
+ 2024-04-12 10:13:47,726 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6199
+ 2024-04-12 10:13:51,774 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6200
+ 2024-04-12 10:13:54,945 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6201
+ 2024-04-12 10:13:54,946 DEBUG SenderThread:485 [sender.py:send():379] send: history
6202
+ 2024-04-12 10:13:54,946 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6203
+ 2024-04-12 10:13:54,947 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6204
+ 2024-04-12 10:13:55,771 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6205
+ 2024-04-12 10:13:57,740 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6206
+ 2024-04-12 10:13:58,772 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6207
+ 2024-04-12 10:14:02,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6208
+ 2024-04-12 10:14:02,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6209
+ 2024-04-12 10:14:02,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6210
+ 2024-04-12 10:14:02,775 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6211
+ 2024-04-12 10:14:07,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6212
+ 2024-04-12 10:14:12,777 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6213
+ 2024-04-12 10:14:15,344 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6214
+ 2024-04-12 10:14:17,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6215
+ 2024-04-12 10:14:17,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6216
+ 2024-04-12 10:14:17,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6217
+ 2024-04-12 10:14:17,824 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6218
+ 2024-04-12 10:14:19,964 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6219
+ 2024-04-12 10:14:19,966 DEBUG SenderThread:485 [sender.py:send():379] send: history
6220
+ 2024-04-12 10:14:19,966 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6221
+ 2024-04-12 10:14:19,966 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6222
+ 2024-04-12 10:14:20,781 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6223
+ 2024-04-12 10:14:22,781 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6224
+ 2024-04-12 10:14:23,735 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6225
+ 2024-04-12 10:14:28,736 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6226
+ 2024-04-12 10:14:32,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6227
+ 2024-04-12 10:14:32,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6228
+ 2024-04-12 10:14:32,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6229
+ 2024-04-12 10:14:33,770 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6230
+ 2024-04-12 10:14:38,771 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6231
+ 2024-04-12 10:14:43,772 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6232
+ 2024-04-12 10:14:45,345 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6233
+ 2024-04-12 10:14:46,045 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6234
+ 2024-04-12 10:14:46,047 DEBUG SenderThread:485 [sender.py:send():379] send: history
6235
+ 2024-04-12 10:14:46,047 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6236
+ 2024-04-12 10:14:46,047 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6237
+ 2024-04-12 10:14:46,790 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6238
+ 2024-04-12 10:14:47,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6239
+ 2024-04-12 10:14:47,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6240
+ 2024-04-12 10:14:47,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6241
+ 2024-04-12 10:14:48,779 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6242
+ 2024-04-12 10:14:48,791 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6243
+ 2024-04-12 10:14:53,780 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6244
+ 2024-04-12 10:14:58,780 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6245
+ 2024-04-12 10:15:02,150 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6246
+ 2024-04-12 10:15:02,726 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6247
+ 2024-04-12 10:15:02,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6248
+ 2024-04-12 10:15:04,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6249
+ 2024-04-12 10:15:09,777 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6250
+ 2024-04-12 10:15:11,471 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6251
+ 2024-04-12 10:15:11,473 DEBUG SenderThread:485 [sender.py:send():379] send: history
6252
+ 2024-04-12 10:15:11,473 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6253
+ 2024-04-12 10:15:11,473 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6254
+ 2024-04-12 10:15:11,800 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6255
+ 2024-04-12 10:15:14,801 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6256
+ 2024-04-12 10:15:15,346 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6257
+ 2024-04-12 10:15:15,346 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6258
+ 2024-04-12 10:15:17,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6259
+ 2024-04-12 10:15:17,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6260
+ 2024-04-12 10:15:17,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6261
+ 2024-04-12 10:15:20,849 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6262
+ 2024-04-12 10:15:25,850 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6263
+ 2024-04-12 10:15:30,851 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6264
+ 2024-04-12 10:15:32,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6265
+ 2024-04-12 10:15:32,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6266
+ 2024-04-12 10:15:32,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6267
+ 2024-04-12 10:15:36,771 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6268
+ 2024-04-12 10:15:37,274 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6269
+ 2024-04-12 10:15:37,275 DEBUG SenderThread:485 [sender.py:send():379] send: history
6270
+ 2024-04-12 10:15:37,275 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6271
+ 2024-04-12 10:15:37,276 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6272
+ 2024-04-12 10:15:37,812 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6273
+ 2024-04-12 10:15:40,814 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6274
+ 2024-04-12 10:15:42,138 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6275
+ 2024-04-12 10:15:45,347 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6276
+ 2024-04-12 10:15:47,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6277
+ 2024-04-12 10:15:47,348 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6278
+ 2024-04-12 10:15:47,727 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6279
+ 2024-04-12 10:15:47,727 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6280
+ 2024-04-12 10:15:52,845 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6281
+ 2024-04-12 10:15:57,846 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6282
+ 2024-04-12 10:16:01,901 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6283
+ 2024-04-12 10:16:01,902 DEBUG SenderThread:485 [sender.py:send():379] send: history
6284
+ 2024-04-12 10:16:01,902 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6285
+ 2024-04-12 10:16:01,902 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6286
+ 2024-04-12 10:16:02,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6287
+ 2024-04-12 10:16:02,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6288
+ 2024-04-12 10:16:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6289
+ 2024-04-12 10:16:02,822 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6290
+ 2024-04-12 10:16:03,786 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6291
+ 2024-04-12 10:16:04,823 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6292
+ 2024-04-12 10:16:08,787 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6293
+ 2024-04-12 10:16:13,788 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6294
+ 2024-04-12 10:16:15,348 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6295
+ 2024-04-12 10:16:17,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6296
+ 2024-04-12 10:16:17,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6297
+ 2024-04-12 10:16:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6298
+ 2024-04-12 10:16:18,871 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6299
+ 2024-04-12 10:16:23,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6300
+ 2024-04-12 10:16:26,797 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6301
+ 2024-04-12 10:16:26,799 DEBUG SenderThread:485 [sender.py:send():379] send: history
6302
+ 2024-04-12 10:16:26,799 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6303
+ 2024-04-12 10:16:26,800 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6304
+ 2024-04-12 10:16:26,831 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6305
+ 2024-04-12 10:16:28,832 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6306
+ 2024-04-12 10:16:29,614 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6307
+ 2024-04-12 10:16:32,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6308
+ 2024-04-12 10:16:32,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6309
+ 2024-04-12 10:16:32,742 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6310
+ 2024-04-12 10:16:34,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6311
+ 2024-04-12 10:16:39,777 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6312
+ 2024-04-12 10:16:44,778 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6313
+ 2024-04-12 10:16:45,349 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6314
+ 2024-04-12 10:16:47,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6315
+ 2024-04-12 10:16:47,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6316
+ 2024-04-12 10:16:47,742 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6317
+ 2024-04-12 10:16:48,822 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6318
+ 2024-04-12 10:16:48,823 DEBUG SenderThread:485 [sender.py:send():379] send: history
6319
+ 2024-04-12 10:16:48,823 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6320
+ 2024-04-12 10:16:48,823 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6321
+ 2024-04-12 10:16:48,840 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6322
+ 2024-04-12 10:16:50,644 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6323
+ 2024-04-12 10:16:50,841 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6324
+ 2024-04-12 10:16:55,645 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6325
+ 2024-04-12 10:17:00,646 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6326
+ 2024-04-12 10:17:02,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6327
+ 2024-04-12 10:17:02,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6328
+ 2024-04-12 10:17:02,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6329
+ 2024-04-12 10:17:05,826 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6330
+ 2024-04-12 10:17:10,827 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6331
+ 2024-04-12 10:17:12,898 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6332
+ 2024-04-12 10:17:12,899 DEBUG SenderThread:485 [sender.py:send():379] send: history
6333
+ 2024-04-12 10:17:12,899 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6334
+ 2024-04-12 10:17:12,900 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6335
+ 2024-04-12 10:17:13,850 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6336
+ 2024-04-12 10:17:14,851 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6337
+ 2024-04-12 10:17:15,349 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6338
+ 2024-04-12 10:17:16,351 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6339
+ 2024-04-12 10:17:17,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6340
+ 2024-04-12 10:17:17,742 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6341
+ 2024-04-12 10:17:17,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6342
+ 2024-04-12 10:17:21,786 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6343
+ 2024-04-12 10:17:26,787 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6344
+ 2024-04-12 10:17:31,788 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6345
+ 2024-04-12 10:17:32,151 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6346
+ 2024-04-12 10:17:32,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6347
+ 2024-04-12 10:17:32,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6348
+ 2024-04-12 10:17:36,496 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6349
+ 2024-04-12 10:17:36,497 DEBUG SenderThread:485 [sender.py:send():379] send: history
6350
+ 2024-04-12 10:17:36,497 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6351
+ 2024-04-12 10:17:36,498 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6352
+ 2024-04-12 10:17:36,859 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6353
+ 2024-04-12 10:17:37,298 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6354
+ 2024-04-12 10:17:38,860 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6355
+ 2024-04-12 10:17:42,299 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6356
+ 2024-04-12 10:17:45,350 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6357
+ 2024-04-12 10:17:47,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6358
+ 2024-04-12 10:17:47,351 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6359
+ 2024-04-12 10:17:47,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6360
+ 2024-04-12 10:17:47,743 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6361
+ 2024-04-12 10:17:52,794 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6362
+ 2024-04-12 10:17:57,795 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6363
+ 2024-04-12 10:18:00,349 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6364
+ 2024-04-12 10:18:00,350 DEBUG SenderThread:485 [sender.py:send():379] send: history
6365
+ 2024-04-12 10:18:00,350 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6366
+ 2024-04-12 10:18:00,351 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6367
+ 2024-04-12 10:18:00,868 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6368
+ 2024-04-12 10:18:02,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6369
+ 2024-04-12 10:18:02,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6370
+ 2024-04-12 10:18:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6371
+ 2024-04-12 10:18:02,869 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6372
+ 2024-04-12 10:18:03,774 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6373
+ 2024-04-12 10:18:08,775 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6374
+ 2024-04-12 10:18:13,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6375
+ 2024-04-12 10:18:15,351 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6376
+ 2024-04-12 10:18:17,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6377
+ 2024-04-12 10:18:17,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6378
+ 2024-04-12 10:18:17,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6379
+ 2024-04-12 10:18:18,822 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6380
+ 2024-04-12 10:18:23,823 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6381
+ 2024-04-12 10:18:24,303 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6382
+ 2024-04-12 10:18:24,304 DEBUG SenderThread:485 [sender.py:send():379] send: history
6383
+ 2024-04-12 10:18:24,304 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6384
+ 2024-04-12 10:18:24,305 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6385
+ 2024-04-12 10:18:24,877 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6386
+ 2024-04-12 10:18:26,878 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6387
+ 2024-04-12 10:18:29,106 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6388
+ 2024-04-12 10:18:32,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6389
+ 2024-04-12 10:18:32,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6390
+ 2024-04-12 10:18:32,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6391
+ 2024-04-12 10:18:34,841 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6392
+ 2024-04-12 10:18:39,842 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6393
+ 2024-04-12 10:18:44,844 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6394
+ 2024-04-12 10:18:45,352 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6395
+ 2024-04-12 10:18:47,152 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6396
+ 2024-04-12 10:18:47,743 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6397
+ 2024-04-12 10:18:47,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6398
+ 2024-04-12 10:18:47,903 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6399
+ 2024-04-12 10:18:47,904 DEBUG SenderThread:485 [sender.py:send():379] send: history
6400
+ 2024-04-12 10:18:47,904 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6401
+ 2024-04-12 10:18:47,904 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6402
+ 2024-04-12 10:18:48,888 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6403
+ 2024-04-12 10:18:50,720 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6404
+ 2024-04-12 10:18:50,889 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6405
+ 2024-04-12 10:18:55,721 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6406
+ 2024-04-12 10:19:00,722 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6407
+ 2024-04-12 10:19:02,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6408
+ 2024-04-12 10:19:02,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6409
+ 2024-04-12 10:19:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6410
+ 2024-04-12 10:19:05,864 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6411
+ 2024-04-12 10:19:10,865 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6412
+ 2024-04-12 10:19:11,774 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6413
+ 2024-04-12 10:19:11,775 DEBUG SenderThread:485 [sender.py:send():379] send: history
6414
+ 2024-04-12 10:19:11,775 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6415
+ 2024-04-12 10:19:11,776 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6416
+ 2024-04-12 10:19:11,897 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6417
+ 2024-04-12 10:19:14,899 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6418
+ 2024-04-12 10:19:15,353 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6419
+ 2024-04-12 10:19:16,354 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6420
+ 2024-04-12 10:19:17,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6421
+ 2024-04-12 10:19:17,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6422
+ 2024-04-12 10:19:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6423
+ 2024-04-12 10:19:21,865 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6424
+ 2024-04-12 10:19:26,866 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6425
+ 2024-04-12 10:19:31,867 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6426
+ 2024-04-12 10:19:32,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6427
+ 2024-04-12 10:19:32,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6428
+ 2024-04-12 10:19:32,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6429
+ 2024-04-12 10:19:36,393 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6430
+ 2024-04-12 10:19:36,395 DEBUG SenderThread:485 [sender.py:send():379] send: history
6431
+ 2024-04-12 10:19:36,395 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6432
+ 2024-04-12 10:19:36,395 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6433
+ 2024-04-12 10:19:36,910 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6434
+ 2024-04-12 10:19:37,177 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6435
+ 2024-04-12 10:19:38,911 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6436
+ 2024-04-12 10:19:42,178 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6437
+ 2024-04-12 10:19:45,354 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6438
+ 2024-04-12 10:19:47,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6439
+ 2024-04-12 10:19:47,355 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6440
+ 2024-04-12 10:19:47,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6441
+ 2024-04-12 10:19:47,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6442
+ 2024-04-12 10:19:52,848 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6443
+ 2024-04-12 10:19:57,849 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6444
+ 2024-04-12 10:20:00,902 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6445
+ 2024-04-12 10:20:00,904 DEBUG SenderThread:485 [sender.py:send():379] send: history
6446
+ 2024-04-12 10:20:00,904 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6447
+ 2024-04-12 10:20:00,904 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6448
+ 2024-04-12 10:20:00,919 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6449
+ 2024-04-12 10:20:02,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6450
+ 2024-04-12 10:20:02,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6451
+ 2024-04-12 10:20:02,744 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6452
+ 2024-04-12 10:20:02,920 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6453
+ 2024-04-12 10:20:03,817 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6454
+ 2024-04-12 10:20:08,818 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6455
+ 2024-04-12 10:20:13,819 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6456
+ 2024-04-12 10:20:15,359 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6457
+ 2024-04-12 10:20:17,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6458
+ 2024-04-12 10:20:17,744 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6459
+ 2024-04-12 10:20:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6460
+ 2024-04-12 10:20:19,783 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6461
+ 2024-04-12 10:20:23,839 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6462
+ 2024-04-12 10:20:23,840 DEBUG SenderThread:485 [sender.py:send():379] send: history
6463
+ 2024-04-12 10:20:23,841 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6464
+ 2024-04-12 10:20:23,841 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6465
+ 2024-04-12 10:20:23,928 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6466
+ 2024-04-12 10:20:25,630 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6467
+ 2024-04-12 10:20:26,929 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6468
+ 2024-04-12 10:20:30,631 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6469
+ 2024-04-12 10:20:32,153 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6470
+ 2024-04-12 10:20:32,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6471
+ 2024-04-12 10:20:32,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6472
+ 2024-04-12 10:20:35,833 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6473
+ 2024-04-12 10:20:40,834 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6474
+ 2024-04-12 10:20:45,356 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6475
+ 2024-04-12 10:20:46,357 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6476
+ 2024-04-12 10:20:46,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6477
+ 2024-04-12 10:20:46,873 DEBUG SenderThread:485 [sender.py:send():379] send: history
6478
+ 2024-04-12 10:20:46,874 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6479
+ 2024-04-12 10:20:46,876 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6480
+ 2024-04-12 10:20:46,937 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6481
+ 2024-04-12 10:20:47,379 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6482
+ 2024-04-12 10:20:47,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6483
+ 2024-04-12 10:20:47,750 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6484
+ 2024-04-12 10:20:48,938 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6485
+ 2024-04-12 10:20:51,821 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6486
+ 2024-04-12 10:20:56,822 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6487
+ 2024-04-12 10:21:01,823 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6488
+ 2024-04-12 10:21:02,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6489
+ 2024-04-12 10:21:02,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6490
+ 2024-04-12 10:21:02,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6491
+ 2024-04-12 10:21:06,842 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6492
+ 2024-04-12 10:21:11,843 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6493
+ 2024-04-12 10:21:13,261 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6494
+ 2024-04-12 10:21:13,262 DEBUG SenderThread:485 [sender.py:send():379] send: history
6495
+ 2024-04-12 10:21:13,263 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6496
+ 2024-04-12 10:21:13,263 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6497
+ 2024-04-12 10:21:13,949 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6498
+ 2024-04-12 10:21:14,950 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6499
+ 2024-04-12 10:21:15,357 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6500
+ 2024-04-12 10:21:17,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6501
+ 2024-04-12 10:21:17,358 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6502
+ 2024-04-12 10:21:17,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6503
+ 2024-04-12 10:21:17,745 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6504
+ 2024-04-12 10:21:22,834 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6505
+ 2024-04-12 10:21:27,835 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6506
+ 2024-04-12 10:21:32,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6507
+ 2024-04-12 10:21:32,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6508
+ 2024-04-12 10:21:32,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6509
+ 2024-04-12 10:21:32,868 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6510
+ 2024-04-12 10:21:37,404 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6511
+ 2024-04-12 10:21:37,405 DEBUG SenderThread:485 [sender.py:send():379] send: history
6512
+ 2024-04-12 10:21:37,405 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6513
+ 2024-04-12 10:21:37,408 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6514
+ 2024-04-12 10:21:37,958 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6515
+ 2024-04-12 10:21:38,270 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6516
+ 2024-04-12 10:21:38,959 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6517
+ 2024-04-12 10:21:43,271 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6518
+ 2024-04-12 10:21:45,358 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6519
+ 2024-04-12 10:21:47,332 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6520
+ 2024-04-12 10:21:47,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6521
+ 2024-04-12 10:21:47,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6522
+ 2024-04-12 10:21:48,882 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6523
+ 2024-04-12 10:21:53,883 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6524
+ 2024-04-12 10:21:58,884 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6525
+ 2024-04-12 10:22:01,505 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6526
+ 2024-04-12 10:22:01,506 DEBUG SenderThread:485 [sender.py:send():379] send: history
6527
+ 2024-04-12 10:22:01,506 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6528
+ 2024-04-12 10:22:01,507 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6529
+ 2024-04-12 10:22:01,968 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6530
+ 2024-04-12 10:22:02,388 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6531
+ 2024-04-12 10:22:02,745 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6532
+ 2024-04-12 10:22:02,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6533
+ 2024-04-12 10:22:04,867 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6534
+ 2024-04-12 10:22:04,970 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6535
+ 2024-04-12 10:22:09,868 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6536
+ 2024-04-12 10:22:14,869 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6537
+ 2024-04-12 10:22:15,359 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6538
+ 2024-04-12 10:22:17,388 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6539
+ 2024-04-12 10:22:17,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6540
+ 2024-04-12 10:22:17,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6541
+ 2024-04-12 10:22:20,815 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6542
+ 2024-04-12 10:22:25,816 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6543
+ 2024-04-12 10:22:26,384 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6544
+ 2024-04-12 10:22:26,385 DEBUG SenderThread:485 [sender.py:send():379] send: history
6545
+ 2024-04-12 10:22:26,386 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6546
+ 2024-04-12 10:22:26,386 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6547
+ 2024-04-12 10:22:26,978 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6548
+ 2024-04-12 10:22:28,979 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6549
+ 2024-04-12 10:22:31,190 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6550
+ 2024-04-12 10:22:32,388 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6551
+ 2024-04-12 10:22:32,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6552
+ 2024-04-12 10:22:32,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6553
+ 2024-04-12 10:22:36,860 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6554
+ 2024-04-12 10:22:41,861 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6555
+ 2024-04-12 10:22:45,359 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6556
+ 2024-04-12 10:22:47,360 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6557
+ 2024-04-12 10:22:47,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6558
+ 2024-04-12 10:22:47,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6559
+ 2024-04-12 10:22:47,746 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6560
+ 2024-04-12 10:22:52,776 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6561
+ 2024-04-12 10:22:53,042 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6562
+ 2024-04-12 10:22:53,043 DEBUG SenderThread:485 [sender.py:send():379] send: history
6563
+ 2024-04-12 10:22:53,043 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6564
+ 2024-04-12 10:22:53,044 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6565
+ 2024-04-12 10:22:53,990 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6566
+ 2024-04-12 10:22:54,989 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6567
+ 2024-04-12 10:22:57,830 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6568
+ 2024-04-12 10:23:02,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6569
+ 2024-04-12 10:23:02,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6570
+ 2024-04-12 10:23:02,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6571
+ 2024-04-12 10:23:02,855 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6572
+ 2024-04-12 10:23:07,856 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6573
+ 2024-04-12 10:23:12,857 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6574
+ 2024-04-12 10:23:15,360 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6575
+ 2024-04-12 10:23:17,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6576
+ 2024-04-12 10:23:17,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6577
+ 2024-04-12 10:23:17,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6578
+ 2024-04-12 10:23:17,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6579
+ 2024-04-12 10:23:18,306 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6580
+ 2024-04-12 10:23:18,307 DEBUG SenderThread:485 [sender.py:send():379] send: history
6581
+ 2024-04-12 10:23:18,308 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6582
+ 2024-04-12 10:23:18,308 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6583
+ 2024-04-12 10:23:18,999 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6584
+ 2024-04-12 10:23:21,000 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6585
+ 2024-04-12 10:23:23,270 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6586
+ 2024-04-12 10:23:28,271 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6587
+ 2024-04-12 10:23:32,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6588
+ 2024-04-12 10:23:32,746 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6589
+ 2024-04-12 10:23:32,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6590
+ 2024-04-12 10:23:33,825 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6591
+ 2024-04-12 10:23:38,826 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6592
+ 2024-04-12 10:23:41,868 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6593
+ 2024-04-12 10:23:41,869 DEBUG SenderThread:485 [sender.py:send():379] send: history
6594
+ 2024-04-12 10:23:41,869 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6595
+ 2024-04-12 10:23:41,870 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6596
+ 2024-04-12 10:23:42,008 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6597
+ 2024-04-12 10:23:44,635 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6598
+ 2024-04-12 10:23:45,009 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6599
+ 2024-04-12 10:23:45,361 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6600
+ 2024-04-12 10:23:47,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6601
+ 2024-04-12 10:23:47,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6602
+ 2024-04-12 10:23:47,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6603
+ 2024-04-12 10:23:49,819 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6604
+ 2024-04-12 10:23:54,820 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6605
+ 2024-04-12 10:23:59,821 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6606
+ 2024-04-12 10:24:02,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6607
+ 2024-04-12 10:24:02,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6608
+ 2024-04-12 10:24:02,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6609
+ 2024-04-12 10:24:05,812 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6610
+ 2024-04-12 10:24:06,104 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6611
+ 2024-04-12 10:24:06,105 DEBUG SenderThread:485 [sender.py:send():379] send: history
6612
+ 2024-04-12 10:24:06,105 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6613
+ 2024-04-12 10:24:06,106 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6614
+ 2024-04-12 10:24:07,018 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6615
+ 2024-04-12 10:24:09,019 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6616
+ 2024-04-12 10:24:10,925 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6617
+ 2024-04-12 10:24:15,362 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6618
+ 2024-04-12 10:24:16,363 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6619
+ 2024-04-12 10:24:17,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6620
+ 2024-04-12 10:24:17,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6621
+ 2024-04-12 10:24:17,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6622
+ 2024-04-12 10:24:21,814 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6623
+ 2024-04-12 10:24:26,815 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6624
+ 2024-04-12 10:24:30,705 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6625
+ 2024-04-12 10:24:30,706 DEBUG SenderThread:485 [sender.py:send():379] send: history
6626
+ 2024-04-12 10:24:30,706 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6627
+ 2024-04-12 10:24:30,707 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6628
+ 2024-04-12 10:24:31,028 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6629
+ 2024-04-12 10:24:31,956 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6630
+ 2024-04-12 10:24:32,389 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6631
+ 2024-04-12 10:24:32,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6632
+ 2024-04-12 10:24:32,747 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6633
+ 2024-04-12 10:24:33,029 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6634
+ 2024-04-12 10:24:37,788 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6635
+ 2024-04-12 10:24:42,789 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6636
+ 2024-04-12 10:24:45,363 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6637
+ 2024-04-12 10:24:47,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6638
+ 2024-04-12 10:24:47,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6639
+ 2024-04-12 10:24:47,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6640
+ 2024-04-12 10:24:47,840 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6641
+ 2024-04-12 10:24:52,839 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6642
+ 2024-04-12 10:24:54,851 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6643
+ 2024-04-12 10:24:54,852 DEBUG SenderThread:485 [sender.py:send():379] send: history
6644
+ 2024-04-12 10:24:54,853 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6645
+ 2024-04-12 10:24:54,853 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6646
+ 2024-04-12 10:24:55,037 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6647
+ 2024-04-12 10:24:57,038 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6648
+ 2024-04-12 10:24:58,630 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6649
+ 2024-04-12 10:25:02,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6650
+ 2024-04-12 10:25:02,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6651
+ 2024-04-12 10:25:02,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6652
+ 2024-04-12 10:25:03,797 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6653
+ 2024-04-12 10:25:08,798 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6654
+ 2024-04-12 10:25:13,799 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6655
+ 2024-04-12 10:25:15,364 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6656
+ 2024-04-12 10:25:17,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6657
+ 2024-04-12 10:25:17,747 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6658
+ 2024-04-12 10:25:17,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6659
+ 2024-04-12 10:25:18,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6660
+ 2024-04-12 10:25:23,282 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6661
+ 2024-04-12 10:25:23,283 DEBUG SenderThread:485 [sender.py:send():379] send: history
6662
+ 2024-04-12 10:25:23,283 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6663
+ 2024-04-12 10:25:23,284 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6664
+ 2024-04-12 10:25:24,051 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6665
+ 2024-04-12 10:25:24,095 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6666
+ 2024-04-12 10:25:25,052 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6667
+ 2024-04-12 10:25:29,096 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6668
+ 2024-04-12 10:25:32,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6669
+ 2024-04-12 10:25:32,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6670
+ 2024-04-12 10:25:32,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6671
+ 2024-04-12 10:25:34,942 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6672
+ 2024-04-12 10:25:39,944 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6673
+ 2024-04-12 10:25:44,945 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6674
+ 2024-04-12 10:25:45,365 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6675
+ 2024-04-12 10:25:47,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6676
+ 2024-04-12 10:25:47,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6677
+ 2024-04-12 10:25:47,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6678
+ 2024-04-12 10:25:48,570 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6679
+ 2024-04-12 10:25:48,572 DEBUG SenderThread:485 [sender.py:send():379] send: history
6680
+ 2024-04-12 10:25:48,572 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6681
+ 2024-04-12 10:25:48,572 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6682
+ 2024-04-12 10:25:49,061 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6683
+ 2024-04-12 10:25:50,512 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6684
+ 2024-04-12 10:25:51,062 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6685
+ 2024-04-12 10:25:55,513 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6686
+ 2024-04-12 10:26:00,514 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6687
+ 2024-04-12 10:26:02,390 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6688
+ 2024-04-12 10:26:02,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6689
+ 2024-04-12 10:26:02,748 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6690
+ 2024-04-12 10:26:05,869 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6691
+ 2024-04-12 10:26:10,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6692
+ 2024-04-12 10:26:14,272 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6693
+ 2024-04-12 10:26:14,273 DEBUG SenderThread:485 [sender.py:send():379] send: history
6694
+ 2024-04-12 10:26:14,273 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6695
+ 2024-04-12 10:26:14,274 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6696
+ 2024-04-12 10:26:15,071 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6697
+ 2024-04-12 10:26:15,366 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6698
+ 2024-04-12 10:26:16,367 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6699
+ 2024-04-12 10:26:17,072 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6700
+ 2024-04-12 10:26:17,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6701
+ 2024-04-12 10:26:17,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6702
+ 2024-04-12 10:26:17,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6703
+ 2024-04-12 10:26:21,972 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6704
+ 2024-04-12 10:26:26,973 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6705
+ 2024-04-12 10:26:31,974 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6706
+ 2024-04-12 10:26:32,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6707
+ 2024-04-12 10:26:32,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6708
+ 2024-04-12 10:26:32,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6709
+ 2024-04-12 10:26:37,824 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6710
+ 2024-04-12 10:26:40,162 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6711
+ 2024-04-12 10:26:40,164 DEBUG SenderThread:485 [sender.py:send():379] send: history
6712
+ 2024-04-12 10:26:40,164 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6713
+ 2024-04-12 10:26:40,164 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6714
+ 2024-04-12 10:26:41,082 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6715
+ 2024-04-12 10:26:42,990 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6716
+ 2024-04-12 10:26:43,083 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6717
+ 2024-04-12 10:26:45,367 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6718
+ 2024-04-12 10:26:47,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6719
+ 2024-04-12 10:26:47,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6720
+ 2024-04-12 10:26:47,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6721
+ 2024-04-12 10:26:48,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6722
+ 2024-04-12 10:26:53,872 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6723
+ 2024-04-12 10:26:58,873 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6724
+ 2024-04-12 10:27:02,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6725
+ 2024-04-12 10:27:02,748 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6726
+ 2024-04-12 10:27:02,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6727
+ 2024-04-12 10:27:02,907 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6728
+ 2024-04-12 10:27:02,908 DEBUG SenderThread:485 [sender.py:send():379] send: history
6729
+ 2024-04-12 10:27:02,908 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6730
+ 2024-04-12 10:27:02,909 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6731
+ 2024-04-12 10:27:03,092 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6732
+ 2024-04-12 10:27:04,718 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6733
+ 2024-04-12 10:27:05,093 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6734
+ 2024-04-12 10:27:09,719 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6735
+ 2024-04-12 10:27:14,720 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6736
+ 2024-04-12 10:27:15,368 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6737
+ 2024-04-12 10:27:17,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6738
+ 2024-04-12 10:27:17,749 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6739
+ 2024-04-12 10:27:17,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6740
+ 2024-04-12 10:27:19,785 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6741
+ 2024-04-12 10:27:24,786 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6742
+ 2024-04-12 10:27:26,672 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6743
+ 2024-04-12 10:27:26,673 DEBUG SenderThread:485 [sender.py:send():379] send: history
6744
+ 2024-04-12 10:27:26,673 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6745
+ 2024-04-12 10:27:26,673 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6746
+ 2024-04-12 10:27:27,102 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6747
+ 2024-04-12 10:27:29,103 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6748
+ 2024-04-12 10:27:30,472 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6749
+ 2024-04-12 10:27:32,391 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6750
+ 2024-04-12 10:27:32,749 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6751
+ 2024-04-12 10:27:32,749 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6752
+ 2024-04-12 10:27:35,870 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6753
+ 2024-04-12 10:27:40,871 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6754
+ 2024-04-12 10:27:45,369 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6755
+ 2024-04-12 10:27:46,370 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6756
+ 2024-04-12 10:27:47,392 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6757
+ 2024-04-12 10:27:47,751 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6758
+ 2024-04-12 10:27:47,751 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6759
+ 2024-04-12 10:27:50,577 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6760
+ 2024-04-12 10:27:50,579 DEBUG SenderThread:485 [sender.py:send():379] send: history
6761
+ 2024-04-12 10:27:50,579 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6762
+ 2024-04-12 10:27:50,579 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6763
+ 2024-04-12 10:27:51,112 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6764
+ 2024-04-12 10:27:51,378 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6765
+ 2024-04-12 10:27:53,113 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6766
+ 2024-04-12 10:27:56,380 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6767
+ 2024-04-12 10:28:01,381 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6768
+ 2024-04-12 10:28:02,396 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6769
+ 2024-04-12 10:28:02,750 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6770
+ 2024-04-12 10:28:02,750 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6771
+ 2024-04-12 10:28:06,807 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6772
+ 2024-04-12 10:28:11,808 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6773
+ 2024-04-12 10:28:15,370 DEBUG SenderThread:485 [sender.py:send():379] send: stats
6774
+ 2024-04-12 10:28:15,841 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6775
+ 2024-04-12 10:28:15,843 DEBUG SenderThread:485 [sender.py:send():379] send: history
6776
+ 2024-04-12 10:28:15,843 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6777
+ 2024-04-12 10:28:15,844 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6778
+ 2024-04-12 10:28:16,122 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6779
+ 2024-04-12 10:28:17,123 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
6780
+ 2024-04-12 10:28:17,395 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6781
+ 2024-04-12 10:28:17,636 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6782
+ 2024-04-12 10:28:17,750 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6783
+ 2024-04-12 10:28:17,750 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6784
+ 2024-04-12 10:28:22,869 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6785
+ 2024-04-12 10:28:27,871 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6786
+ 2024-04-12 10:28:32,402 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: internal_messages
6787
+ 2024-04-12 10:28:32,750 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: stop_status
6788
+ 2024-04-12 10:28:32,751 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: stop_status
6789
+ 2024-04-12 10:28:32,883 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6790
+ 2024-04-12 10:28:37,353 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: partial_history
6791
+ 2024-04-12 10:28:37,355 DEBUG SenderThread:485 [sender.py:send():379] send: history
6792
+ 2024-04-12 10:28:37,355 DEBUG SenderThread:485 [sender.py:send_request():406] send_request: summary_record
6793
+ 2024-04-12 10:28:37,355 INFO SenderThread:485 [sender.py:_save_file():1390] saving file wandb-summary.json with policy end
6794
+ 2024-04-12 10:28:38,132 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/wandb-summary.json
6795
+ 2024-04-12 10:28:38,197 DEBUG HandlerThread:485 [handler.py:handle_request():146] handle_request: status_report
6796
+ 2024-04-12 10:28:39,132 INFO Thread-12 :485 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240412_074044-widovcn3/files/output.log
wandb/run-20240412_074044-widovcn3/run-widovcn3.wandb CHANGED
Binary files a/wandb/run-20240412_074044-widovcn3/run-widovcn3.wandb and b/wandb/run-20240412_074044-widovcn3/run-widovcn3.wandb differ