File size: 22,309 Bytes
3814f26 00948e4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 |
2024-03-22 04:34:07,566 INFO StreamThr :4275 [internal.py:wandb_internal():86] W&B internal server running at pid: 4275, started at: 2024-03-22 04:34:07.562793
2024-03-22 04:34:07,589 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status
2024-03-22 04:34:12,594 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:12,594 INFO WriterThread:4275 [datastore.py:open_for_write():87] open: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/run-hize5f4p.wandb
2024-03-22 04:34:12,609 DEBUG SenderThread:4275 [sender.py:send():378] send: header
2024-03-22 04:34:12,616 DEBUG SenderThread:4275 [sender.py:send():378] send: run
2024-03-22 04:34:13,047 INFO SenderThread:4275 [dir_watcher.py:__init__():211] watching files in: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files
2024-03-22 04:34:13,047 INFO SenderThread:4275 [sender.py:_start_run_threads():1099] run started: hize5f4p with start time 1711082047.565462
2024-03-22 04:34:13,064 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: check_version
2024-03-22 04:34:13,064 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: check_version
2024-03-22 04:34:13,182 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: run_start
2024-03-22 04:34:13,192 DEBUG HandlerThread:4275 [system_info.py:__init__():26] System info init
2024-03-22 04:34:13,193 DEBUG HandlerThread:4275 [system_info.py:__init__():41] System info init done
2024-03-22 04:34:13,193 INFO HandlerThread:4275 [system_monitor.py:start():194] Starting system monitor
2024-03-22 04:34:13,193 INFO SystemMonitor:4275 [system_monitor.py:_start():158] Starting system asset monitoring threads
2024-03-22 04:34:13,193 INFO HandlerThread:4275 [system_monitor.py:probe():214] Collecting system info
2024-03-22 04:34:13,194 INFO SystemMonitor:4275 [interfaces.py:start():190] Started cpu monitoring
2024-03-22 04:34:13,195 INFO SystemMonitor:4275 [interfaces.py:start():190] Started disk monitoring
2024-03-22 04:34:13,196 INFO SystemMonitor:4275 [interfaces.py:start():190] Started gpu monitoring
2024-03-22 04:34:13,197 INFO SystemMonitor:4275 [interfaces.py:start():190] Started memory monitoring
2024-03-22 04:34:13,198 INFO SystemMonitor:4275 [interfaces.py:start():190] Started network monitoring
2024-03-22 04:34:13,203 DEBUG HandlerThread:4275 [system_info.py:probe():150] Probing system
2024-03-22 04:34:13,208 DEBUG HandlerThread:4275 [gitlib.py:_init_repo():56] git repository is invalid
2024-03-22 04:34:13,208 DEBUG HandlerThread:4275 [system_info.py:probe():198] Probing system done
2024-03-22 04:34:13,208 DEBUG HandlerThread:4275 [system_monitor.py:probe():223] {'os': 'Linux-6.1.58+-x86_64-with-glibc2.35', 'python': '3.10.12', 'heartbeatAt': '2024-03-22T04:34:13.203984', 'startedAt': '2024-03-22T04:34:07.465777', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'Mistral-7B-Finetuning-Insurance.ipynb', 'codePathLocal': None, 'colab': 'https://colab.research.google.com/notebook#fileId=1yggNDPxtSX3bri4ON4ppYc-G-Q2zFtkQ', 'host': '6b98cdd15d10', 'username': 'root', 'executable': '/usr/bin/python3', 'cpu_count': 1, 'cpu_count_logical': 2, 'cpu_freq': {'current': 2000.142, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.142, 'min': 0.0, 'max': 0.0}, {'current': 2000.142, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 78.1898422241211, 'used': 42.876033782958984}}, 'gpu': 'Tesla T4', 'gpu_count': 1, 'gpu_devices': [{'name': 'Tesla T4', 'memory_total': 16106127360}], 'memory': {'total': 12.674789428710938}}
2024-03-22 04:34:13,208 INFO HandlerThread:4275 [system_monitor.py:probe():224] Finished collecting system info
2024-03-22 04:34:13,208 INFO HandlerThread:4275 [system_monitor.py:probe():227] Publishing system info
2024-03-22 04:34:13,226 INFO HandlerThread:4275 [system_monitor.py:probe():229] Finished publishing system info
2024-03-22 04:34:13,232 DEBUG SenderThread:4275 [sender.py:send():378] send: files
2024-03-22 04:34:13,233 INFO SenderThread:4275 [sender.py:_save_file():1365] saving file wandb-metadata.json with policy now
2024-03-22 04:34:13,733 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: python_packages
2024-03-22 04:34:13,733 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: python_packages
2024-03-22 04:34:13,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:34:13,735 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:34:13,741 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:34:13,888 INFO wandb-upload_0:4275 [upload_job.py:push():131] Uploaded file /tmp/tmp3i0vj0_swandb/55hqq2eg-wandb-metadata.json
2024-03-22 04:34:13,972 DEBUG SenderThread:4275 [sender.py:send():378] send: telemetry
2024-03-22 04:34:13,973 DEBUG SenderThread:4275 [sender.py:send():378] send: config
2024-03-22 04:34:13,975 DEBUG SenderThread:4275 [sender.py:send():378] send: metric
2024-03-22 04:34:13,975 DEBUG SenderThread:4275 [sender.py:send():378] send: telemetry
2024-03-22 04:34:13,976 DEBUG SenderThread:4275 [sender.py:send():378] send: metric
2024-03-22 04:34:13,976 WARNING SenderThread:4275 [sender.py:send_metric():1316] Seen metric with glob (shouldn't happen)
2024-03-22 04:34:13,976 DEBUG SenderThread:4275 [sender.py:send():378] send: telemetry
2024-03-22 04:34:14,052 INFO Thread-12 :4275 [dir_watcher.py:_on_file_created():271] file/dir created: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files/requirements.txt
2024-03-22 04:34:14,053 INFO Thread-12 :4275 [dir_watcher.py:_on_file_created():271] file/dir created: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files/wandb-metadata.json
2024-03-22 04:34:17,977 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:22,978 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:27,978 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:28,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:34:28,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:34:28,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:34:33,912 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:38,918 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:40,099 INFO Thread-12 :4275 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files/config.yaml
2024-03-22 04:34:43,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:34:43,736 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:34:43,776 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:34:44,947 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:49,948 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:54,949 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:34:58,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:34:58,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:34:58,774 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:35:00,929 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:05,930 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:10,931 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:13,199 DEBUG SystemMonitor:4275 [system_monitor.py:_start():172] Starting system metrics aggregation loop
2024-03-22 04:35:13,200 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:35:13,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:35:13,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:35:13,774 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:35:16,922 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:21,922 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:26,923 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:28,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:35:28,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:35:28,775 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:35:31,951 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:36,952 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:41,952 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:43,201 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:35:43,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:35:43,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:35:43,775 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:35:47,914 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:52,914 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:57,915 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:35:58,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:35:58,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:35:58,775 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:36:02,958 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:07,958 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:12,959 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:13,202 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:36:13,734 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:36:13,736 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:36:13,776 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:36:18,925 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:23,926 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:28,735 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:36:28,735 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:36:28,777 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:36:28,959 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:33,960 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:38,377 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: partial_history
2024-03-22 04:36:38,384 DEBUG SenderThread:4275 [sender.py:send():378] send: metric
2024-03-22 04:36:38,385 DEBUG SenderThread:4275 [sender.py:send():378] send: metric
2024-03-22 04:36:38,385 DEBUG SenderThread:4275 [sender.py:send():378] send: metric
2024-03-22 04:36:38,385 DEBUG SenderThread:4275 [sender.py:send():378] send: metric
2024-03-22 04:36:38,386 DEBUG SenderThread:4275 [sender.py:send():378] send: history
2024-03-22 04:36:38,386 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: summary_record
2024-03-22 04:36:38,395 INFO SenderThread:4275 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
2024-03-22 04:36:39,314 INFO Thread-12 :4275 [dir_watcher.py:_on_file_created():271] file/dir created: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files/wandb-summary.json
2024-03-22 04:36:39,396 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:43,202 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:36:49,784 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:36:50,358 INFO Thread-12 :4275 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files/config.yaml
2024-03-22 04:36:50,399 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:36:50,401 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:36:50,402 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:36:55,622 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:14,735 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:14,752 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:14,753 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:14,753 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:37:14,753 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:37:14,754 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:37:14,783 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:37:16,284 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:21,289 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:31,399 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:31,407 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:37:31,408 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: internal_messages
2024-03-22 04:37:31,408 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:31,409 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:37:36,568 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:41,569 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:43,204 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:37:44,743 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:37:44,744 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:37:46,973 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:51,974 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:56,975 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:37:59,743 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:37:59,743 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:38:01,989 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:06,990 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:11,990 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:13,205 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:38:14,743 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:38:14,744 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:38:17,990 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:22,991 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:27,992 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:29,743 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:38:29,744 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:38:33,963 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:38,964 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:43,206 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:38:44,207 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:44,744 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:38:44,744 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:38:49,987 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:54,988 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:38:59,743 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:38:59,744 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:39:00,966 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:05,966 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:10,497 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: partial_history
2024-03-22 04:39:10,500 DEBUG SenderThread:4275 [sender.py:send():378] send: history
2024-03-22 04:39:10,501 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: summary_record
2024-03-22 04:39:10,506 INFO SenderThread:4275 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
2024-03-22 04:39:10,715 INFO Thread-12 :4275 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/gdrive/MyDrive/LLM/Mistral-7B-Finetuning-Insurance/wandb/run-20240322_043407-hize5f4p/files/wandb-summary.json
2024-03-22 04:39:11,508 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:13,206 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:39:17,208 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:20,656 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:39:20,657 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:39:22,874 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:27,875 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:37,621 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:37,629 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: stop_status
2024-03-22 04:39:37,630 DEBUG SenderThread:4275 [sender.py:send_request():405] send_request: stop_status
2024-03-22 04:39:38,874 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:43,207 DEBUG SenderThread:4275 [sender.py:send():378] send: stats
2024-03-22 04:39:44,209 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
2024-03-22 04:39:49,213 DEBUG HandlerThread:4275 [handler.py:handle_request():146] handle_request: status_report
|