2024-07-03 09:12:28 | INFO | model_worker | args: Namespace(awq_ckpt=None, awq_groupsize=-1, awq_wbits=16, controller_address='http://localhost:21001', conv_template=None, cpu_offloading=False, debug=False, device='cuda', dtype=None, embed_in_truncate=False, enable_exllama=False, enable_xft=False, exllama_cache_8bit=False, exllama_gpu_split=None, exllama_max_seq_len=4096, gptq_act_order=False, gptq_ckpt=None, gptq_groupsize=-1, gptq_wbits=16, gpus=None, host='127.0.0.1', limit_worker_concurrency=5, load_8bit=False, max_gpu_memory=None, model_names=None, model_path='lmsys/vicuna-7b-v1.5', no_register=False, num_gpus=1, port=31000, revision='main', seed=None, ssl=False, stream_interval=2, worker_address='http://localhost:31000', xft_dtype=None, xft_max_seq_len=4096)
2024-07-03 09:12:28 | INFO | model_worker | Loading the model ['vicuna-7b-v1.5'] on worker 8276027a ...
2024-07-03 09:12:29 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/torch/storage.py:315: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.
2024-07-03 09:12:29 | ERROR | stderr |   warnings.warn(message, UserWarning)
2024-07-03 09:12:29 | ERROR | stderr | 
Loading checkpoint shards:   0%|                                                                                     | 0/2 [00:00<?, ?it/s]
2024-07-03 09:12:33 | ERROR | stderr | 
Loading checkpoint shards:  50%|██████████████████████████████████████▌                                      | 1/2 [00:04<00:04,  4.65s/it]
2024-07-03 09:12:35 | ERROR | stderr | 
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00,  3.05s/it]
2024-07-03 09:12:35 | ERROR | stderr | 
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████| 2/2 [00:06<00:00,  3.29s/it]
2024-07-03 09:12:35 | ERROR | stderr | 
2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
2024-07-03 09:12:35 | ERROR | stderr |   warnings.warn(
2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.
2024-07-03 09:12:35 | ERROR | stderr |   warnings.warn(
2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:540: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
2024-07-03 09:12:35 | ERROR | stderr |   warnings.warn(
2024-07-03 09:12:35 | ERROR | stderr | /usr/local/lib/python3.8/dist-packages/transformers/generation/configuration_utils.py:545: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`.
2024-07-03 09:12:35 | ERROR | stderr |   warnings.warn(
2024-07-03 09:13:01 | INFO | model_worker | Register to controller
2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last):
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 196, in _new_conn
2024-07-03 09:13:01 | ERROR | stderr |     sock = connection.create_connection(
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 85, in create_connection
2024-07-03 09:13:01 | ERROR | stderr |     raise err
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/util/connection.py", line 73, in create_connection
2024-07-03 09:13:01 | ERROR | stderr |     sock.connect(sa)
2024-07-03 09:13:01 | ERROR | stderr | ConnectionRefusedError: [Errno 111] Connection refused
2024-07-03 09:13:01 | ERROR | stderr | 
2024-07-03 09:13:01 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-07-03 09:13:01 | ERROR | stderr | 
2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last):
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 789, in urlopen
2024-07-03 09:13:01 | ERROR | stderr |     response = self._make_request(
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 495, in _make_request
2024-07-03 09:13:01 | ERROR | stderr |     conn.request(
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 398, in request
2024-07-03 09:13:01 | ERROR | stderr |     self.endheaders()
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
2024-07-03 09:13:01 | ERROR | stderr |     self._send_output(message_body, encode_chunked=encode_chunked)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
2024-07-03 09:13:01 | ERROR | stderr |     self.send(msg)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/lib/python3.8/http/client.py", line 951, in send
2024-07-03 09:13:01 | ERROR | stderr |     self.connect()
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 236, in connect
2024-07-03 09:13:01 | ERROR | stderr |     self.sock = self._new_conn()
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 211, in _new_conn
2024-07-03 09:13:01 | ERROR | stderr |     raise NewConnectionError(
2024-07-03 09:13:01 | ERROR | stderr | urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f1861a58130>: Failed to establish a new connection: [Errno 111] Connection refused
2024-07-03 09:13:01 | ERROR | stderr | 
2024-07-03 09:13:01 | ERROR | stderr | The above exception was the direct cause of the following exception:
2024-07-03 09:13:01 | ERROR | stderr | 
2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last):
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 667, in send
2024-07-03 09:13:01 | ERROR | stderr |     resp = conn.urlopen(
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 843, in urlopen
2024-07-03 09:13:01 | ERROR | stderr |     retries = retries.increment(
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 519, in increment
2024-07-03 09:13:01 | ERROR | stderr |     raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
2024-07-03 09:13:01 | ERROR | stderr | urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=21001): Max retries exceeded with url: /register_worker (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1861a58130>: Failed to establish a new connection: [Errno 111] Connection refused'))
2024-07-03 09:13:01 | ERROR | stderr | 
2024-07-03 09:13:01 | ERROR | stderr | During handling of the above exception, another exception occurred:
2024-07-03 09:13:01 | ERROR | stderr | 
2024-07-03 09:13:01 | ERROR | stderr | Traceback (most recent call last):
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
2024-07-03 09:13:01 | ERROR | stderr |     return _run_code(code, main_globals, None,
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
2024-07-03 09:13:01 | ERROR | stderr |     exec(code, run_globals)
2024-07-03 09:13:01 | ERROR | stderr |   File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 414, in <module>
2024-07-03 09:13:01 | ERROR | stderr |     args, worker = create_model_worker()
2024-07-03 09:13:01 | ERROR | stderr |   File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 385, in create_model_worker
2024-07-03 09:13:01 | ERROR | stderr |     worker = ModelWorker(
2024-07-03 09:13:01 | ERROR | stderr |   File "/LLM_32T/evelyn/FastChat/fastchat/serve/model_worker.py", line 102, in __init__
2024-07-03 09:13:01 | ERROR | stderr |     self.init_heart_beat()
2024-07-03 09:13:01 | ERROR | stderr |   File "/LLM_32T/evelyn/FastChat/fastchat/serve/base_model_worker.py", line 81, in init_heart_beat
2024-07-03 09:13:01 | ERROR | stderr |     self.register_to_controller()
2024-07-03 09:13:01 | ERROR | stderr |   File "/LLM_32T/evelyn/FastChat/fastchat/serve/base_model_worker.py", line 99, in register_to_controller
2024-07-03 09:13:01 | ERROR | stderr |     r = requests.post(url, json=data)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 115, in post
2024-07-03 09:13:01 | ERROR | stderr |     return request("post", url, data=data, json=json, **kwargs)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/requests/api.py", line 59, in request
2024-07-03 09:13:01 | ERROR | stderr |     return session.request(method=method, url=url, **kwargs)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 589, in request
2024-07-03 09:13:01 | ERROR | stderr |     resp = self.send(prep, **send_kwargs)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 703, in send
2024-07-03 09:13:01 | ERROR | stderr |     r = adapter.send(request, **kwargs)
2024-07-03 09:13:01 | ERROR | stderr |   File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 700, in send
2024-07-03 09:13:01 | ERROR | stderr |     raise ConnectionError(e, request=request)
2024-07-03 09:13:01 | ERROR | stderr | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=21001): Max retries exceeded with url: /register_worker (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f1861a58130>: Failed to establish a new connection: [Errno 111] Connection refused'))