Concurrent Thread Failure in Image Inference
When I attempt to concurrently initiate two threads for image inference in one pipe, the process fails and the following information is displayed as logs during the inference.
[2024-11-01 07:32:12,370] INFO in app: {'prompt': '', 'promptEnglish': 'A capybara holding a sign that reads Hello Fast World', 'negative_prompt': '', 'negativePromptEnglish': None, 'style': 'natural', 'size': '1024x1024'}
[2024-11-01 07:32:12,371] INFO in app: cinematic photo A capybara holding a sign that reads Hello Fast World οΌ photograph, film, bokeh, professional, 4k, highly detailed|drawing, painting, crayon, sketch, graphite, impressionist, noisy, blurry, soft, deformed, ugly
4%|ββββ | 1/28 [00:01<00:28, 1.05s/it][2024-11-01 07:32:14,858] INFO in app: {'prompt': '', 'promptEnglish': 'A capybara holding a sign that reads Hello Fast World', 'negative_prompt': '', 'negativePromptEnglish': None, 'style': 'natural', 'size': '1024x1024'}
[2024-11-01 07:32:14,859] INFO in app: cinematic photo A capybara holding a sign that reads Hello Fast World οΌ photograph, film, bokeh, professional, 4k, highly detailed|drawing, painting, crayon, sketch, graphite, impressionist, noisy, blurry, soft, deformed, ugly
64%|βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ | 18/28 [00:32<00:18, 1.82s/it]
[2024-11-01 07:32:46,144] ERROR in app: Exception on /api/text_to_img [POST] | 15/28 [00:28<00:24, 1.86s/it]
Traceback (most recent call last):
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/data/dev/model-checkpoint/stable-diffusion-3.5-large-turbo/app.py", line 121, in route
res["data"] = image_generation(prompt, negative_prompt, width, height, './')
File "/data/dev/model-checkpoint/stable-diffusion-3.5-large-turbo/app.py", line 66, in image_generation
image = pipeline(
File "/root/anaconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/root/anaconda3/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py", line 902, in call
latents = self.scheduler.step(noise_pred, t, latents, return_dict=False)[0]
File "/root/anaconda3/lib/python3.10/site-packages/diffusers/schedulers/scheduling_flow_match_euler_discrete.py", line 295, in step
sigma_next = self.sigmas[self.step_index + 1]
IndexError: index 29 is out of bounds for dimension 0 with size 29
127.0.0.1 - - [01/Nov/2024 07:32:46] "POST /api/text_to_img HTTP/1.1" 500 -
54%|ββββββββββββββββββββββββββββββββββββββββββββββββββββββ | 15/28 [00:29<00:25, 1.95s/it]
[2024-11-01 07:32:46,968] ERROR in app: Exception on /api/text_to_img [POST]
Traceback (most recent call last):
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda3/lib/python3.10/site-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/data/dev/model-checkpoint/stable-diffusion-3.5-large-turbo/app.py", line 121, in route
res["data"] = image_generation(prompt, negative_prompt, width, height, './')
File "/data/dev/model-checkpoint/stable-diffusion-3.5-large-turbo/app.py", line 66, in image_generation
image = pipeline(
File "/root/anaconda3/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/root/anaconda3/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_3/pipeline_stable_diffusion_3.py", line 902, in call
latents = self.scheduler.step(noise_pred, t, latents, return_dict=False)[0]
File "/root/anaconda3/lib/python3.10/site-packages/diffusers/schedulers/scheduling_flow_match_euler_discrete.py", line 295, in step
sigma_next = self.sigmas[self.step_index + 1]
IndexError: index 29 is out of bounds for dimension 0 with size 29
127.0.0.1 - - [01/Nov/2024 07:32:46] "POST /api/text_to_img HTTP/1.1" 500 -