Spaces:
Running
Error occured. local variable 'pipe' referenced before assignment
Hello!
@John6666
So I keep getting this error when I try to convert one of my models models, not sure what the problem is.
Anyways here is the entirety of the console log (the safetensors file is valid Yet it yeilds a headerTooSmall error??). If you are able to go through the code and find any irregularities, it'd be much appreciated!
Also can you add .safetensors Inference in VP?? (So I can inference in . safetensors and diffusers)
Error while deserializing header: HeaderTooSmall
Model type: SDXL / VAE: https://civitai.com/api/download/models/719135 / CLIP: / T5: / Scheduler: Euler a / dtype: bf16 / EMA: True / Base repo: camenduru/FLUX.1-dev-diffusers / LoRAs: {'': 1}
Start downloading: https://civitai.com/api/download/models/719135
Downloading https://civitai.com/api/download/models/719135?$$
[#2faa33 115MiB/159MiB(72%) CN:16 DL:222MiB]
Download Results:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
2faa33|OK | 182MiB/s|/tmp/tmpuct3k3kr/xlVAEC_f2.safetensors
Status Legend:
(OK):download completed.
Download completed: https://civitai.com/api/download/models/719135
Deleted: /tmp/tmpuct3k3kr/xlVAEC_f2.safetensors
Failed to load pipeline. Unable to load weights from checkpoint file for '/tmp/tmpuct3k3kr/.cache/huggingface/.gitignore' at '/tmp/tmpuct3k3kr/.cache/huggingface/.gitignore'.
Failed to convert. [Errno 2] No such file or directory: '/home/user/app/_gitignore/.gitignore'
Error occured. Failed to convert.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/shutil.py", line 816, in move
os.rename(src, real_dst)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpuct3k3kr/.cache/huggingface/.gitignore' -> '/home/user/app/_gitignore/.gitignore'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 383, in convert_url_to_diffusers
if not is_repo_name(new_file) and is_upload_sf: shutil.move(str(Path(new_file).resolve()), str(Path(new_dir, Path(new_file).name).resolve()))
File "/usr/local/lib/python3.10/shutil.py", line 836, in move
copy_function(src, real_dst)
File "/usr/local/lib/python3.10/shutil.py", line 434, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/local/lib/python3.10/shutil.py", line 256, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/app/_gitignore/.gitignore'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 411, in convert_url_to_diffusers_repo
new_path = convert_url_to_diffusers(dl_url, civitai_key, is_upload_sf, dtype, vae, clip, t5, scheduler, ema, image_size, safety_checker, base_repo, mtype, lora_dict, is_local)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 390, in convert_url_to_diffusers
raise Exception("Failed to convert.") from e
Exception: Failed to convert.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 2047, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1594, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 428, in convert_url_to_diffusers_repo
raise gr.Error(f"Error occured. {e}")
gradio.exceptions.Error: 'Error occured. Failed to convert.'
Start downloading:
Downloading
to /tmp/tmpuct3k3kr
Download failed:
Failed to convert. Safetensors file not found: https://huggingface.co./xi0v/illust-vpred-test-models/resolve/main/20250101_convert_H100bs256rate85e_2debias/rate85e_2debias-step3100.safetensors
Error occured. local variable 'pipe' referenced before assignment
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 314, in convert_url_to_diffusers
if not new_file: raise Exception(f"Safetensors file not found: {url}")
Exception: Safetensors file not found:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 390, in convert_url_to_diffusers
raise Exception("Failed to convert.") from e
Exception: Failed to convert.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 411, in convert_url_to_diffusers_repo
new_path = convert_url_to_diffusers(dl_url, civitai_key, is_upload_sf, dtype, vae, clip, t5, scheduler, ema, image_size, safety_checker, base_repo, mtype, lora_dict, is_local)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 392, in convert_url_to_diffusers
del pipe
UnboundLocalError: local variable 'pipe' referenced before assignment
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 2047, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1594, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 428, in convert_url_to_diffusers_repo
raise gr.Error(f"Error occured. {e}")
gradio.exceptions.Error: "Error occured. local variable 'pipe' referenced before assignment"
Start downloading:
Downloading
to /tmp/tmpuct3k3kr
Download failed:
Failed to convert. Safetensors file not found:
Error occured. local variable 'pipe' referenced before assignment
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 314, in convert_url_to_diffusers
if not new_file: raise Exception(f"Safetensors file not found: {url}")
Exception: Safetensors file not found: https://huggingface.co./xi0v/models/resolve/main/tests/model.safetensors
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 390, in convert_url_to_diffusers
raise Exception("Failed to convert.") from e
Exception: Failed to convert.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 411, in convert_url_to_diffusers_repo
new_path = convert_url_to_diffusers(dl_url, civitai_key, is_upload_sf, dtype, vae, clip, t5, scheduler, ema, image_size, safety_checker, base_repo, mtype, lora_dict, is_local)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 392, in convert_url_to_diffusers
del pipe
UnboundLocalError: local variable 'pipe' referenced before assignment
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 2047, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1594, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 428, in convert_url_to_diffusers_repo
raise gr.Error(f"Error occured. {e}")
gradio.exceptions.Error: "Error occured. local variable 'pipe' referenced before assignment"
This space doesn't currently support VAE downloads from Civitai...😪
Also can you add .safetensors Inference in VP?? (So I can inference in . safetensors and diffusers)
The code itself is not difficult, but it requires modification of the DiffuseCraft-derived part. I'll see if I can do it nicely.
Edit:
From the error message, it seems that the model file download itself failed before the conversion. Instead, .gitigore is being returned...?
Edit:
I made the error handling a little stricter. The problem is why the download is failing. Maybe the URL is not what I expected. Maybe it's falling outside the regular expression pattern for some reason.
# from HF
s = list(re.findall(r'^(?:https?://huggingface.co/)(?:(datasets|spaces)/)?(.+?/.+?)/\w+?/.+?/(?:(.+)/)?(.+?.\w+)(?:\?download=true)?$', url)[0])
I'll see if I can do it nicely
Take your time!
This space doesn't currently support VAE downloads from Civitai...😪
I've been doing it a lot, and I think it does work?
.gitigore is being returned
Not even sure why, there is no .gitignore in my repo
Here is the log:
Start downloading:
Downloading
to /tmp/tmppt8sus_d
Download completed:
Deleted: /tmp/tmppt8sus_d/.cache/huggingface/.gitignore
Failed to convert. Safetensors file not found:
Error occured. Failed to convert. Safetensors file not found:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 317, in convert_url_to_diffusers
raise Exception(f"Safetensors file not found: {url}")
Exception: Safetensors file not found:
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 413, in convert_url_to_diffusers_repo
new_path = convert_url_to_diffusers(dl_url, civitai_key, is_upload_sf, dtype, vae, clip, t5, scheduler, ema, image_size, safety_checker, base_repo, mtype, lora_dict, is_local)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 392, in convert_url_to_diffusers
raise Exception(f"Failed to convert. {e}") from e
Exception: Failed to convert. Safetensors file not found:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 2047, in process_api
result = await self.call_function(
File "/usr/local/lib/python3.10/site-packages/gradio/blocks.py", line 1594, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/usr/local/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2505, in run_sync_in_worker_thread
return await future
File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1005, in run
result = context.run(func, *args)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/gradio/utils.py", line 869, in wrapper
response = f(*args, **kwargs)
File "/home/user/app/convert_url_to_diffusers_multi_gr.py", line 430, in convert_url_to_diffusers_repo
raise gr.Error(f"Error occured. {e}")
gradio.exceptions.Error: 'Error occured. Failed to convert. Safetensors file not found:
Maybe the file name is special, or it's been specified as a revision, or something like that?
The original purpose of the download function was different, so it does not support revision specifications. Or rather, I don't know the specification of the HF revision path.
I've been doing it a lot, and I think it does work?
Oh, I forgot to be supported.😆
Maybe the file name is special, or it's been specified as a revision
Nope, it's a normal file name being grabbed from the latest revision automatically (main)
For example in my case it's: https://huggingface.co./xi0v/model-repo/resolve/main/models-directory/model.safetensors
Oh, I forgot to be supported.😆
😆😆
For example in my case it's: https://huggingface.co./xi0v/model-repo/resolve/main/models-directory/model.safetensors
It's like “the normal file URL”.😅 I don't think there will be a problem with other files with similar file names.
If I had to say, the subdirectory is suspicious, but it shouldn't be a problem...
I'll try some experiments later.
If there really is a problem with the header, as the error message says, it should be possible to try deleting the header and then loading it.
Edit:
It was the file name... but I'm still trying to figure out why...
If I had to say, the subdirectory is suspicious, but it shouldn't be a problem...
I'll seems like the subdirectory is the problem. the conversion is going through normally
I found the cause. Because there is no way to know the exact path before downloading (although it is technically possible, it would be very difficult if it included anything other than HF), I was scanning the files that had increased after downloading, but it seems that the huggingface_hub library had changed its specifications and was creating caches in strange locations, and it was picking up those. I fixed it.
Edit:
I also added a function to download individual files to DCMod and VP. Because r3gm had already done some of the work, it didn't take much effort. I'll switch to the official implementation when it comes out.
Thank you very much!
I'll reopen if I find any issues