CiaraRowles

/

TemporalNet2

like 122

Hi, having a look now

ok set everything up from scratch, ran it like the instructions, no reproduction,

A) Is your stable diffusion install up to date? (just in case)

b) Are you using inpainting in the script? Does it work without it? oooooh, you mean you want to use inpainting but my API modification is on a version without it?

A valid concern if so, the controlnet version i used is pretty old

Hi there, thank you for your concern regarding the bug. I have the same error using the most recent webui. The same error happens in img2img mode as soon as I am using any control net even directly in the webui without using the temporalvideo.py, and no it is the img2img tab without applying inpainting.

ok set everything up from scratch, ran it like the instructions, no reproduction,

A) Is your stable diffusion install up to date? (just in case)

b) Are you using inpainting in the script? Does it work without it? oooooh, you mean you want to use inpainting but my API modification is on a version without it?

A valid concern if so, the controlnet version i used is pretty old

Thank you for looking into it. Yes, my files are updated. No, I am just using temporalvideo.py downloaded from your files with my values. I modified the prompts, values and control net model (new Hed). softedge_hed, control_v11p_sd15_softedge [a8575a2a]. Migrated to your ControNet branch.

For some reason, it's calling the inpainting request. Do you mean I need to use the old hed (control_hed-fp16 [13fee50b]) module you're using in your script?

edit: even running temporalvideo.py without any modification triggers the issue.

How strange, must be a dependency issue due to the older controlnet commit somewhere.

For some reason, it's calling the inpainting request. Do you mean I need to use the old hed (control_hed-fp16 [13fee50b]) module you're using in your script?

This shouldn't effect it to my knowledge.

I have updated https://github.com/CiaraStrawberry/sd-webui-controlnet-TemporalNet-API to the most recent controlnet version, please drop a message if it continues to reproduce.

Awesome, I will test it and let you know!

So now i have a new error, and PixelPerfect still does not work

Error running process: C:\Stable Diffusion\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-controlnet-TemporalNet-API-main\scripts\controlnet.py
Traceback (most recent call last):
File "C:\Stable Diffusion\A1111 Web UI Autoinstaller\stable-diffusion-webui\modules\scripts.py", line 451, in process
script.process(p, *script_args)
File "C:\Stable Diffusion\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-controlnet-TemporalNet-API-main\scripts\controlnet.py", line 689, in process
input_image, resize_mode_overwrite = Script.choose_input_image(p, unit, idx)
File "C:\Stable Diffusion\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-controlnet-TemporalNet-API-main\scripts\controlnet.py", line 603, in choose_input_image
input_image = HWC3(image['image'])
File "C:\Stable Diffusion\A1111 Web UI Autoinstaller\stable-diffusion-webui\extensions\sd-webui-controlnet-TemporalNet-API-main\annotator\util.py", line 11, in HWC3
assert C == 1 or C == 3 or C == 4
AssertionError

@HdRi I've created a PR against Ciara's github repo which rebases the changes against the latest sd-webui-controlnet commit and works with the the temporalvideo.py file out of the box. https://github.com/JasonGilholme/sd-webui-controlnet-TemporalNet-API/tree/temporalnet2-api-updates

While I was at it, I also posted an updated temporalvideo.py in a PR against the huggingface repo here: https://huggingface.co./CiaraRowles/TemporalNet2/discussions/7 It's got a few niceties so you should be able to call it without having to edit parts of the script. e.g. python temporalvideo.py "sweet, prompt, goes, here" --input-dir /path/to/dir --output-dir /path/to/output --width 1920 --height 1080

Hopefully those two things combined will work for you!

Jase

@JasonGilholme thank you! Can you please elaborate more on your temporalvideo.py arguments? How do we call the models? I don't have control_hed-fp16.yaml (please share it)

btw, it's finally working! using your script. I had to manually insert the new softedge module into the script.

The new version of the script will try it's best to find an available hed model (there's a few versions available out there). As long as you've installed it in the extensions/sd-webui-controlnet/models directory, the new script should be able to find it.

FWIW, I also didn't have the control_hed-fp16 model or config file and I ended up using the control_sd15_hed one instead. The new script was able to select that one automatically for me so you shouldn't need to manually insert it into the script.

Eitherway, it sounds like you got it working.

@JasonGilholme @CiaraRowles , yes it's working thanks again; temporalnetv2 is very impressive compared to the first one! will share more feedback soon. By the way, I am a professional compositor and have been doing extensive testing on all our SD temporal methods so far.

Interesting, what have you found to work the best so far?

I would have PM'd you but it looks like I can't do that on HF.

@JasonGilholme let's take it to Reddit if you like

@JasonGilholme let's take it to Reddit if you like

Would love to join this conversation, what is your reddit?