StevenZhang a-r-r-o-w HF staff commited on
Commit
1a71ab6
·
verified ·
1 Parent(s): 5af5c01

Update README.md (#2)

Browse files

- Update README.md (b066818a73d200d3d2bebf3a4744020646f4f438)


Co-authored-by: Aryan V S <[email protected]>

Files changed (1) hide show
  1. README.md +28 -2
README.md CHANGED
@@ -54,13 +54,13 @@ This repository hosts our T2V-1.3B model, a versatile solution for video generat
54
  - [x] Multi-GPU Inference code of the 14B and 1.3B models
55
  - [x] Checkpoints of the 14B and 1.3B models
56
  - [x] Gradio demo
57
- - [ ] Diffusers integration
58
  - [ ] ComfyUI integration
59
  - Wan2.1 Image-to-Video
60
  - [x] Multi-GPU Inference code of the 14B model
61
  - [x] Checkpoints of the 14B model
62
  - [x] Gradio demo
63
- - [ ] Diffusers integration
64
  - [ ] ComfyUI integration
65
 
66
 
@@ -163,6 +163,32 @@ pip install "xfuser>=0.4.1"
163
  torchrun --nproc_per_node=8 generate.py --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --dit_fsdp --t5_fsdp --ulysses_size 8 --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."
164
  ```
165
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
166
 
167
  ##### (2) Using Prompt Extention
168
 
 
54
  - [x] Multi-GPU Inference code of the 14B and 1.3B models
55
  - [x] Checkpoints of the 14B and 1.3B models
56
  - [x] Gradio demo
57
+ - [x] Diffusers integration
58
  - [ ] ComfyUI integration
59
  - Wan2.1 Image-to-Video
60
  - [x] Multi-GPU Inference code of the 14B model
61
  - [x] Checkpoints of the 14B model
62
  - [x] Gradio demo
63
+ - [x] Diffusers integration
64
  - [ ] ComfyUI integration
65
 
66
 
 
163
  torchrun --nproc_per_node=8 generate.py --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --dit_fsdp --t5_fsdp --ulysses_size 8 --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."
164
  ```
165
 
166
+ Wan can also be run directly using 🤗 Diffusers!
167
+
168
+ ```python
169
+ import torch
170
+ from diffusers import AutoencoderKLWan, WanPipeline
171
+ from diffusers.utils import export_to_video
172
+
173
+ # Available models: Wan-AI/Wan2.1-T2V-14B-Diffusers, Wan-AI/Wan2.1-T2V-1.3B-Diffusers
174
+ model_id = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
175
+ vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
176
+ pipe = WanPipeline.from_pretrained(model_id, vae=vae, torch_dtype=torch.bfloat16)
177
+ pipe.to("cuda")
178
+
179
+ prompt = "A cat walks on the grass, realistic"
180
+ negative_prompt = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"
181
+
182
+ output = pipe(
183
+ prompt=prompt,
184
+ negative_prompt=negative_prompt,
185
+ height=480,
186
+ width=832,
187
+ num_frames=81,
188
+ guidance_scale=5.0
189
+ ).frames[0]
190
+ export_to_video(output, "output.mp4", fps=15)
191
+ ```
192
 
193
  ##### (2) Using Prompt Extention
194