File size: 2,004 Bytes
0b67277 2372cf7 0b67277 7ead06b 0b67277 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
- Tuned by
- https://huggingface.co./datasets/svjack/video-dataset-Lily-Bikini-organized
- To test Mochi-1 have ability to learn concept (object or person) in tiny dataset (trained on low resolution)
# Installtion
```bash
pip install git+https://github.com/huggingface/diffusers.git peft transformers torch sentencepiece opencv-python
```
# Example
## LandScape Example
```python
from diffusers import MochiPipeline
from diffusers.utils import export_to_video
import torch
pipe = MochiPipeline.from_pretrained("genmo/mochi-1-preview", torch_dtype = torch.float16)
pipe.load_lora_weights("svjack/mochi_Lily_Bikini_early_lora")
pipe.enable_model_cpu_offload()
pipe.enable_sequential_cpu_offload()
pipe.vae.enable_slicing()
pipe.vae.enable_tiling()
i = 50
generator = torch.Generator("cpu").manual_seed(i)
prompt = "Lily: The video features a woman with blonde hair wearing a black one-piece swimsuit. She is standing on a sandy beach with the ocean in the background, where waves are visible crashing onto the shore. The sky is clear with a few scattered clouds, suggesting a sunny day. The woman appears to be holding a piece of driftwood or a similar object in her right hand. Her stance and expression suggest she is posing for the camera."
pipeline_args = {
"prompt": prompt,
"num_inference_steps": 64,
"height": 480,
"width": 848,
"max_sequence_length": 1024,
"output_type": "np",
"num_frames": 19,
"generator": generator
}
video = pipe(**pipeline_args).frames[0]
export_to_video(video, "Lily_Lora.mp4")
from IPython import display
display.clear_output(wait = True)
display.Video("Lily_Lora.mp4")
```
- With lora
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/634dffc49b777beec3bc6448/8gjaB5OazSPkHFeuXgasM.mp4"></video>
- With lora + Upscale
<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/634dffc49b777beec3bc6448/VVafjjROHoTubGK7H2z99.mp4"></video>
|