File size: 1,242 Bytes
0f444c7 a0192bf 12f772a 06f86b5 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
---
license: apache-2.0
---
## Updates 🔥🔥🔥
We have released the Gradio demo for **Hybrid (Trajectory + Landmark)** Controls [HERE](https://huggingface.co./MyNiuuu/MOFA-Video-Hybrid)!
## Introduction
This repo provides the inference Gradio demo for Trajectory Control of MOFA-Video.
## Environment Setup
`pip install -r requirements.txt`
## Download checkpoints
1. Download the pretrained checkpoints of [SVD_xt](https://huggingface.co./stabilityai/stable-video-diffusion-img2vid-xt-1-1) from huggingface to `./ckpts`.
2. Download the checkpint of [MOFA-Adapter](https://huggingface.co./MyNiuuu/MOFA-Video-Traj) from huggingface to `./ckpts`.
The final structure of checkpoints should be:
```text
./ckpts/
|-- controlnet
| |-- config.json
| `-- diffusion_pytorch_model.safetensors
|-- stable-video-diffusion-img2vid-xt-1-1
| |-- feature_extractor
| |-- ...
| |-- image_encoder
| |-- ...
| |-- scheduler
| |-- ...
| |-- unet
| |-- ...
| |-- vae
| |-- ...
| |-- svd_xt_1_1.safetensors
| `-- model_index.json
```
## Run Gradio Demo
`python run_gradio.py`
Please refer to the instructions on the gradio interface during the inference process.
## Paper
arxiv.org/abs/2405.20222 |