Update model card
Browse filesUpdated the model card with information from the Github README, including the latest news, the models zoo, and additional usage instructions. Also, added `transformers` and `diffusers` to the library name, as indicated in the config files.
README.md
CHANGED
@@ -1,224 +1,291 @@
|
|
1 |
-
---
|
2 |
-
|
3 |
-
|
4 |
-
|
5 |
-
|
6 |
-
|
7 |
-
|
8 |
-
|
9 |
-
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
---
|
14 |
-
|
15 |
-
<p align="center">
|
16 |
-
<img src="./assets/images/teaser.jpg">
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
<a href=https://
|
22 |
-
<a href=https://huggingface.co/tencent/Hunyuan3D-2
|
23 |
-
<a href=https://
|
24 |
-
<a href=https://
|
25 |
-
|
26 |
-
|
27 |
-
|
28 |
-
|
29 |
-
|
30 |
-
|
31 |
-
|
32 |
-
|
33 |
-
|
34 |
-
|
35 |
-
|
36 |
-
|
37 |
-
|
38 |
-
|
39 |
-
|
40 |
-
|
41 |
-
|
42 |
-
|
43 |
-
|
44 |
-
|
45 |
-
|
46 |
-
|
47 |
-
|
48 |
-
|
49 |
-
|
50 |
-
|
51 |
-
|
52 |
-
|
53 |
-
|
54 |
-
|
55 |
-
|
56 |
-
|
57 |
-
|
58 |
-
|
59 |
-
|
60 |
-
|
61 |
-
|
62 |
-
|
63 |
-
|
64 |
-
|
65 |
-
|
66 |
-
|
67 |
-
|
68 |
-
|
69 |
-
|
70 |
-
|
71 |
-
|
72 |
-
|
73 |
-
|
74 |
-
|
75 |
-
|
76 |
-
|
77 |
-
|
78 |
-
|
79 |
-
|
80 |
-
|
81 |
-
|
82 |
-
|
83 |
-
|
84 |
-
|
85 |
-
|
86 |
-
|
87 |
-
|
88 |
-
|
89 |
-
|
90 |
-
|
91 |
-
|
92 |
-
|
93 |
-
|
94 |
-
|
95 |
-
|
96 |
-
|
97 |
-
|
98 |
-
|
99 |
-
|
100 |
-
|
101 |
-
|
102 |
-
|
103 |
-
|
104 |
-
|
105 |
-
|
106 |
-
|
107 |
-
|
108 |
-
|
109 |
-
|
110 |
-
|
111 |
-
|
112 |
-
|
113 |
-
|
114 |
-
|
115 |
-
|
116 |
-
|
117 |
-
|
118 |
-
|
119 |
-
|
120 |
-
|
121 |
-
|
122 |
-
|
123 |
-
|
124 |
-
|
125 |
-
|
126 |
-
|
127 |
-
|
128 |
-
|
129 |
-
|
130 |
-
|
131 |
-
|
132 |
-
|
133 |
-
|
134 |
-
|
135 |
-
|
136 |
-
|
137 |
-
|
138 |
-
|
139 |
-
|
140 |
-
|
141 |
-
|
142 |
-
|
143 |
-
|
144 |
-
|
145 |
-
|
146 |
-
|
147 |
-
|
148 |
-
|
149 |
-
|
150 |
-
|
151 |
-
|
152 |
-
|
153 |
-
|
154 |
-
|
155 |
-
|
156 |
-
|
157 |
-
|
158 |
-
|
159 |
-
###
|
160 |
-
|
161 |
-
|
162 |
-
|
163 |
-
|
164 |
-
|
165 |
-
|
166 |
-
```
|
167 |
-
|
168 |
-
|
169 |
-
|
170 |
-
|
171 |
-
|
172 |
-
|
173 |
-
|
174 |
-
|
175 |
-
|
176 |
-
-
|
177 |
-
|
178 |
-
|
179 |
-
|
180 |
-
|
181 |
-
|
182 |
-
|
183 |
-
|
184 |
-
|
185 |
-
|
186 |
-
|
187 |
-
|
188 |
-
|
189 |
-
|
190 |
-
|
191 |
-
|
192 |
-
|
193 |
-
|
194 |
-
|
195 |
-
|
196 |
-
|
197 |
-
|
198 |
-
|
199 |
-
|
200 |
-
|
201 |
-
|
202 |
-
|
203 |
-
|
204 |
-
|
205 |
-
|
206 |
-
|
207 |
-
|
208 |
-
|
209 |
-
|
210 |
-
|
211 |
-
|
212 |
-
|
213 |
-
|
214 |
-
|
215 |
-
|
216 |
-
|
217 |
-
|
218 |
-
|
219 |
-
|
220 |
-
|
221 |
-
|
222 |
-
|
223 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
224 |
</a>
|
|
|
1 |
+
---
|
2 |
+
language:
|
3 |
+
- en
|
4 |
+
- zh
|
5 |
+
library_name: transformers, diffusers
|
6 |
+
license: other
|
7 |
+
license_name: tencent-hunyuan-community
|
8 |
+
license_link: https://huggingface.co/tencent/Hunyuan3D-2/blob/main/LICENSE.txt
|
9 |
+
pipeline_tag: image-to-3d
|
10 |
+
tags:
|
11 |
+
- image-to-3d
|
12 |
+
- text-to-3d
|
13 |
+
---
|
14 |
+
|
15 |
+
<p align="center">
|
16 |
+
<img src="./assets/images/teaser.jpg">
|
17 |
+
|
18 |
+
</p>
|
19 |
+
|
20 |
+
<div align="center">
|
21 |
+
<a href=https://3d.hunyuan.tencent.com target="_blank"><img src=https://img.shields.io/badge/Official%20Site-333399.svg?logo=homepage height=22px></a>
|
22 |
+
<a href=https://huggingface.co/spaces/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Demo-276cb4.svg height=22px></a>
|
23 |
+
<a href=https://huggingface.co/tencent/Hunyuan3D-2 target="_blank"><img src=https://img.shields.io/badge/%F0%9F%A4%97%20Models-d96902.svg height=22px></a>
|
24 |
+
<a href=https://3d-models.hunyuan.tencent.com/ target="_blank"><img src= https://img.shields.io/badge/Page-bb8a2e.svg?logo=github height=22px></a>
|
25 |
+
<a href=https://discord.gg/dNBrdrGGMa target="_blank"><img src= https://img.shields.io/badge/Discord-white.svg?logo=discord height=22px></a>
|
26 |
+
<a href=https://arxiv.org/abs/2501.12202 target="_blank"><img src=https://img.shields.io/badge/Report-b5212f.svg?logo=arxiv height=22px></a>
|
27 |
+
<a href=https://x.com/txhunyuan target="_blank"><img src=https://img.shields.io/badge/Hunyuan-black.svg?logo=x height=22px></a>
|
28 |
+
<a href="#community-resources" target="_blank"><img src=https://img.shields.io/badge/Community-lavender.svg?logo=homeassistantcommunitystore height=22px></a>
|
29 |
+
</div>
|
30 |
+
|
31 |
+
<br>
|
32 |
+
|
33 |
+
|
34 |
+
> Join our **[Wechat](#)** and **[Discord](https://discord.gg/dNBrdrGGMa)** group to discuss and find help from us.
|
35 |
+
|
36 |
+
| Wechat Group | Xiaohongshu | X | Discord |
|
37 |
+
|--------------------------------------------------|-------------------------------------------------------|---------------------------------------------|---------------------------------------------------|
|
38 |
+
| <img src="assets/qrcode/wechat.png" height=140> | <img src="assets/qrcode/xiaohongshu.png" height=140> | <img src="assets/qrcode/x.png" height=140> | <img src="assets/qrcode/discord.png" height=140> |
|
39 |
+
|
40 |
+
---
|
41 |
+
|
42 |
+
<p align="center">
|
43 |
+
β Living out everyoneβs imagination on creating and manipulating 3D assets.β
|
44 |
+
</p>
|
45 |
+
|
46 |
+
This repository contains the models of the paper [Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation](https://hf.co/papers/2501.12202).
|
47 |
+
For code and more details on how to use it, refer to the [Github repository](https://github.com/Tencent/Hunyuan3D-2).
|
48 |
+
|
49 |
+
|
50 |
+
## π₯ News
|
51 |
+
|
52 |
+
- Feb 14, 2025: π οΈ Release texture enhancement module, please obtain high-definition textures via [here](minimal_demo.py)!
|
53 |
+
- Feb 3, 2025: π
|
54 |
+
Release [Hunyuan3D-DiT-v2-0-Fast](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0-fast), our
|
55 |
+
guidance distillation model that could half the dit inference time, see [here](minimal_demo.py) for usage.
|
56 |
+
- Jan 27, 2025: π οΈ Release Blender addon for Hunyuan3D 2.0, Check it out [here](#blender-addon).
|
57 |
+
- Jan 23, 2025: π¬ We thank community members for
|
58 |
+
creating [Windows installation tool](https://github.com/YanWenKun/Hunyuan3D-2-WinPortable), ComfyUI support
|
59 |
+
with [ComfyUI-Hunyuan3DWrapper](https://github.com/kijai/ComfyUI-Hunyuan3DWrapper)
|
60 |
+
and [ComfyUI-3D-Pack](https://github.com/MrForExample/ComfyUI-3D-Pack) and other
|
61 |
+
awesome [extensions](#community-resources).
|
62 |
+
- Jan 21, 2025: π¬ Enjoy exciting 3D generation on our website [Hunyuan3D Studio](https://3d.hunyuan.tencent.com)!
|
63 |
+
- Jan 21, 2025: π€ Release inference code and pretrained models
|
64 |
+
of [Hunyuan3D 2.0](https://huggingface.co/tencent/Hunyuan3D-2). Please give it a try
|
65 |
+
via [huggingface space](https://huggingface.co/spaces/tencent/Hunyuan3D-2) and
|
66 |
+
our [official site](https://3d.hunyuan.tencent.com)!
|
67 |
+
|
68 |
+
## **Abstract**
|
69 |
+
|
70 |
+
We present Hunyuan3D 2.0, an advanced large-scale 3D synthesis system for generating high-resolution textured 3D assets.
|
71 |
+
This system includes two foundation components: a large-scale shape generation model - Hunyuan3D-DiT, and a large-scale
|
72 |
+
texture synthesis model - Hunyuan3D-Paint.
|
73 |
+
The shape generative model, built on a scalable flow-based diffusion transformer, aims to create geometry that properly
|
74 |
+
aligns with a given condition image, laying a solid foundation for downstream applications.
|
75 |
+
The texture synthesis model, benefiting from strong geometric and diffusion priors, produces high-resolution and vibrant
|
76 |
+
texture maps for either generated or hand-crafted meshes.
|
77 |
+
Furthermore, we build Hunyuan3D-Studio - a versatile, user-friendly production platform that simplifies the re-creation
|
78 |
+
process of 3D assets. It allows both professional and amateur users to manipulate or even animate their meshes
|
79 |
+
efficiently.
|
80 |
+
We systematically evaluate our models, showing that Hunyuan3D 2.0 outperforms previous state-of-the-art models,
|
81 |
+
including the open-source models and closed-source models in geometry details, condition alignment, texture quality, and
|
82 |
+
e.t.c.
|
83 |
+
|
84 |
+
|
85 |
+
|
86 |
+
<p align="center">
|
87 |
+
<img src="assets/images/system.jpg">
|
88 |
+
</p>
|
89 |
+
|
90 |
+
## β―οΈ **Hunyuan3D 2.0**
|
91 |
+
|
92 |
+
### Architecture
|
93 |
+
|
94 |
+
Hunyuan3D 2.0 features a two-stage generation pipeline, starting with the creation of a bare mesh, followed by the
|
95 |
+
synthesis of a texture map for that mesh. This strategy is effective for decoupling the difficulties of shape and
|
96 |
+
texture generation and also provides flexibility for texturing either generated or handcrafted meshes.
|
97 |
+
|
98 |
+
<p align="left">
|
99 |
+
<img src="assets/images/arch.jpg">
|
100 |
+
</p>
|
101 |
+
|
102 |
+
### Performance
|
103 |
+
|
104 |
+
We have evaluated Hunyuan3D 2.0 with other open-source as well as close-source 3d-generation methods.
|
105 |
+
The numerical results indicate that Hunyuan3D 2.0 surpasses all baselines in the quality of generated textured 3D assets
|
106 |
+
and the condition following ability.
|
107 |
+
|
108 |
+
| Model | CMMD(β¬) | FID_CLIP(β¬) | FID(β¬) | CLIP-score(β¬) |
|
109 |
+
|-------------------------|-----------|-------------|-------------|---------------|
|
110 |
+
| Top Open-source Model1 | 3.591 | 54.639 | 289.287 | 0.787 |
|
111 |
+
| Top Close-source Model1 | 3.600 | 55.866 | 305.922 | 0.779 |
|
112 |
+
| Top Close-source Model2 | 3.368 | 49.744 | 294.628 | 0.806 |
|
113 |
+
| Top Close-source Model3 | 3.218 | 51.574 | 295.691 | 0.799 |
|
114 |
+
| Hunyuan3D 2.0 | **3.193** | **49.165** | **282.429** | **0.809** |
|
115 |
+
|
116 |
+
Generation results of Hunyuan3D 2.0:
|
117 |
+
<p align="left">
|
118 |
+
<img src="assets/images/e2e-1.gif" height=250>
|
119 |
+
<img src="assets/images/e2e-2.gif" height=250>
|
120 |
+
</p>
|
121 |
+
|
122 |
+
|
123 |
+
## π Models Zoo
|
124 |
+
|
125 |
+
It takes 11.5 GB VRAM for shape generation and 24.5 GB for shape and texture generation in total.
|
126 |
+
|
127 |
+
| Model | Description | Date | Size | Huggingface |
|
128 |
+
|-------------------------|-----------------------------|------------|------|------------------------------------------------------------------------------------------|
|
129 |
+
| Hunyuan3D-DiT-v2-0-Fast | Guidance Distillation Model | 2025-02-03 | 2.6B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0-fast) |
|
130 |
+
| Hunyuan3D-DiT-v2-0 | Image to Shape Model | 2025-01-21 | 2.6B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-dit-v2-0) |
|
131 |
+
| Hunyuan3D-Paint-v2-0 | Texture Generation Model | 2025-01-21 | 1.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-paint-v2-0) |
|
132 |
+
| Hunyuan3D-Delight-v2-0 | Image Delight Model | 2025-01-21 | 1.3B | [Download](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) |
|
133 |
+
|
134 |
+
|
135 |
+
## π€ Get Started with Hunyuan3D 2.0
|
136 |
+
|
137 |
+
You may follow the next steps to use Hunyuan3D 2.0 via:
|
138 |
+
|
139 |
+
- [Code](#code-usage)
|
140 |
+
- [Gradio App](#gradio-app)
|
141 |
+
- [API Server](#api-server)
|
142 |
+
- [Blender Addon](#blender-addon)
|
143 |
+
- [Official Site](#official-site)
|
144 |
+
|
145 |
+
### Install Requirements
|
146 |
+
|
147 |
+
Please install Pytorch via the [official](https://pytorch.org/) site. Then install the other requirements via
|
148 |
+
|
149 |
+
```bash
|
150 |
+
pip install -r requirements.txt
|
151 |
+
# for texture
|
152 |
+
cd hy3dgen/texgen/custom_rasterizer
|
153 |
+
python3 setup.py install
|
154 |
+
cd ../../..
|
155 |
+
cd hy3dgen/texgen/differentiable_renderer
|
156 |
+
python3 setup.py install
|
157 |
+
```
|
158 |
+
|
159 |
+
### Code Usage
|
160 |
+
|
161 |
+
We designed a diffusers-like API to use our shape generation model - Hunyuan3D-DiT and texture synthesis model -
|
162 |
+
Hunyuan3D-Paint.
|
163 |
+
|
164 |
+
You could assess **Hunyuan3D-DiT** via:
|
165 |
+
|
166 |
+
```python
|
167 |
+
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
|
168 |
+
|
169 |
+
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
|
170 |
+
mesh = pipeline(image='assets/demo.png')[0]
|
171 |
+
```
|
172 |
+
|
173 |
+
The output mesh is a [trimesh object](https://trimesh.org/trimesh.html), which you could save to glb/obj (or other
|
174 |
+
format) file.
|
175 |
+
|
176 |
+
For **Hunyuan3D-Paint**, do the following:
|
177 |
+
|
178 |
+
```python
|
179 |
+
from hy3dgen.texgen import Hunyuan3DPaintPipeline
|
180 |
+
from hy3dgen.shapegen import Hunyuan3DDiTFlowMatchingPipeline
|
181 |
+
|
182 |
+
# let's generate a mesh first
|
183 |
+
pipeline = Hunyuan3DDiTFlowMatchingPipeline.from_pretrained('tencent/Hunyuan3D-2')
|
184 |
+
mesh = pipeline(image='assets/demo.png')[0]
|
185 |
+
|
186 |
+
pipeline = Hunyuan3DPaintPipeline.from_pretrained('tencent/Hunyuan3D-2')
|
187 |
+
mesh = pipeline(mesh, image='assets/demo.png')
|
188 |
+
```
|
189 |
+
|
190 |
+
Please visit [minimal_demo.py](minimal_demo.py) for more advanced usage, such as **text to 3D** and **texture generation
|
191 |
+
for handcrafted mesh**.
|
192 |
+
|
193 |
+
### Gradio App
|
194 |
+
|
195 |
+
You could also host a [Gradio](https://www.gradio.app/) App in your own computer via:
|
196 |
+
|
197 |
+
```bash
|
198 |
+
python3 gradio_app.py
|
199 |
+
```
|
200 |
+
|
201 |
+
### API Server
|
202 |
+
|
203 |
+
You could launch an API server locally, which you could post web request for Image/Text to 3D, Texturing existing mesh,
|
204 |
+
and e.t.c.
|
205 |
+
|
206 |
+
```bash
|
207 |
+
python api_server.py --host 0.0.0.0 --port 8080
|
208 |
+
```
|
209 |
+
|
210 |
+
A demo post request for image to 3D without texture.
|
211 |
+
|
212 |
+
```bash
|
213 |
+
img_b64_str=$(base64 -i assets/demo.png)
|
214 |
+
curl -X POST "http://localhost:8080/generate" \
|
215 |
+
-H "Content-Type: application/json" \
|
216 |
+
-d '{
|
217 |
+
"image": "'"$img_b64_str"'",
|
218 |
+
}' \
|
219 |
+
-o test2.glb
|
220 |
+
```
|
221 |
+
|
222 |
+
### Blender Addon
|
223 |
+
|
224 |
+
With an API server launched, you could also directly use Hunyuan3D 2.0 in your blender with
|
225 |
+
our [Blender Addon](blender_addon.py). Please follow our tutorial to install and use.
|
226 |
+
|
227 |
+
https://github.com/user-attachments/assets/8230bfb5-32b1-4e48-91f4-a977c54a4f3e
|
228 |
+
|
229 |
+
### Official Site
|
230 |
+
|
231 |
+
Don't forget to visit [Hunyuan3D](https://3d.hunyuan.tencent.com) for quick use, if you don't want to host yourself.
|
232 |
+
|
233 |
+
## π Open-Source Plan
|
234 |
+
|
235 |
+
- [x] Inference Code
|
236 |
+
- [x] Model Checkpoints
|
237 |
+
- [x] Technical Report
|
238 |
+
- [ ] ComfyUI
|
239 |
+
- [ ] TensorRT Version
|
240 |
+
|
241 |
+
## π BibTeX
|
242 |
+
|
243 |
+
If you found this repository helpful, please cite our reports:
|
244 |
+
|
245 |
+
```bibtex
|
246 |
+
@misc{hunyuan3d22025tencent,
|
247 |
+
title={Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation},
|
248 |
+
author={Tencent Hunyuan3D Team},
|
249 |
+
year={2025},
|
250 |
+
eprint={2501.12202},
|
251 |
+
archivePrefix={arXiv},
|
252 |
+
primaryClass={cs.CV}
|
253 |
+
}
|
254 |
+
|
255 |
+
@misc{yang2024hunyuan3d,
|
256 |
+
title={Hunyuan3D 1.0: A Unified Framework for Text-to-3D and Image-to-3D Generation},
|
257 |
+
author={Tencent Hunyuan3D Team},
|
258 |
+
year={2024},
|
259 |
+
eprint={2411.02293},
|
260 |
+
archivePrefix={arXiv},
|
261 |
+
primaryClass={cs.CV}
|
262 |
+
}
|
263 |
+
```
|
264 |
+
|
265 |
+
## Community Resources
|
266 |
+
|
267 |
+
Thanks for the contributions of community members, here we have these great extensions of Hunyuan3D 2.0:
|
268 |
+
|
269 |
+
- [ComfyUI-3D-Pack](https://github.com/MrForExample/ComfyUI-3D-Pack)
|
270 |
+
- [ComfyUI-Hunyuan3DWrapper](https://github.com/kijai/ComfyUI-Hunyuan3DWrapper)
|
271 |
+
- [Hunyuan3D-2-for-windows](https://github.com/sdbds/Hunyuan3D-2-for-windows)
|
272 |
+
- [π¦ A bundle for running on Windows | ζ΄εε
](https://github.com/YanWenKun/Hunyuan3D-2-WinPortable)
|
273 |
+
- [Hunyuan3D-2GP](https://github.com/deepbeepmeep/Hunyuan3D-2GP)
|
274 |
+
- [Kaggle Notebook](https://github.com/darkon12/Hunyuan3D-2GP_Kaggle)
|
275 |
+
|
276 |
+
## Acknowledgements
|
277 |
+
|
278 |
+
We would like to thank the contributors to
|
279 |
+
the [DINOv2](https://github.com/facebookresearch/dinov2), [Stable Diffusion](https://github.com/Stability-AI/stablediffusion), [FLUX](https://github.com/black-forest-labs/flux), [diffusers](https://github.com/huggingface/diffusers), [HuggingFace](https://huggingface.co), [CraftsMan3D](https://github.com/wyysf-98/CraftsMan3D),
|
280 |
+
and [Michelangelo](https://github.com/NeuralCarver/Michelangelo/tree/main) repositories, for their open research and
|
281 |
+
exploration.
|
282 |
+
|
283 |
+
## Star History
|
284 |
+
|
285 |
+
<a href="https://star-history.com/#Tencent/Hunyuan3D-2&Date">
|
286 |
+
<picture>
|
287 |
+
<source media="(prefers-color-scheme: dark)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date&theme=dark" />
|
288 |
+
<source media="(prefers-color-scheme: light)" srcset="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
|
289 |
+
<img alt="Star History Chart" src="https://api.star-history.com/svg?repos=Tencent/Hunyuan3D-2&type=Date" />
|
290 |
+
</picture>
|
291 |
</a>
|