Update README.md
Browse files
README.md
CHANGED
|
@@ -14,9 +14,12 @@ Github: https://github.com/VITA-MLLM/Long-VITA
|
|
| 14 |
## 👀 Overview
|
| 15 |
Long-VITA is a strong long-context visual language model and supports more than 1 million tokens.
|
| 16 |
|
| 17 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
|
| 19 |
-
- To infer and evaluate on Nvidia GPUs, we also implemented Long-VITA on Megatron with the Transformer Engine. The converted weight is in https://huggingface.co/VITA-MLLM/Long-VITA-128K_MG.
|
| 20 |
|
| 21 |
|
| 22 |
## 📈 Experimental Results
|
|
@@ -39,6 +42,16 @@ Long-VITA is a strong long-context visual language model and supports more than
|
|
| 39 |
|
| 40 |
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
|
| 43 |
## ACCEPTABLE USE POLICY
|
| 44 |
|
|
|
|
| 14 |
## 👀 Overview
|
| 15 |
Long-VITA is a strong long-context visual language model and supports more than 1 million tokens.
|
| 16 |
|
| 17 |
+
- Long-VITA-128K weights are trained on Ascend NPUs with MindSpeed. The original weight is at https://huggingface.co/VITA-MLLM/Long-VITA-128K.
|
| 18 |
+
|
| 19 |
+
- We also implemented Long-VITA on Megatron with the Transformer Engine to infer and evaluate on Nvidia GPUs. The converted weight is at https://huggingface.co/VITA-MLLM/Long-VITA-128K_MG.
|
| 20 |
+
|
| 21 |
+
- We also implemented Long-VITA on DeepSpeed with the Huggingface Transformers to infer and evaluate on Nvidia GPUs. The converted weight is at https://huggingface.co/VITA-MLLM/Long-VITA-128K_HF.
|
| 22 |
|
|
|
|
| 23 |
|
| 24 |
|
| 25 |
## 📈 Experimental Results
|
|
|
|
| 42 |
|
| 43 |
|
| 44 |
|
| 45 |
+
## Models
|
| 46 |
+
|
| 47 |
+
Model | LLM Size | Training Context | Training Frames | MindSpeed Weights | Megatron Weights | Huggingface Weights
|
| 48 |
+
---------------:|---------:|-----------------:|----------------:|------------------------------------------------:|---------------------------------------------------:|---------------------------------------------------:
|
| 49 |
+
Long-VITA-16K | 14B | 16,384 | 64 | https://huggingface.co/VITA-MLLM/Long-VITA-16K | https://huggingface.co/VITA-MLLM/Long-VITA-16K_MG | https://huggingface.co/VITA-MLLM/Long-VITA-16K_HF
|
| 50 |
+
Long-VITA-128K | 14B | 131,072 | 512 | https://huggingface.co/VITA-MLLM/Long-VITA-128K | https://huggingface.co/VITA-MLLM/Long-VITA-128K_MG | https://huggingface.co/VITA-MLLM/Long-VITA-128K_HF
|
| 51 |
+
Long-VITA-1M | 14B | 1,048,576 | 4,096 | https://huggingface.co/VITA-MLLM/Long-VITA-1M | https://huggingface.co/VITA-MLLM/Long-VITA-1M_MG | https://huggingface.co/VITA-MLLM/Long-VITA-1M_HF
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
|
| 55 |
|
| 56 |
## ACCEPTABLE USE POLICY
|
| 57 |
|