Update README.md
Browse files
README.md
CHANGED
|
@@ -88,6 +88,19 @@ text = "Which company does this image represent?"
|
|
| 88 |
|
| 89 |
```
|
| 90 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
### Generate the model
|
| 92 |
Here is the sample command to reproduce the model.
|
| 93 |
```bash
|
|
|
|
| 88 |
|
| 89 |
```
|
| 90 |
|
| 91 |
+
## Evaluation the model
|
| 92 |
+
pip3 install git+https://github.com/open-compass/VLMEvalKit.git@7de2dcb.
|
| 93 |
+
```bash
|
| 94 |
+
auto-round-mllm --eval --model OPEA/Molmo-7B-D-0924-int4-sym-inc --tasks MMBench_DEV_EN_V11,ScienceQA_VAL,TextVQA_VAL,POPE --output_dir "./eval_result"
|
| 95 |
+
```
|
| 96 |
+
|Metric |16bits| INT4 |
|
| 97 |
+
|:------------------|:-----|:------|
|
| 98 |
+
|avg |69.22 |68.90 |
|
| 99 |
+
|MMBench_DEV_EN_V11 |70.89 |69.20 |
|
| 100 |
+
|ScienceQA_VAL |35.81 |36.34 |
|
| 101 |
+
|TextVQA_VAL |81.16 |80.64 |
|
| 102 |
+
|POPE |89.02 |89.40 |
|
| 103 |
+
|
| 104 |
### Generate the model
|
| 105 |
Here is the sample command to reproduce the model.
|
| 106 |
```bash
|