File size: 1,023 Bytes
65e6cde e3e8f9a 65e6cde 7c25fd6 65e6cde |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
---
license: mit
---
# **Phi-3.5-vision-mlx-int4**
This is a quantized INT4 model based on Apple MLX Framework Phi-3.5-vision-instruct. You can deploy it on Apple Silicon devices (M1,M2,M3).
<b><span style="text-decoration:underline">Note: This is unoffical version,just for test and dev.</span></b>
## **Installation**
```bash
pip install -U mlx-vlm
```
## **Conversion**
```bash
python -m mlxv_lm.convert --hf-path microsoft/Phi-3.5-vision-instruct -q
```
## **Samples**
```python
import mlx.core as mx
from mlx_vlm import load, generate
model_path = "Your Phi-3.5-vision-mlx-int4 Path"
model, processor = load(model_path,processor_config={"trust_remote_code":"True"})
messages = [
{"role": "user", "content": "<|image|>\n Summarize the pic"},
]
prompt = processor.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True,
)
output = generate(model, processor, "Your Image Path", placeholder+prompt, verbose=False, max_tokens=1024)
output
``` |