Added Qwen 1.5 4B
Browse files
README.md
CHANGED
@@ -24,10 +24,12 @@ Right now, only converted the following models:
|
|
24 |
| Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588 |
|
25 |
| Llama 2 7B | 7B | https://huggingface.co/Pelochus/llama2-chat-7b-hf-rk3588 |
|
26 |
| Llama 2 13B | 13B | https://huggingface.co/Pelochus/llama2-chat-13b-hf-rk3588 |
|
|
|
27 |
| TinyLlama v1 (broken) | 1.1B | https://huggingface.co/Pelochus/tinyllama-v1-rk3588 |
|
28 |
|
29 |
-
However, RKLLM also supports Qwen
|
30 |
For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
|
|
|
31 |
|
32 |
## Downloading a model
|
33 |
Use:
|
@@ -51,7 +53,7 @@ All models are optimized.
|
|
51 |
|
52 |
## Future additions
|
53 |
- [x] Converting Llama 2 (70B currently in conversion, but that won't run even with 32GB RAM)
|
54 |
-
- [
|
55 |
- [ ] Adding other compatible Rockchip's SoCs
|
56 |
|
57 |
## More info
|
|
|
24 |
| Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588 |
|
25 |
| Llama 2 7B | 7B | https://huggingface.co/Pelochus/llama2-chat-7b-hf-rk3588 |
|
26 |
| Llama 2 13B | 13B | https://huggingface.co/Pelochus/llama2-chat-13b-hf-rk3588 |
|
27 |
+
| Qwen 1.5 Chat | 4B | https://huggingface.co/Pelochus/qwen1.5-chat-4B-rk3588 |
|
28 |
| TinyLlama v1 (broken) | 1.1B | https://huggingface.co/Pelochus/tinyllama-v1-rk3588 |
|
29 |
|
30 |
+
However, RKLLM also supports Qwen 2 (supossedly). Llama 2 was converted using Azure servers.
|
31 |
For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
|
32 |
+
Converting Llama 2 7B peaked at about 32 GBs of RAM + 50 GB of swap.
|
33 |
|
34 |
## Downloading a model
|
35 |
Use:
|
|
|
53 |
|
54 |
## Future additions
|
55 |
- [x] Converting Llama 2 (70B currently in conversion, but that won't run even with 32GB RAM)
|
56 |
+
- [x] Converting Qwen 1.5 (from 0.5 to 7B, except 4B, already converted)
|
57 |
- [ ] Adding other compatible Rockchip's SoCs
|
58 |
|
59 |
## More info
|