Added how to install
Browse files
README.md
CHANGED
@@ -16,13 +16,28 @@ This repo contains the converted models for running on the RK3588 NPU found in S
|
|
16 |
Right now, only converted the following models:
|
17 |
| LLM | Parameters | Link |
|
18 |
| --------------------- | ----------- | ---------------------------------------------------------- |
|
19 |
-
| Qwen Chat | 1.8B | https://huggingface.co/Pelochus/qwen-1_8B-rk3588
|
20 |
-
| Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588
|
21 |
| TinyLlama v1 | 1.1B | https://huggingface.co/Pelochus/tinyllama-v1-rk3588 |
|
22 |
|
23 |
However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
|
24 |
For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
|
25 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
26 |
## Future additions
|
27 |
- [ ] Converting Qwen 2 and Llama 2
|
28 |
- [ ] Adding other compatible Rockchip's SoCs
|
|
|
16 |
Right now, only converted the following models:
|
17 |
| LLM | Parameters | Link |
|
18 |
| --------------------- | ----------- | ---------------------------------------------------------- |
|
19 |
+
| Qwen Chat | 1.8B | https://huggingface.co/Pelochus/qwen-1_8B-rk3588 |
|
20 |
+
| Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588 |
|
21 |
| TinyLlama v1 | 1.1B | https://huggingface.co/Pelochus/tinyllama-v1-rk3588 |
|
22 |
|
23 |
However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
|
24 |
For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
|
25 |
|
26 |
+
## Downloading a model
|
27 |
+
Use:
|
28 |
+
|
29 |
+
`git clone LINK_FROM_PREVIOUS_TABLE_HERE`
|
30 |
+
|
31 |
+
And then (may not be necessary):
|
32 |
+
|
33 |
+
`git lfs pull`
|
34 |
+
|
35 |
+
If the first clone gives you problems (takes too long) you can also:
|
36 |
+
|
37 |
+
`GIT_LFS_SKIP_SMUDGE=1 git clone LINK_FROM_PREVIOUS_TABLE_HERE`
|
38 |
+
|
39 |
+
Then your model will be inside the downloaded folder.
|
40 |
+
|
41 |
## Future additions
|
42 |
- [ ] Converting Qwen 2 and Llama 2
|
43 |
- [ ] Adding other compatible Rockchip's SoCs
|