Pelochus commited on
Commit
7fea569
1 Parent(s): f14d644

Added how to install

Browse files
Files changed (1) hide show
  1. README.md +17 -2
README.md CHANGED
@@ -16,13 +16,28 @@ This repo contains the converted models for running on the RK3588 NPU found in S
16
  Right now, only converted the following models:
17
  | LLM | Parameters | Link |
18
  | --------------------- | ----------- | ---------------------------------------------------------- |
19
- | Qwen Chat | 1.8B | https://huggingface.co/Pelochus/qwen-1_8B-rk3588/tree/main |
20
- | Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588/tree/main |
21
  | TinyLlama v1 | 1.1B | https://huggingface.co/Pelochus/tinyllama-v1-rk3588 |
22
 
23
  However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
24
  For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
25
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
26
  ## Future additions
27
  - [ ] Converting Qwen 2 and Llama 2
28
  - [ ] Adding other compatible Rockchip's SoCs
 
16
  Right now, only converted the following models:
17
  | LLM | Parameters | Link |
18
  | --------------------- | ----------- | ---------------------------------------------------------- |
19
+ | Qwen Chat | 1.8B | https://huggingface.co/Pelochus/qwen-1_8B-rk3588 |
20
+ | Microsoft Phi-2 | 2.7B | https://huggingface.co/Pelochus/phi-2-rk3588 |
21
  | TinyLlama v1 | 1.1B | https://huggingface.co/Pelochus/tinyllama-v1-rk3588 |
22
 
23
  However, RKLLM also supports Qwen 2 and Llama 2 7B, but I can't convert them due to my PC only having 16 GBs of RAM.
24
  For reference, converting Phi-2 peaked at about 15 GBs of RAM + 25 GBs of swap (counting OS, but that was using about 2 GBs max).
25
 
26
+ ## Downloading a model
27
+ Use:
28
+
29
+ `git clone LINK_FROM_PREVIOUS_TABLE_HERE`
30
+
31
+ And then (may not be necessary):
32
+
33
+ `git lfs pull`
34
+
35
+ If the first clone gives you problems (takes too long) you can also:
36
+
37
+ `GIT_LFS_SKIP_SMUDGE=1 git clone LINK_FROM_PREVIOUS_TABLE_HERE`
38
+
39
+ Then your model will be inside the downloaded folder.
40
+
41
  ## Future additions
42
  - [ ] Converting Qwen 2 and Llama 2
43
  - [ ] Adding other compatible Rockchip's SoCs