Hugging Face's logo

jxke
/
Qwen1.5-4B-chat-rkllm

Text Generation
Chinese
English
qwen
Qwen1.5-4B-Chat
rkllm
rk3588
Model card Files Community
Qwen1.5-4B-chat-rkllm
File size: 2,048 Bytes 
fef7ac9
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
---
language:
- zh
- en
pipeline_tag: text-generation
tags:
- qwen 
- Qwen1.5-4B-Chat
- rkllm
- rk3588
---

# Qwen1.5-4B-Chat-rkllm 
This is a conversion from [Qwen/Qwen1.5-4B-Chat](https://huggingface.co./Qwen/Qwen1.5-4B-Chat)
 to the RKLLM format for chat in Rockchip devices. 


- [Qwen1.5-4B-Chat-rkllm](#qwen15-4b-chat-rkllm)
  - [Support Devices](#support-devices)
  - [Convert tools](#convert-tools)
  - [Converted with RKLLM runtime](#converted-with-rkllm-runtime)
  - [License](#license)
  - [Trouble shot](#trouble-shot)
  - [Reference](#reference)


## Support Devices
* RK3588/RK3588s

## Convert tools
To Converting LLMs for Rockchip's NPUs, please see the artical<sup>1,2</sup> for model details.

## Converted with RKLLM runtime
*  RKLLM runtime `1.0.1`

## License
Same as the original [Qwen/Qwen1.5-4B-Chat](https://huggingface.co./Qwen/Qwen1.5-4B-Chat)


## Trouble shot
* `E RKNN: [10:48:59.683] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory` 
    ```shell
    firefly@firefly:~/Documents/rknn-llm$ rkllm ./chatglm3-6b.rkllm
    rkllm init start
    rkllm-runtime version: 1.0.1, rknpu driver version: 0.8.2, platform: RK3588
    Warning: Your rknpu driver version is too low, please upgrade to 0.9.6.
    E RKNN: [10:48:59.683] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory

    can not create weight memory for domain1
    E RKNN: [10:49:00.480] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory

    can not create weight memory for domain2
    E RKNN: [10:49:05.216] failed to convert handle(1020) to fd, ret: -1, errno: 24, errstr: Too many open files

    # Solution
    firefly@firefly:~/Documents/rknn-llm$ ulimit -n 102400
    ```

## Reference
1. [airockchip/rknn-llm](https://github.com/airockchip/rknn-llm)
1. [Pelochus/ezrknn-llm](https://github.com/Pelochus/ezrknn-llm)
2. [Qwen/Qwen1.5-4B-Chat](https://huggingface.co./Qwen/Qwen1.5-4B-Chat)
3. [跑大模型遇到问题 #62](https://github.com/airockchip/rknn-llm/issues/62)