|
--- |
|
language: |
|
- zh |
|
- en |
|
pipeline_tag: text-generation |
|
tags: |
|
- qwen |
|
- Qwen1.5-4B-Chat |
|
- rkllm |
|
- rk3588 |
|
--- |
|
|
|
# Qwen1.5-4B-Chat-rkllm |
|
This is a conversion from [Qwen/Qwen1.5-4B-Chat](https://huggingface.co./Qwen/Qwen1.5-4B-Chat) |
|
to the RKLLM format for chat in Rockchip devices. |
|
|
|
|
|
- [Qwen1.5-4B-Chat-rkllm](#qwen15-4b-chat-rkllm) |
|
- [Support Devices](#support-devices) |
|
- [Convert tools](#convert-tools) |
|
- [Converted with RKLLM runtime](#converted-with-rkllm-runtime) |
|
- [License](#license) |
|
- [Trouble shot](#trouble-shot) |
|
- [Reference](#reference) |
|
|
|
|
|
## Support Devices |
|
* RK3588/RK3588s |
|
|
|
## Convert tools |
|
To Converting LLMs for Rockchip's NPUs, please see the artical<sup>1,2</sup> for model details. |
|
|
|
## Converted with RKLLM runtime |
|
* RKLLM runtime `1.0.1` |
|
|
|
## License |
|
Same as the original [Qwen/Qwen1.5-4B-Chat](https://huggingface.co./Qwen/Qwen1.5-4B-Chat) |
|
|
|
|
|
## Trouble shot |
|
* `E RKNN: [10:48:59.683] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory` |
|
```shell |
|
firefly@firefly:~/Documents/rknn-llm$ rkllm ./chatglm3-6b.rkllm |
|
rkllm init start |
|
rkllm-runtime version: 1.0.1, rknpu driver version: 0.8.2, platform: RK3588 |
|
Warning: Your rknpu driver version is too low, please upgrade to 0.9.6. |
|
E RKNN: [10:48:59.683] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory |
|
|
|
can not create weight memory for domain1 |
|
E RKNN: [10:49:00.480] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory |
|
|
|
can not create weight memory for domain2 |
|
E RKNN: [10:49:05.216] failed to convert handle(1020) to fd, ret: -1, errno: 24, errstr: Too many open files |
|
|
|
# Solution |
|
firefly@firefly:~/Documents/rknn-llm$ ulimit -n 102400 |
|
``` |
|
|
|
## Reference |
|
1. [airockchip/rknn-llm](https://github.com/airockchip/rknn-llm) |
|
1. [Pelochus/ezrknn-llm](https://github.com/Pelochus/ezrknn-llm) |
|
2. [Qwen/Qwen1.5-4B-Chat](https://huggingface.co./Qwen/Qwen1.5-4B-Chat) |
|
3. [跑大模型遇到问题 #62](https://github.com/airockchip/rknn-llm/issues/62) |
|
|
