Convert Qwen1.5-4B-Chat to RKLLM format for Rockchip devices

README.md

@@ -0,0 +1,63 @@
+---
+language:
+- zh
+- en
+pipeline_tag: text-generation
+tags:
+- qwen 
+- Qwen1.5-4B-Chat
+- rkllm
+- rk3588
+---
+
+# Qwen1.5-4B-Chat-rkllm 
+This is a conversion from [Qwen/Qwen1.5-4B-Chat](https://huggingface.co/Qwen/Qwen1.5-4B-Chat)
+ to the RKLLM format for chat in Rockchip devices. 
+
+
+- [Qwen1.5-4B-Chat-rkllm](#qwen15-4b-chat-rkllm)
+  - [Support Devices](#support-devices)
+  - [Convert tools](#convert-tools)
+  - [Converted with RKLLM runtime](#converted-with-rkllm-runtime)
+  - [License](#license)
+  - [Trouble shot](#trouble-shot)
+  - [Reference](#reference)
+
+
+## Support Devices
+* RK3588/RK3588s
+
+## Convert tools
+To Converting LLMs for Rockchip's NPUs, please see the artical<sup>1,2</sup> for model details.
+
+## Converted with RKLLM runtime
+*  RKLLM runtime `1.0.1`
+
+## License
+Same as the original [Qwen/Qwen1.5-4B-Chat](https://huggingface.co/Qwen/Qwen1.5-4B-Chat)
+
+
+## Trouble shot
+* `E RKNN: [10:48:59.683] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory` 
+    ```shell
+    firefly@firefly:~/Documents/rknn-llm$ rkllm ./chatglm3-6b.rkllm
+    rkllm init start
+    rkllm-runtime version: 1.0.1, rknpu driver version: 0.8.2, platform: RK3588
+    Warning: Your rknpu driver version is too low, please upgrade to 0.9.6.
+    E RKNN: [10:48:59.683] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory
+
+    can not create weight memory for domain1
+    E RKNN: [10:49:00.480] failed to allocate handle, ret: -1, errno: 12, errstr: Cannot allocate memory
+
+    can not create weight memory for domain2
+    E RKNN: [10:49:05.216] failed to convert handle(1020) to fd, ret: -1, errno: 24, errstr: Too many open files
+
+    # Solution
+    firefly@firefly:~/Documents/rknn-llm$ ulimit -n 102400
+    ```
+
+## Reference
+1. [airockchip/rknn-llm](https://github.com/airockchip/rknn-llm)
+1. [Pelochus/ezrknn-llm](https://github.com/Pelochus/ezrknn-llm)
+2. [Qwen/Qwen1.5-4B-Chat](https://huggingface.co/Qwen/Qwen1.5-4B-Chat)
+3. [跑大模型遇到问题 #62](https://github.com/airockchip/rknn-llm/issues/62)