lokinfey commited on
Commit
731e725
·
verified ·
1 Parent(s): d766327

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +32 -3
README.md CHANGED
@@ -1,3 +1,32 @@
1
- ---
2
- license: mit
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ ---
4
+
5
+ # **Phi-3.5-mini-instruct-onnx-gpu Unofficial version**
6
+
7
+ This is a Phi-3.5-mini-instruct version of ONNX GPU, based on ONNX Runtime for GenAI [https://github.com/microsoft/onnxruntime-genai](https://github.com/microsoft/onnxruntime-genai). Convert with the following command
8
+
9
+ ## **1. Install the SDK**
10
+
11
+
12
+ ```
13
+
14
+ pip install torch transformers onnx onnxruntime
15
+
16
+
17
+ pip install --pre onnxruntime-genai
18
+
19
+ ```
20
+
21
+ ## **2. Convert GPU ONNX Support**
22
+
23
+
24
+ ```bash
25
+
26
+ python3 -m onnxruntime_genai.models.builder -m microsoft/Phi-3.5-mini-instruct -o ./onnx-gpu -p int4 -e cuda -c ./Phi-3.5-mini-instruct
27
+
28
+ ```
29
+
30
+
31
+
32
+ This is a conversion, but no specific optimization has been done. Please look forward to the official version.