Create README.md
Browse files
README.md
ADDED
@@ -0,0 +1,26 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: mit
|
3 |
+
---
|
4 |
+
Series of models to test the benefits of CoreML joint compression on iOS 18/macOS 15.
|
5 |
+
|
6 |
+
# mlp-*.mlpackage
|
7 |
+
Simple Up/Gate/Silu/Down MLP repeated four times with the Llama 2 7B dimensions.
|
8 |
+
|
9 |
+
All using 'CPU and Neural Engine' compute unit, measured in Xcode.
|
10 |
+
|
11 |
+
|Device|Model |Precision |Minimum (ms)|Median (ms)|
|
12 |
+
|:-- |:-- |:-- |--: |--: |
|
13 |
+
|M1 Max|mlp-float16 |float16 |19.30 |19.42 |
|
14 |
+
|M1 Max|mlp-4bit |4-bit LUT |5.93 |5.98 |
|
15 |
+
|M1 Max|mlp-2bit |2-bit LUT |5.92 |6.11 |
|
16 |
+
|M1 Max|mlp-4bit-int8|4-bit int8 LUT + A8|6.02 |6.31 |
|
17 |
+
|M1 Max|mlp-2bit-int8|2-bit int8 LUT + A8|6.00 |6.18 |
|
18 |
+
|
19 |
+
# Download
|
20 |
+
```
|
21 |
+
huggingface-cli download \
|
22 |
+
--local-dir . \
|
23 |
+
--local-dir-use-symlinks False \
|
24 |
+
smpanaro/coreml-joint-compression-test \
|
25 |
+
--include "*.mlpackage/*"
|
26 |
+
```
|