Update README.md
Browse files
README.md
CHANGED
@@ -13,6 +13,35 @@ The Model [fastx-ai/Marco-o1-int-4](https://huggingface.co/fastx-ai/Marco-o1-int
|
|
13 |
converted to MLX format from [AIDC-AI/Marco-o1](https://huggingface.co/AIDC-AI/Marco-o1)
|
14 |
using mlx-lm version **0.20.1**.
|
15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
## Use with mlx
|
17 |
|
18 |
```bash
|
@@ -22,7 +51,7 @@ pip install mlx-lm
|
|
22 |
```python
|
23 |
from mlx_lm import load, generate
|
24 |
|
25 |
-
model, tokenizer = load("fastx-ai/Marco-o1-
|
26 |
|
27 |
prompt="hello"
|
28 |
|
|
|
13 |
converted to MLX format from [AIDC-AI/Marco-o1](https://huggingface.co/AIDC-AI/Marco-o1)
|
14 |
using mlx-lm version **0.20.1**.
|
15 |
|
16 |
+
```python
|
17 |
+
prompt="hello, can you teach me why 2 + 4 = 6 ?"
|
18 |
+
```
|
19 |
+
|
20 |
+
```shell
|
21 |
+
==========
|
22 |
+
Prompt: <|im_start|>system
|
23 |
+
|
24 |
+
你是一个经过良好训练的AI助手,你的名字是Marco-o1.
|
25 |
+
|
26 |
+
## 重要!!!!!
|
27 |
+
当你回答问题时,你的思考应该在<Thought>内完成,<Output>内输出你的结果。
|
28 |
+
<Thought>应该尽可能是英文,但是有2个特例,一个是对原文中的引用,另一个是是数学应该使用markdown格式,<Output>内的输出需要遵循用户输入的语言。
|
29 |
+
<|im_end|>
|
30 |
+
<|im_start|>user
|
31 |
+
hello, can you teach me why 2 + 4 = 6 ?<|im_end|>
|
32 |
+
<|im_start|>assistant
|
33 |
+
|
34 |
+
<Thought>
|
35 |
+
Alright, I need to explain why 2 plus 4 equals 6. Let's start by recalling the basic principles of addition. Addition is the process of combining two or more numbers to find their total. So, when we add 2 and 4, we're essentially combining two quantities.
|
36 |
+
|
37 |
+
First, let's visualize this. Imagine you have 2 apples and someone gives you 4 more apples. Now, how many apples do you have in total? Counting them out
|
38 |
+
==========
|
39 |
+
Prompt: 118 tokens, 698.640 tokens-per-sec
|
40 |
+
Generation: 100 tokens, 103.937 tokens-per-sec
|
41 |
+
Peak memory: 4.386 GB
|
42 |
+
|
43 |
+
```
|
44 |
+
|
45 |
## Use with mlx
|
46 |
|
47 |
```bash
|
|
|
51 |
```python
|
52 |
from mlx_lm import load, generate
|
53 |
|
54 |
+
model, tokenizer = load("fastx-ai/Marco-o1-1.2B-mlx-int4")
|
55 |
|
56 |
prompt="hello"
|
57 |
|