Add fast tokenizer
#9
by
Narsil
HF staff
- opened
- NOTICE +0 -24
- README.md +10 -11
- tokenizer.json +0 -0
NOTICE
DELETED
@@ -1,24 +0,0 @@
|
|
1 |
-
Copyright 2023-2024 01.AI
|
2 |
-
|
3 |
-
This work, [Your Model Name], is based on the work originally authored by 01.AI.
|
4 |
-
|
5 |
-
Licensed under the Apache License, Version 2.0 (the "License");
|
6 |
-
you may not use this file except in compliance with the License.
|
7 |
-
You may obtain a copy of the License at
|
8 |
-
|
9 |
-
http://www.apache.org/licenses/LICENSE-2.0
|
10 |
-
|
11 |
-
Unless required by applicable law or agreed to in writing, software
|
12 |
-
distributed under the License is distributed on an "AS IS" BASIS,
|
13 |
-
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
14 |
-
See the License for the specific language governing permissions and
|
15 |
-
limitations under the License.
|
16 |
-
|
17 |
-
Attribution
|
18 |
-
|
19 |
-
If you create derivative works based on this model, please include the following attribution in your derivative works:
|
20 |
-
|
21 |
-
This work is a derivative of [The Yi-1.5 Series Model You Base On] by 01.AI, used under the Apache 2.0 License.
|
22 |
-
|
23 |
-
|
24 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
@@ -16,7 +16,6 @@ license: apache-2.0
|
|
16 |
<a href="https://github.com/01-ai/Yi-1.5/issues/2">π¬ WeChat</a>
|
17 |
<br/>
|
18 |
<a href="https://arxiv.org/abs/2403.04652">π Paper</a> β’
|
19 |
-
<a href="https://01-ai.github.io/">πͺ Tech Blog</a> β’
|
20 |
<a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">π FAQ</a> β’
|
21 |
<a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">π Learning Hub</a>
|
22 |
</p>
|
@@ -43,11 +42,11 @@ Model | Context Length | Pre-trained Tokens
|
|
43 |
|
44 |
| Name | Download |
|
45 |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
46 |
-
| Yi-1.5-34B-Chat | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
47 |
-
| Yi-1.5-34B-Chat-16K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
48 |
-
| Yi-1.5-9B-Chat | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
49 |
-
| Yi-1.5-9B-Chat-16K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
50 |
-
| Yi-1.5-6B-Chat | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
51 |
|
52 |
</div>
|
53 |
|
@@ -57,11 +56,11 @@ Model | Context Length | Pre-trained Tokens
|
|
57 |
|
58 |
| Name | Download |
|
59 |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
60 |
-
| Yi-1.5-34B | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
61 |
-
| Yi-1.5-34B-32K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
62 |
-
| Yi-1.5-9B | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
63 |
-
| Yi-1.5-9B-32K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
64 |
-
| Yi-1.5-6B | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai)
|
65 |
|
66 |
</div>
|
67 |
|
|
|
16 |
<a href="https://github.com/01-ai/Yi-1.5/issues/2">π¬ WeChat</a>
|
17 |
<br/>
|
18 |
<a href="https://arxiv.org/abs/2403.04652">π Paper</a> β’
|
|
|
19 |
<a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#faq">π FAQ</a> β’
|
20 |
<a href="https://github.com/01-ai/Yi/tree/main?tab=readme-ov-file#learning-hub">π Learning Hub</a>
|
21 |
</p>
|
|
|
42 |
|
43 |
| Name | Download |
|
44 |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
45 |
+
| Yi-1.5-34B-Chat | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
46 |
+
| Yi-1.5-34B-Chat-16K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
47 |
+
| Yi-1.5-9B-Chat | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
48 |
+
| Yi-1.5-9B-Chat-16K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
49 |
+
| Yi-1.5-6B-Chat | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
50 |
|
51 |
</div>
|
52 |
|
|
|
56 |
|
57 |
| Name | Download |
|
58 |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
59 |
+
| Yi-1.5-34B | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
60 |
+
| Yi-1.5-34B-32K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
61 |
+
| Yi-1.5-9B | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
62 |
+
| Yi-1.5-9B-32K | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
63 |
+
| Yi-1.5-6B | β’ [π€ Hugging Face](https://huggingface.co/collections/01-ai/yi-15-2024-05-663f3ecab5f815a3eaca7ca8) β’ [π€ ModelScope](https://www.modelscope.cn/organization/01ai) |
|
64 |
|
65 |
</div>
|
66 |
|
tokenizer.json
DELETED
The diff for this file is too large to render.
See raw diff
|
|