Update README.md
Browse files
README.md
CHANGED
@@ -29,6 +29,33 @@ This is a test model on a the following
|
|
29 |
- slight customization on llama3 template (no new tokens | no new configs)
|
30 |
- Works with Ollama create with just "FROM path/to/model" as Modelfile (llama3 template works no issues)
|
31 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
32 |
# NOTE: DISCLAIMER
|
33 |
|
34 |
Please note this is not for the purpose of production, but result of Fine Tuning through self learning
|
|
|
29 |
- slight customization on llama3 template (no new tokens | no new configs)
|
30 |
- Works with Ollama create with just "FROM path/to/model" as Modelfile (llama3 template works no issues)
|
31 |
|
32 |
+
# HOW TO USE
|
33 |
+
|
34 |
+
The whole point of conversion for me was I wanted to be able to to use it through Ollama or (other local options)
|
35 |
+
For Ollama, it required to be a GGUF file and one you have this it is pretty straight forward (if it is in llama3 which this model is)
|
36 |
+
|
37 |
+
Quick Start:
|
38 |
+
- You must already have Ollama running in your setting
|
39 |
+
- Download the unsloth.Q4_K_M.gguf model from Files
|
40 |
+
- In the same directory create a file call "Modelfile"
|
41 |
+
- Inside the "Modelfile" type
|
42 |
+
|
43 |
+
```
|
44 |
+
FROM ./unsloth.Q4_K_M.gguf
|
45 |
+
|
46 |
+
```
|
47 |
+
- Save a go back to the folder (folder where model + Modelfile exisit)
|
48 |
+
- Now in terminal make sure you are in the same location of the folder and type in the following command
|
49 |
+
|
50 |
+
```
|
51 |
+
ollama create mycustomai # "mycustomai" <- you can name it anything u want
|
52 |
+
```
|
53 |
+
|
54 |
+
This GGUF is based on llama3-3-8B-Instruct thus ollama doesn't need anything else to auto configure this model
|
55 |
+
|
56 |
+
After than you should be able to use this model to chat!
|
57 |
+
|
58 |
+
|
59 |
# NOTE: DISCLAIMER
|
60 |
|
61 |
Please note this is not for the purpose of production, but result of Fine Tuning through self learning
|