Nohobby commited on
Commit
7963f64
1 Parent(s): 257b6d3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +132 -11
README.md CHANGED
@@ -1,29 +1,150 @@
1
  ---
2
  base_model:
3
  - unsloth/Mistral-Small-Instruct-2409
 
 
 
 
 
 
 
 
 
 
4
  library_name: transformers
5
  tags:
6
  - mergekit
7
  - merge
8
-
 
 
9
  ---
10
- # Schisandra-22B-v0.2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
 
12
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
 
 
 
 
 
 
 
 
13
 
14
  ## Merge Details
15
- ### Merge Method
16
 
17
- This model was merged using the della_linear merge method using SchisandraVA2 as a base.
 
18
 
19
- ### Models Merged
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
20
 
21
- The following models were included in the merge:
22
- * [unsloth/Mistral-Small-Instruct-2409](https://huggingface.co/unsloth/Mistral-Small-Instruct-2409)
23
 
24
- ### Configuration
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
- The following YAML configuration was used to produce this model:
27
 
28
  ```yaml
29
  dtype: bfloat16
@@ -61,4 +182,4 @@ models:
61
  - filter: down_proj
62
  value: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
63
  - value: 1
64
- ```
 
1
  ---
2
  base_model:
3
  - unsloth/Mistral-Small-Instruct-2409
4
+ - TheDrummer/Cydonia-22B-v1.2
5
+ - Gryphe/Pantheon-RP-Pure-1.6.2-22b-Small
6
+ - anthracite-org/magnum-v4-22b
7
+ - ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1
8
+ - spow12/ChatWaifu_v2.0_22B
9
+ - rAIfle/Acolyte-22B
10
+ - Envoid/Mistral-Small-NovusKyver
11
+ - InferenceIllusionist/SorcererLM-22B
12
+ - allura-org/MS-Meadowlark-22B
13
+ - crestf411/MS-sunfall-v0.7.0
14
  library_name: transformers
15
  tags:
16
  - mergekit
17
  - merge
18
+ license: other
19
+ language:
20
+ - en
21
  ---
22
+ ***
23
+ ## Schisandra
24
+
25
+ Many thanks to the authors of the models used!
26
+
27
+ [RPMax v1.1](https://huggingface.co/ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1) | [Pantheon-RP](https://huggingface.co/Gryphe/Pantheon-RP-Pure-1.6.2-22b-Small) | [UnslopSmall-v1](https://huggingface.co/TheDrummer/UnslopSmall-22B-v1) | [Magnum V4](https://huggingface.co/anthracite-org/magnum-v4-22b) | [ChatWaifu v2.0](https://huggingface.co/spow12/ChatWaifu_v2.0_22B) | [SorcererLM](https://huggingface.co/InferenceIllusionist/SorcererLM-22B) | [Acolyte](https://huggingface.co/rAIfle/Acolyte-22B) | [NovusKyver](https://huggingface.co/Envoid/Mistral-Small-NovusKyver) | (Meadowlark)[https://huggingface.co/allura-org/MS-Meadowlark-22B] | (Sunfall)[https://huggingface.co/crestf411/MS-sunfall-v0.7.0]
28
+ ***
29
+
30
+ ### Overview
31
+
32
+ Main uses: RP, Storywriting
33
+
34
+ An intelligent model that is attentive to details and has a low-slop writing style. This time with a stable tokenizer.
35
+
36
+ Oh, and it now contains 10 finetunes! Not sure if some of them actually contribute to the output, but it's nice to see the numbers growing.
37
+
38
+ ***
39
+
40
+ ### Quants
41
+
42
+ GGUF: [Static](https://huggingface.co/mradermacher/MS-Schisandra-22B-v0.2-GGUF) | [Imatrix](https://huggingface.co/mradermacher/MS-Schisandra-22B-v0.2-i1-GGUF)
43
+
44
+ exl2: [4.65bpw](https://huggingface.co/waldie/MS-Schisandra-22B-v0.2-4.65bpw-h6-exl2) [5.5bpw](https://huggingface.co/waldie/MS-Schisandra-22B-v0.2-5.5bpw-h6-exl2) [6.5bpw](https://huggingface.co/waldie/MS-Schisandra-22B-v0.2-6.5bpw-h6-exl2)
45
 
46
+ ***
47
+
48
+ ### Settings
49
+
50
+ Prompt format: Mistral-V3 or (this)[https://huggingface.co/ToastyPigeon/ST-Presets-Mistral-Small/resolve/main/ST-formatting-Mistral-Small-Oneshot.json]
51
+
52
+ Samplers: [These](https://qu.ax/BCRKS.json) or [These](https://qu.ax/OusTx.json)
53
+
54
+ ***
55
 
56
  ## Merge Details
57
+ ### Merging steps
58
 
59
+ ## Step1
60
+ (Config partially taken from (here)[https://huggingface.co/Casual-Autopsy/L3-Super-Nova-RP-8B])
61
 
62
+ ```yaml
63
+ base_model: spow12/ChatWaifu_v2.0_22B
64
+ parameters:
65
+ int8_mask: true
66
+ rescale: true
67
+ normalize: false
68
+ dtype: bfloat16
69
+ tokenizer_source: base
70
+ merge_method: della
71
+ models:
72
+ - model: Envoid/Mistral-Small-NovusKyver
73
+ parameters:
74
+ density: [0.35, 0.65, 0.5, 0.65, 0.35]
75
+ epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
76
+ lambda: 0.85
77
+ weight: [-0.01891, 0.01554, -0.01325, 0.01791, -0.01458]
78
+ - model: rAIfle/Acolyte-22B
79
+ parameters:
80
+ density: [0.6, 0.4, 0.5, 0.4, 0.6]
81
+ epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
82
+ lambda: 0.85
83
+ weight: [0.01847, -0.01468, 0.01503, -0.01822, 0.01459]
84
+ ```
85
+
86
+ ## Step2
87
+ (Config partially taken from (here)[https://huggingface.co/Casual-Autopsy/L3-Super-Nova-RP-8B])
88
+
89
+ ```yaml
90
+ base_model: InferenceIllusionist/SorcererLM-22B
91
+ parameters:
92
+ int8_mask: true
93
+ rescale: true
94
+ normalize: false
95
+ dtype: bfloat16
96
+ tokenizer_source: base
97
+ merge_method: della
98
+ models:
99
+ - model: crestf411/MS-sunfall-v0.7.0
100
+ parameters:
101
+ density: [0.35, 0.65, 0.5, 0.65, 0.35]
102
+ epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
103
+ lambda: 0.85
104
+ weight: [-0.01891, 0.01554, -0.01325, 0.01791, -0.01458]
105
+ - model: anthracite-org/magnum-v4-22b
106
+ parameters:
107
+ density: [0.6, 0.4, 0.5, 0.4, 0.6]
108
+ epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
109
+ lambda: 0.85
110
+ weight: [0.01847, -0.01468, 0.01503, -0.01822, 0.01459]
111
+ ```
112
 
113
+ ## SchisandraVA2
114
+ (Config taken from (here)[https://huggingface.co/HiroseKoichi/Llama-3-8B-Stroganoff-4.0])
115
 
116
+ ```yaml
117
+ merge_method: della_linear
118
+ dtype: bfloat16
119
+ parameters:
120
+ normalize: true
121
+ int8_mask: true
122
+ tokenizer_source: base
123
+ base_model: TheDrummer/UnslopSmall-22B-v1
124
+ models:
125
+ - model: ArliAI/Mistral-Small-22B-ArliAI-RPMax-v1.1
126
+ parameters:
127
+ density: 0.55
128
+ weight: 1
129
+ - model: Gryphe/Pantheon-RP-Pure-1.6.2-22b-Small
130
+ parameters:
131
+ density: 0.55
132
+ weight: 1
133
+ - model: Step1
134
+ parameters:
135
+ density: 0.55
136
+ weight: 1
137
+ - model: allura-org/MS-Meadowlark-22B
138
+ parameters:
139
+ density: 0.55
140
+ weight: 1
141
+ - model: Step2
142
+ parameters:
143
+ density: 0.55
144
+ weight: 1
145
+ ```
146
 
147
+ ## Schisandra-v0.2
148
 
149
  ```yaml
150
  dtype: bfloat16
 
182
  - filter: down_proj
183
  value: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
184
  - value: 1
185
+ ```