Hugging Face's logo

redrix
/
GodSlayer-12B-ABYSS

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
12b
chat
roleplay
creative-writing
DELLA-linear
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
redrix commited on
Commit
b602ebc
·
verified ·
1 Parent(s): 1a60c42

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +172 -0
README.md ADDED
@@ -0,0 +1,172 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model:
3
+ - LatitudeGames/Wayfarer-12B
4
+ - ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
5
+ - PocketDoc/Dans-PersonalityEngine-V1.1.0-12b
6
+ - HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407
7
+ - TheDrummer/UnslopNemo-12B-v4
8
+ - romaingrx/red-teamer-mistral-nemo
9
+ - DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS
10
+ - rAIfle/Questionable-MN-bf16
11
+ - allura-org/MN-12b-RP-Ink
12
+ - IntervitensInc/Mistral-Nemo-Base-2407-chatml
13
+ library_name: transformers
14
+ tags:
15
+ - mergekit
16
+ - merge
17
+ - 12b
18
+ - chat
19
+ - roleplay
20
+ - creative-writing
21
+ - DELLA-linear
22
+
23
+ ---
24
+ # GodSlayer-12B-ABYSS
25
+
26
+ This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
27
+
28
+ The goal of this model is to remain fairly stable and coherent, while counteracting positivity-bias and improving realism and diverse responses.
29
+
30
+ Model #12
31
+ ## Merge Details
32
+ ### Merge Method
33
+
34
+ This model was merged using the NuSLERP merge method using [IntervitensInc/Mistral-Nemo-Base-2407-chatml](https://huggingface.co/IntervitensInc/Mistral-Nemo-Base-2407-chatml) as a base.
35
+
36
+ ### Models Merged
37
+
38
+ The following models were included in the merge:
39
+ * [LatitudeGames/Wayfarer-12B](https://huggingface.co/LatitudeGames/Wayfarer-12B)
40
+ * [ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2](https://huggingface.co/ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2)
41
+ * [PocketDoc/Dans-PersonalityEngine-V1.1.0-12b](https://huggingface.co/PocketDoc/Dans-PersonalityEngine-V1.1.0-12b)
42
+ * [HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407](https://huggingface.co/HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407)
43
+ * [romaingrx/red-teamer-mistral-nemo](https://huggingface.co/romaingrx/red-teamer-mistral-nemo)
44
+ * [DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS](https://huggingface.co/DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS)
45
+ * [rAIfle/Questionable-MN-bf16](https://huggingface.co/rAIfle/Questionable-MN-bf16)
46
+ * [allura-org/MN-12b-RP-Ink](https://huggingface.co/allura-org/MN-12b-RP-Ink)
47
+
48
+ ### Configuration
49
+
50
+ The following YAML configurations were used to produce this model:
51
+
52
+ ```yaml
53
+ # P1:
54
+ models:
55
+ - model: PocketDoc/Dans-PersonalityEngine-V1.1.0-12b
56
+ parameters:
57
+ weight:
58
+ - filter: self_attn
59
+ value: 0.2
60
+ - filter: mlp
61
+ value: 0.2
62
+ - value: 0.2
63
+ density: 0.6
64
+ - model: ArliAI/Mistral-Nemo-12B-ArliAI-RPMax-v1.2
65
+ parameters:
66
+ weight:
67
+ - filter: self_attn
68
+ value: 0.15
69
+ - filter: mlp
70
+ value: 0.15
71
+ - value: 0.15
72
+ density: 0.55
73
+ - model: HumanLLMs/Human-Like-Mistral-Nemo-Instruct-2407
74
+ parameters:
75
+ weight:
76
+ - filter: self_attn
77
+ value: 0.1
78
+ - filter: mlp
79
+ value: 0.1
80
+ - value: 0.1
81
+ density: 0.5
82
+ - model: LatitudeGames/Wayfarer-12B
83
+ parameters:
84
+ weight:
85
+ - filter: self_attn
86
+ value: 0.25
87
+ - filter: mlp
88
+ value: 0.25
89
+ - value: 0.25
90
+ density: 0.65
91
+ base_model: TheDrummer/UnslopNemo-12B-v4
92
+ merge_method: della_linear
93
+ dtype: bfloat16
94
+ chat_template: "chatml"
95
+ tokenizer_source: union
96
+ parameters:
97
+ normalize: true
98
+ int8_mask: true
99
+ epsilon: 0.1
100
+ lambda: 1
101
+ ```
102
+
103
+ ```yaml
104
+ # P2:
105
+ models:
106
+ - model: rAIfle/Questionable-MN-bf16
107
+ parameters:
108
+ weight:
109
+ - filter: self_attn
110
+ value: 0.2
111
+ - filter: mlp
112
+ value: 0.2
113
+ - value: 0.2
114
+ density: 0.6
115
+ - model: DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS
116
+ parameters:
117
+ weight:
118
+ - filter: self_attn
119
+ value: 0.3
120
+ - filter: mlp
121
+ value: 0.3
122
+ - value: 0.3
123
+ density: 0.7
124
+ - model: allura-org/MN-12b-RP-Ink
125
+ parameters:
126
+ weight:
127
+ - filter: self_attn
128
+ value: 0.35
129
+ - filter: mlp
130
+ value: 0.35
131
+ - value: 0.35
132
+ density: 0.75
133
+ - model: romaingrx/red-teamer-mistral-nemo
134
+ parameters:
135
+ weight:
136
+ - filter: self_attn
137
+ value: 0.25
138
+ - filter: mlp
139
+ value: 0.25
140
+ - value: 0.25
141
+ density: 0.65
142
+ base_model: TheDrummer/UnslopNemo-12B-v4
143
+ merge_method: della_linear
144
+ dtype: bfloat16
145
+ chat_template: "chatml"
146
+ tokenizer_source: union
147
+ parameters:
148
+ normalize: true
149
+ int8_mask: true
150
+ epsilon: 0.1
151
+ lambda: 1
152
+ ```
153
+
154
+ ```yaml
155
+ # Final:
156
+ models:
157
+ - model: P1
158
+ parameters:
159
+ weight: 0.5
160
+ - model: P2
161
+ parameters:
162
+ weight: 0.5
163
+ base_model: IntervitensInc/Mistral-Nemo-Base-2407-chatml
164
+ merge_method: nuslerp
165
+ dtype: bfloat16
166
+ chat_template: "chatml"
167
+ tokenizer:
168
+ source: union
169
+ parameters:
170
+ normalize: true
171
+ int8_mask: true
172
+ ```