Epiculous commited on
Commit
23f5f95
1 Parent(s): ecc1781

Upload ./README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +79 -0
README.md ADDED
@@ -0,0 +1,79 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ datasets:
4
+ - Epiculous/SynthRP-Gens-v1.1-Filtered-n-Cleaned
5
+ - anthracite-org/stheno-filtered-v1.1
6
+ - PJMixers/hieunguyenminh_roleplay-deduped-ShareGPT
7
+ - Gryphe/Sonnet3.5-Charcard-Roleplay
8
+ - Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
9
+ - anthracite-org/kalo-opus-instruct-22k-no-refusal
10
+ - anthracite-org/nopm_claude_writing_fixed
11
+ - anthracite-org/kalo_opus_misc_240827
12
+ language:
13
+ - en
14
+ - fr
15
+ - de
16
+ - es
17
+ - it
18
+ - pt
19
+ - ru
20
+ - zh
21
+ - ja
22
+ pipeline_tag: text-generation
23
+ tags:
24
+ - merge
25
+ ---
26
+ ### exl2 quant (measurement.json in main branch)
27
+ ---
28
+ ### check revisions for quants
29
+ ---
30
+
31
+
32
+ ![image/png](https://cdn-uploads.huggingface.co/production/uploads/64adfd277b5ff762771e4571/P962FQhRG4I8nbU_DJolY.png)
33
+
34
+ Now for something a bit different, Violet_Twilight-v0.2! This model is a SLERP merge of Azure_Dusk-v0.2 and Crimson_Dawn-v0.2!
35
+
36
+ # Quants!
37
+ <strong>full</strong> / [exl2]() / [gguf]()
38
+
39
+ ## Prompting
40
+ The v0.2 models are trained on ChatML, the prompting structure goes a little something like this:
41
+
42
+ ```
43
+ <|im_start|>user
44
+ Hi there!<|im_end|>
45
+ <|im_start|>assistant
46
+ Nice to meet you!<|im_end|>
47
+ <|im_start|>user
48
+ Can I ask a question?<|im_end|>
49
+ <|im_start|>assistant
50
+ ```
51
+
52
+ ### Context and Instruct
53
+ The v0.2 models are trained on ChatML, please use that Context and Instruct template.
54
+
55
+ ### Current Top Sampler Settings
56
+ [Spicy_Temp](https://files.catbox.moe/9npj0z.json) <br/>
57
+ [Violet_Twilight-Nitral-Special](https://files.catbox.moe/ot54u3.json) <br/>
58
+
59
+ ## Merging
60
+ The following config was used to merge Azure Dusk and Crimson Dawn
61
+ ```yaml
62
+ slices:
63
+ - sources:
64
+ - model: Epiculous/Azure_Dusk-v0.2
65
+ layer_range: [0, 40]
66
+ - model: Epiculous/Crimson_Dawn-V0.2
67
+ layer_range: [0, 40]
68
+ merge_method: slerp
69
+ base_model: Epiculous/Azure_Dusk-v0.2
70
+ parameters:
71
+ t:
72
+ - filter: self_attn
73
+ value: [0, 0.5, 0.3, 0.7, 1]
74
+ - filter: mlp
75
+ value: [1, 0.5, 0.7, 0.3, 0]
76
+ - value: 0.5 # fallback for rest of tensors
77
+ dtype: bfloat16
78
+
79
+ ```