DavidAU commited on
Commit
b700d0c
1 Parent(s): e5f0d33

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -56
README.md DELETED
@@ -1,56 +0,0 @@
1
- ---
2
- base_model: []
3
- library_name: transformers
4
- tags:
5
- - mergekit
6
- - merge
7
-
8
- ---
9
- # part2
10
-
11
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
12
-
13
- ## Merge Details
14
- ### Merge Method
15
-
16
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using E:\Nemo-Grand-Story-3-16-dense3-bf16\part1 as a base.
17
-
18
- ### Models Merged
19
-
20
- The following models were included in the merge:
21
- * G:/11B/magnum-v2.5-12b-kto
22
- * G:/11B/Rocinante-12B-v1.1
23
- * G:/11B/MN-12B-Celeste-V1.9
24
-
25
- ### Configuration
26
-
27
- The following YAML configuration was used to produce this model:
28
-
29
- ```yaml
30
- base_model: E:\Nemo-Grand-Story-3-16-dense3-bf16\part1
31
- dtype: bfloat16
32
- merge_method: dare_ties
33
- slices:
34
- - sources:
35
- - layer_range: [0, 40]
36
- model: G:/11B/MN-12B-Celeste-V1.9
37
- parameters:
38
- weight: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
39
- 0.0, 0.0, 0.0, 0.0825, 0.0825, 0.0825, 0.0825, 0.5, 0.5, 0.0825, 0.0825, 0.0825,
40
- 0.0825, 0.0825, 0.0825, 0.5, 0.3, 0.2, 0.1, 0.05, 0.03, 0.0, 0.0, 0.0, 0.0,
41
- 0.0]
42
- - layer_range: [0, 40]
43
- model: G:/11B/magnum-v2.5-12b-kto
44
- parameters:
45
- weight: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
46
- 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.2, 0.3, 0.4, 0.5,
47
- 0.0825, 0.6, 0.5, 0.5, 0.45, 0.4, 0.35, 0.3, 0.2, 0.1, 0.0]
48
- - layer_range: [0, 40]
49
- model: G:/11B/Rocinante-12B-v1.1
50
- parameters:
51
- weight: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
52
- 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
53
- 0.0, 0.1, 0.3, 0.4, 0.5, 0.57, 0.65, 0.7, 0.8, 0.9, 1.0]
54
- - layer_range: [0, 40]
55
- model: E:\Nemo-Grand-Story-3-16-dense3-bf16\part1
56
- ```