File size: 2,486 Bytes
6f86232 22740ea 6f86232 f3e28c7 6f86232 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
base_model:
- Undi95/Sushi-v1.4
- ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4
- Sao10K/L3.1-70B-Hanami-x1
- meta-llama/Llama-3.3-70B-Instruct
- NeverSleep/Lumimaid-v0.2-70B
- SicariusSicariiStuff/Negative_LLAMA_70B
library_name: transformers
tags:
- mergekit
- merge
license: llama3.3
---
Part of a multi merge experiment. This one came out very interesting.
The idea behind it is to create 3 individual models:
- Pathos: For ERP and uncensored NSFW content
- Ethos: For prose and storytelling
- Logos: For intelligence and awareness
The three models above will then be combined into:
- Kairos: The best of all three hopefully.
I will be using differnet merge methods for these merges in an attempt to find the best combinations hence the Alpha, Beta and Delta tags you will see on each which represent different merge methods.
# merge
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [Linear DELLA](https://arxiv.org/abs/2406.11617) merge method using [meta-llama/Llama-3.3-70B-Instruct](https://huggingface.co./meta-llama/Llama-3.3-70B-Instruct) as a base.
### Models Merged
The following models were included in the merge:
* [Undi95/Sushi-v1.4](https://huggingface.co./Undi95/Sushi-v1.4)
* [ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4](https://huggingface.co./ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4)
* [Sao10K/L3.1-70B-Hanami-x1](https://huggingface.co./Sao10K/L3.1-70B-Hanami-x1)
* [NeverSleep/Lumimaid-v0.2-70B](https://huggingface.co./NeverSleep/Lumimaid-v0.2-70B)
* [SicariusSicariiStuff/Negative_LLAMA_70B](https://huggingface.co./SicariusSicariiStuff/Negative_LLAMA_70B)
### Configuration
The following YAML configuration was used to produce this model:
```yaml
models:
- model: Undi95/Sushi-v1.4
parameters:
weight: 0.20
density: 0.7
- model: NeverSleep/Lumimaid-v0.2-70B
parameters:
weight: 0.20
density: 0.7
- model: ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4
parameters:
weight: 0.20
density: 0.7
- model: Sao10K/L3.1-70B-Hanami-x1
parameters:
weight: 0.20
density: 0.7
- model: SicariusSicariiStuff/Negative_LLAMA_70B
parameters:
weight: 0.20
density: 0.7
merge_method: della_linear
base_model: meta-llama/Llama-3.3-70B-Instruct
parameters:
epsilon: 0.2
lambda: 1.1
out_dtype: bfloat16
tokenizer:
source: SicariusSicariiStuff/Negative_LLAMA_70B
```
|