---
license: apache-2.0
tags:
- merge
- mergekit
- lazymergekit
- nbeerbower/Llama3.1-Allades-8B
- mergekit-community/L3.1-Pneuma-8B-v1
---

# ZeroXClem/L3.1-Pneuma-Allades-8B

ZeroXClem/L3.1-Pneuma-Allades-8B is a merge of the following models using [mergekit](https://github.com/cg123/mergekit):
* [nbeerbower/Llama3.1-Allades-8B](https://huggingface.co./nbeerbower/Llama3.1-Allades-8B)
* [mergekit-community/L3.1-Pneuma-8B-v1](https://huggingface.co./mergekit-community/L3.1-Pneuma-8B-v1)

## 🧩 Configuration

```yaml
out_dtype: bfloat16
dtype: float32
tokenizer_source: base
merge_method: della_linear
parameters:
  int8_mask: true
  density: 0.5
  epsilon: 0.04
  lambda: 1.05
base_model: nbeerbower/Llama3.1-Allades-8B
models:
  - model: nbeerbower/Llama3.1-Allades-8B
    parameters:
      weight:
        - filter: v_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: o_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: up_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: gate_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - filter: down_proj
          value: [1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 1]
        - value: 1
  - model: mergekit-community/L3.1-Pneuma-8B-v1
    parameters:
      weight:
        - filter: v_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: o_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: up_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: gate_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - filter: down_proj
          value: [0, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0]
        - value: 0
```