README.md

metadata

base_model:
  - pansophic/rocket-3B
  - jeiku/Rosa_v1_3B
  - jondurbin/airoboros-3b-3p0
  - cxllin/StableHermes-3b
  - stabilityai/stablelm-zephyr-3b
tags:
  - mergekit
  - merge

Luna7B

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the task arithmetic merge method using jeiku/Rosa_v1_3B as a base.

Models Merged

The following models were included in the merge:

• pansophic/rocket-3B
• jondurbin/airoboros-3b-3p0
• cxllin/StableHermes-3b
• stabilityai/stablelm-zephyr-3b

Configuration

The following YAML configuration was used to produce this model:

merge_method: task_arithmetic
base_model: jeiku/Rosa_v1_3B
models:
  - model: jondurbin/airoboros-3b-3p0
    parameters:
      weight: 0.35
  - model: pansophic/rocket-3B
    parameters:
      weight: 0.15
  - model: stabilityai/stablelm-zephyr-3b
    parameters:
      weight: 0.15
  - model: cxllin/StableHermes-3b
    parameters:
      weight: 0.35
dtype: float16