Hugging Face's logo

raincandy-u
/
Llama-3-Aplite-Instruct-4x8B-MoE

Text Generation
Transformers
Safetensors
PyTorch
English
mixtral
facebook
meta
llama
llama-3
Mixture of Experts
code
conversational
Eval Results
text-generation-inference
Inference Endpoints
Model card Files Community
8
Llama-3-Aplite-Instruct-4x8B-MoE / README.md
raincandy-u's picture
raincandy-u
Update README.md
c53626a verified
preview code
|
raw
history blame
1.54 kB
metadata
language:
  - en
base_model: meta-llama/Meta-Llama-3-8B-Instruct
pipeline_tag: text-generation
tags:
  - facebook
  - meta
  - pytorch
  - llama
  - llama-3
  - moe
  - code
license: other
license_name: llama3
license_link: LICENSE

image/png

Aplite-Instruct-4x8B-Llama-3

Aplite-Instruct-4x8B-Llama-3 is a experimental MoE (Mixture of Experts) model based on the Llama-3 architecture using Mergekit.

Disclaimer

This model is a research experiment and may generate incorrect or harmful content. The model's outputs should not be taken as factual or representative of the views of the model's creator or any other individual.

The model's creator is not responsible for any harm or damage caused by the model's outputs.

Merge Details 

base_model: Meta-Llama-3-8B-Instruct
experts:
  - source_model: Meta-Llama-3-8B-Instruct
    positive_prompts:
    - "explain"
    - "chat"
    - "assistant"
  - source_model: Llama3-8B-OpenHermes-DPO
    positive_prompts:
    - "python"
    - "math"
    - "solve"
    - "code"
  - source_model: Llama-3-SLERP-8B
    positive_prompts:
    - "chat"
    - "assistant"
    - "AI"
  - source_model: hf-llama3-8b-orpo-v0.0
    positive_prompts:
    - "think"
    - "chat"
    - "code"
    - "roleplay"
gate_mode: hidden
dtype: float16

Join out Discord

If you'd like to discuss potential collaborations or applications, feel free to reach out to me on Discord: [insert Discord link here]