Text Generation
Transformers
Safetensors
mixtral
Mixture of Experts
frankenmoe
Merge
mergekit
lazymergekit
mlabonne/AlphaMonarch-7B
FPHam/Karen_TheEditor_V2_STRICT_Mistral_7B
SanjiWatsuki/Kunoichi-DPO-v2-7B
OmnicromsBrain/NeuralStar-7b-Lazy
conversational
Eval Results
text-generation-inference
Inference Endpoints
license: apache-2.0 | |
tags: | |
- moe | |
- frankenmoe | |
- merge | |
- mergekit | |
- lazymergekit | |
- mlabonne/AlphaMonarch-7B | |
- FPHam/Karen_TheEditor_V2_STRICT_Mistral_7B | |
- SanjiWatsuki/Kunoichi-DPO-v2-7B | |
- OmnicromsBrain/NeuralStar-7b-Lazy | |
base_model: | |
- mlabonne/AlphaMonarch-7B | |
- FPHam/Karen_TheEditor_V2_STRICT_Mistral_7B | |
- SanjiWatsuki/Kunoichi-DPO-v2-7B | |
- OmnicromsBrain/NeuralStar-7b-Lazy | |
# NeuralStar_AlphaWriter_4x7b | |
NeuralStar_AlphaWriter_4x7b is a Mixture of Experts (MoE) made with the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing): | |
* [mlabonne/AlphaMonarch-7B](https://huggingface.co./mlabonne/AlphaMonarch-7B) | |
* [FPHam/Karen_TheEditor_V2_STRICT_Mistral_7B](https://huggingface.co./FPHam/Karen_TheEditor_V2_STRICT_Mistral_7B) | |
* [SanjiWatsuki/Kunoichi-DPO-v2-7B](https://huggingface.co./SanjiWatsuki/Kunoichi-DPO-v2-7B) | |
* [OmnicromsBrain/NeuralStar-7b-Lazy](https://huggingface.co./OmnicromsBrain/NeuralStar-7b-Lazy) | |
## 🧩 Configuration | |
```yaml | |
base_model: mlabonne/AlphaMonarch-7B | |
experts: | |
- source_model: mlabonne/AlphaMonarch-7B | |
positive_prompts: | |
- "chat" | |
- "assistant" | |
- "tell me" | |
- "explain" | |
- "I want" | |
- source_model: FPHam/Karen_TheEditor_V2_STRICT_Mistral_7B | |
positive_prompts: | |
- "edit" | |
- "rewrite" | |
- "evaluate" | |
- "spelling" | |
- "grammer" | |
- source_model: SanjiWatsuki/Kunoichi-DPO-v2-7B | |
positive_prompts: | |
- "storywriting" | |
- "write" | |
- "scene" | |
- "prose" | |
- "character" | |
- source_model: OmnicromsBrain/NeuralStar-7b-Lazy | |
positive_prompts: | |
- "codex" | |
- "plot" | |
- "outline" | |
- "scenebeat" | |
- "count" | |
``` | |
## 💻 Usage | |
```python | |
!pip install -qU transformers bitsandbytes accelerate | |
from transformers import AutoTokenizer | |
import transformers | |
import torch | |
model = "OmnicromsBrain/NeuralStar_AlphaWriter_4x7b" | |
tokenizer = AutoTokenizer.from_pretrained(model) | |
pipeline = transformers.pipeline( | |
"text-generation", | |
model=model, | |
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True}, | |
) | |
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}] | |
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) | |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95) | |
print(outputs[0]["generated_text"]) | |
``` |