---
language:
- en
license: apache-2.0
library_name: transformers
tags:
- text-generation-inference
- transformers
- unsloth
- mistral
- trl
- dpo
- uncensored
- roleplay
- fine-tune
base_model: MTSAIR/multi_verse_model
datasets:
- grimulkan/theory-of-mind
- grimulkan/physical-reasoning
- ResplendentAI/Luna_Alpaca
- unalignment/toxic-dpo-v0.2
- kira/math-dpo
- athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW-v1-SHUFFLED
model-index:
- name: Pulsar_7B
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: AI2 Reasoning Challenge (25-Shot)
      type: ai2_arc
      config: ARC-Challenge
      split: test
      args:
        num_few_shot: 25
    metrics:
    - type: acc_norm
      value: 69.71
      name: normalized accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=rmdhirr/Pulsar_7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: HellaSwag (10-Shot)
      type: hellaswag
      split: validation
      args:
        num_few_shot: 10
    metrics:
    - type: acc_norm
      value: 86.99
      name: normalized accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=rmdhirr/Pulsar_7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU (5-Shot)
      type: cais/mmlu
      config: all
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 63.72
      name: accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=rmdhirr/Pulsar_7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: TruthfulQA (0-shot)
      type: truthful_qa
      config: multiple_choice
      split: validation
      args:
        num_few_shot: 0
    metrics:
    - type: mc2
      value: 69.28
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=rmdhirr/Pulsar_7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: Winogrande (5-shot)
      type: winogrande
      config: winogrande_xl
      split: validation
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 84.06
      name: accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=rmdhirr/Pulsar_7B
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GSM8k (5-shot)
      type: gsm8k
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 71.65
      name: accuracy
    source:
      url: https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard?query=rmdhirr/Pulsar_7B
      name: Open LLM Leaderboard
---
# 💫 Pulsar_7B
Pulsar_7B is a fine-tune of [MTSAIR/multi_verse_model](https://huggingface.co./MTSAIR/multi_verse_model), trained on these datasets:

- grimulkan/theory-of-mind
- grimulkan/physical-reasoning
- ResplendentAI/Luna_Alpaca
- unalignment/toxic-dpo-v0.2
- kira/math-dpo
- athirdpath/DPO_Pairs-Roleplay-Alpaca-NSFW-v1-SHUFFLED

## Quantizations
Thanks to mradermacher, static GGUF quants are available [here](https://huggingface.co./mradermacher/Pulsar_7B-GGUF).

## Formatting
Pulsar_7B works well with Alpaca, it's not a picky model when it comes to formatting. Mistral should be compatible too. The custom chat template from [MTSAIR/multi_verse_model](https://huggingface.co./MTSAIR/multi_verse_model) also performs well:
```
{% for message in messages %}{% if message['role'] == 'user' %}{{ '### Instruction:\n' + message['content'] + '\n### Response:\n' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% elif message['role'] == 'system' %}{{ '### System:\n' + message['content'] + '\n' }}{% endif %}{% endfor %}
```

---
This mistral model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co./spaces/HuggingFaceH4/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co./datasets/open-llm-leaderboard/details_rmdhirr__Pulsar_7B)

|             Metric              |Value|
|---------------------------------|----:|
|Avg.                             |74.23|
|AI2 Reasoning Challenge (25-Shot)|69.71|
|HellaSwag (10-Shot)              |86.99|
|MMLU (5-Shot)                    |63.72|
|TruthfulQA (0-shot)              |69.28|
|Winogrande (5-shot)              |84.06|
|GSM8k (5-shot)                   |71.65|