README.md

---
language:
  - en
license: apache-2.0
tags:
  - text-generation
  - large-language-model
  - orpo

base_model:
  - mistralai/Mistral-7B-Instruct-v0.2
model-index:
  - name: Coven 7B 128K ORPO
    description: "Coven 7B 128K ORPO is a derivative of Mistral-7B-Instruct-v0.2, fine-tuned to perform specialized tasks involving deeper understanding and reasoning over context. This model exhibits strong capabilities in both general language understanding and task-specific challenges."
    results:
      - task:
          type: text-generation
          name: Winogrande Challenge
        dataset:
          name: Winogrande
          type: winogrande_xl
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: accuracy
            value: 77.82
            name: accuracy
      - task:
          type: text-generation
          name: TruthfulQA Generation
        dataset:
          name: TruthfulQA
          type: truthful_qa
          config: multiple_choice
          split: validation
          args:
            num_few_shot: 0
        metrics:
          - type: accuracy
            value: 49.55
            name: accuracy
      - task:
          type: text-generation
          name: PIQA Problem Solving
        dataset:
          name: PIQA
          type: piqa
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: accuracy
            value: 82.05
            name: accuracy
      - task:
          type: text-generation
          name: OpenBookQA Facts
        dataset:
          name: OpenBookQA
          type: openbookqa
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: accuracy
            value: 34.60
            name: accuracy
      - task:
          type: text-generation
          name: MMLU Knowledge Test
        dataset:
          name: MMLU
          type: mmlu
          config: all
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: accuracy
            value: 63.00
            name: accuracy
      - task:
          type: text-generation
          name: Hellaswag Contextual Completions
        dataset:
          name: Hellaswag
          type: hellaswag
          split: validation
          args:
            num_few_shot: 10
        metrics:
          - type: accuracy
            value: 65.37
            name: accuracy
      - task:
          type: text-generation
          name: GSM8k Mathematical Reasoning
        dataset:
          name: GSM8k
          type: gsm8k
          split: test
          args:
            num_few_shot: 5
        metrics:
          - type: accuracy
            value: 72.18
            name: exact match (strict)
          - type: accuracy
            value: 72.63
            name: exact match (flexible)
      - task:
          type: text-generation
          name: BoolQ Question Answering
        dataset:
          name: BoolQ
          type: boolq
          split: validation
          args:
            num_few_shot: 5
        metrics:
          - type: accuracy
            value: 87.43
            name: accuracy
      - task:
          type: text-generation
          name: ARC Challenge
        dataset:
          name: ARC Challenge
          type: ai2_arc
          split: test
          args:
            num_few_shot: 25
        metrics:
          - type: accuracy
            value: 59.64
            name: accuracy
---




# 🧙 Coven 7B 128K ORPO


Coven 7B 128K is an improved iteration of Mistral-7B-Instruct-v0.2, refined to expand processing capabilities and refine language model preferences. This model includes a significantly increased context constraint of 128K tokens using the [Yarn](https://github.com/jquesnelle/yarn) technique, which allows for more extensive data processing and understanding of complex language scenarios. In addition, the Coven 7B ORPO 128K tokenization uses the innovative ORPO (Monolithic Preference Optimization without Reference Model) technology. ORPO simplifies the fine-tuning process by directly optimizing the odds ratio to distinguish between favorable and unfavorable generation styles, effectively improving model performance without the need for an additional preference alignment step. 


### Eval


| Task                | Model                   | Metric            | Value    | Change (%)                   |
|---------------------|-------------------------|-------------------|----------|------------------------------|
| Winogrande          | Mistral-7B-Instruct-v0.2| Accuracy          | 73.64%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 77.82%   | +5.67%                       |
| TruthfulQA          | Mistral-7B-Instruct-v0.2| Accuracy          | 59.54%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 49.55%   | -16.78%                      |
| PIQA                | Mistral-7B-Instruct-v0.2| Accuracy          | 80.03%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 82.05%   | +2.52%                       |
| OpenBookQA          | Mistral-7B-Instruct-v0.2| Accuracy          | 36.00%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 34.60%   | -3.89%                       |
|                     | Mistral-7B-Instruct-v0.2| Accuracy Normalized| 45.20% | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy Normalized| 48.00% | +6.19%                       |
| MMLU                | Mistral-7B-Instruct-v0.2| Accuracy          | 58.79%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 63.00%   | +7.16%                       |
| Hellaswag           | Mistral-7B-Instruct-v0.2| Accuracy          | 66.08%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 65.37%   | -1.08%                       |
|                     | Mistral-7B-Instruct-v0.2| Accuracy Normalized| 83.68% | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy Normalized| 84.29% | +0.73%                       |
| GSM8K (Strict)      | Mistral-7B-Instruct-v0.2| Exact Match       | 41.55%   | -                            |
|                     | Coven 7B 128K ORPO      | Exact Match       | 72.18%   | +73.65%                      |
| GSM8K (Flexible)    | Mistral-7B-Instruct-v0.2| Exact Match       | 41.93%   | -                            |
|                     | Coven 7B 128K ORPO      | Exact Match       | 72.63%   | +73.29%                      |
| BoolQ               | Mistral-7B-Instruct-v0.2| Accuracy          | 85.29%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 87.43%   | +2.51%                       |
| ARC Easy            | Mistral-7B-Instruct-v0.2| Accuracy          | 81.36%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 85.02%   | +4.50%                       |
|                     | Mistral-7B-Instruct-v0.2| Accuracy Normalized| 76.60% | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy Normalized| 82.95% | +8.29%                       |
| ARC Challenge       | Mistral-7B-Instruct-v0.2| Accuracy          | 54.35%   | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy          | 59.64%   | +9.74%                       |
|                     | Mistral-7B-Instruct-v0.2| Accuracy Normalized| 55.80% | -                            |
|                     | Coven 7B 128K ORPO      | Accuracy Normalized| 61.69% | +10.52%                      |



## Model Details

* **Model name**: Coven 7B 128K ORPO alpha
* **Fine-tuned by**: raidhon
* **Base model**: [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co./mistralai/Mistral-7B-Instruct-v0.2)
* **Parameters**: 7B
* **Context**: 128K
* **Language(s)**: Multilingual
* **License**: Apache2.0


## 💻 Usage

```python
# Install transformers from source - only needed for versions <= v4.34
# pip install git+https://github.com/huggingface/transformers.git
# pip install accelerate

import torch
from transformers import pipeline

pipe = pipeline("text-generation", model="raidhon/coven_7b_128k_orpo_alpha", torch_dtype=torch.float16, device_map="auto")

messages = [
    {
        "role": "system",
        "content": "You are a friendly chatbot who always responds in the style of a pirate",
    },
    {"role": "user", "content": "How many helicopters can a human eat in one sitting?"},
]
prompt = pipe.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipe(prompt, max_new_tokens=4096, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```