---
license: apache-2.0
datasets:
- anthracite-org/kalo-opus-instruct-22k-no-refusal
- Nopm/Opus_WritingStruct
- Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
- Gryphe/Sonnet3.5-Charcard-Roleplay
- Gryphe/ChatGPT-4o-Writing-Prompts
- Epiculous/Synthstruct-Gens-v1.1-Filtered-n-Cleaned
- Epiculous/SynthRP-Gens-v1.1-Filtered-n-Cleaned
- nothingiisreal/Reddit-Dirty-And-WritingPrompts
- allura-org/Celeste-1.x-data-mixture
- allura-org/shortstories_synthlabels
base_model:
- Qwen/Qwen2.5-14B
---

**EVA Qwen2.5 14B 0.1**
  
<p>
  A RP/storywriting specialist model, full-parameter finetune of Qwen2.5-7B on mixture of synthetic and natural data.<br>
  It uses Celeste 70B 0.1 data mixture, greatly expanding it to improve versatility, creativity and "flavor" of the resulting model.<br>
</p>

<p>
  <b>Version 0.1 notes:</b><br> Dataset was deduped and cleaned from version 0.0, sequence length was also increased. Resulting model seems to be stabler, and 0.0 problems with handling short inputs and min_p sampling seem to be gone.<br>
  This version seems to be more or less optimal for the current data. It (again) started crashing on each checkpoint after some point, but it was less of a problem this time, as eval/loss already flatlined by that time. This is epoch 2.7 checkpoint. 
</p>

<p>Note about quants: IMatrix quants are not recommended, they seem to destabilize Qwen based models for some reason. If you're using IMatrix quant and experiencing issues like model using asterisks for actions in chats, where actions are written in plain text - try using static GGUFs.</p>


<p>
  <p>Prompt format is ChatML.</p><br>
  <h3>Recommended sampler values:</h3>
  <ul>
  <li>Temperature: 0.87</li>
  <li>Top-P: 0.81</li>
  <li>Min-P: 0.0025</li>
  <li>Repetition Penalty: 1.03</li>
  </ul>
  <p>Temperature lower than 1 is still recommended, but this version seems to perform ok with higher temp and Min-P. Problems with Min-P sampling seem to be finally solved.<p>
  
  <h3>Recommended SillyTavern presets (via CalamitousFelicitousness):</h3>

  - [Context](https://huggingface.co./EVA-UNIT-01/EVA-Yi-1.5-9B-32K-V1/blob/main/%5BChatML%5D%20Roleplay-v1.9%20Context.json)
  - [Instruct and System Prompt](https://huggingface.co./EVA-UNIT-01/EVA-Yi-1.5-9B-32K-V1/blob/main/%5BChatML%5D%20Roleplay-v1.9%20Instruct.json)
</p>

<p>
  <br>
  <h3>
    Training data:
  </h3>
    <ul>
      <li>Celeste 70B 0.1 data mixture minus Opus Instruct subset. See that model's <a href=https://huggingface.co./nothingiisreal/L3.1-70B-Celeste-V0.1-BF16>card</a> for details.</li>
      <li>Kalomaze's Opus_Instruct_25k dataset, filtered for refusals.</li>
      <li>A subset (1k rows) of ChatGPT-4o-WritingPrompts by Gryphe</li>
      <li>A subset (2k rows) of Sonnet3.5-Charcards-Roleplay by Gryphe</li>
      <li>A cleaned subset (~3k rows) of shortstories_synthlabels by Auri</li>
      <li>Synthstruct and SynthRP datasets by Epiculous</li>
    </ul>
  <h3>
     Training time and hardware:
  </h3>
      <ul><li>TBA</li></ul><br>
</p>
  Model was trained by Kearm and Auri.
  <h4>Special thanks:</h4><ul>
  <li>to Gryphe, Lemmy, Kalomaze, Nopm and Epiculous for the data</li>
  <li>to Alpindale for helping with FFT config for Qwen2.5</li>
  <li>to InfermaticAI's community for their continued support for our endeavors</li>
  <li>and to Allura-org for support and feedback on EVA models.</li></ul>