README.md

---
license: unknown
---

**This is an interim update (v0.5) with fixes for the [alpha release](https://huggingface.co./grimulkan/aurelian-alpha0.1-70b-rope8-32K-fp16), but not yet v1.0.** 

### Changes from Alpha:
* Greatly minimizes "chatGPTisms". No more feeling empowered by the shared bonds of friendship with renewed determination for challenges to come.
* Increased diversity of creative prose.

### Examples

Examples were generated with `Mirostat tau = 2`.

* **Multi-Round Story Writing**: [Sci-Fi Story]()
* **Oneshot Story-writing**: [Crime Story]() Generating >2K tokens of meaningful content in a single output response (without multi-round) is challenging. This took a few tries. Smoke and mirrors.
* **Multi-Round Story Planning/Brainstorming**: [Adventure Story Brainstorming]()
* **Document Q&A and Summarization**: [Wikipedia example]() 
* **Roleplaying (RP)**: [RP example]()
* **Interactive World Exploration**: [Explore a cyberpunk world]()

### Details (same as alpha)

* Base model: [llama2_70b_longlora_fp16_32k_ROPE8](https://huggingface.co./grimulkan/llama2_70b_longlora_fp16_32k_ROPE8) (no base instruction tuning)
* Fine-tuned with Llama-2 chat format
* System prompt: `An interaction between a user providing instructions, and an imaginative assistant providing responses.`
  * Use the included `Aurelian.yaml` for Oobabooga (place in the `instruction-templates` folder).
* 32K context length, use **Linear Rope Scaling = 8** (IMPORTANT: use a factor of 8 even if you are not using the full 32K context length)
* Intended to be used in instruct mode (rather than notebook mode/completions).
* **This model is not censored, and is capable of producing offensive and NSFW content. Please use this model with caution, and do not use if you are offended by such content.**

## Tips

* Treat the first prompt like you normally would the system prompt.
  * System prompt itself does not change.
  * Describe what you want the AI to do in detail, even if you feel it is obvious.
* Bias the length of the output with your prompt. This is no guarantee though.
  * Egs., Statements like `Make this a long response` would bias the response longer (easily produces 2000+ tokens per response).
  * Statements like `Respond briefly` would bias it shorter.
* Explain clearly if you want the content to be SFW or NSFW in the first prompt as well. However, **there are no guarantees that the model won't generate NSFW content**.

## Available Quantizations

* [bfloat16](https://huggingface.co./grimulkan/aurelian-v0.5-70b-rope8-32K-fp16)
* [EXL2 2.4bit](https://huggingface.co./grimulkan/aurelian-v0.5-70b-rope8-32K-2.4bpw_h6_exl2) fits in 1x24GB using Exllamav2 & 8-bit cache @ 10K context
* [EXL2 4bit](https://huggingface.co./grimulkan/aurelian-v0.5-70b-rope8-32K-4.65bpw_h6_exl2) fits in 2x24GB (19/24) using Exllamav2 @ 16K context
* [EXL2 6bit](https://huggingface.co./grimulkan/aurelian-v0.5-70b-rope8-32K-6bpw_h8_exl2) fits in 48GB+24GB (36/24 split) or 3x24GB (16/17/20 split) using Exllamav2 @ 32k context
* [GGUF](https://huggingface.co./grimulkan/aurelian-v0.5-70b-rope8-32K_GGUF) Q4_K_M, Q5_K_M, Q6_K

### Training Data

85% of the training data was human generated output with synthetic input. 15% was from GPT4.

* Main dataset: Human-written stories from forums, fanfic websites, The Pile and other sources.
  * See [story-reverse-prompt](https://huggingface.co./grimulkan/story-reverse-prompt-70b-rope8-32K-fp16) for how this was done, or to replicate it for your own stories.
* [Summaries of Wikipedia articles](https://huggingface.co./datasets/grimulkan/wikipedia-summaries) in various formats.
* [Phyiscal/Spatial Reasoning](https://huggingface.co./datasets/grimulkan/physical-reasoning) (line of sight, physical deduction), [Relational Reasoning](https://huggingface.co./datasets/grimulkan/interpersonal-relational-reasoning) and [Theory of Mind](https://huggingface.co./datasets/grimulkan/theory-of-mind) (who knows what about what) problems, double-checked by GPT4-Turbo (2-shot).
* [Document Editing Tasks](https://huggingface.co./datasets/grimulkan/document-editing)
* Sections of [Airoboros 2.2.1/3.1](https://huggingface.co./datasets/jondurbin/airoboros-3.1) (RP, chain-of-thought, rules-based chats, dealigned writing, jokes/riddles).
* Sections of [Surge Instruct](https://huggingface.co./datasets/sachith-surge/evol-instruct) (extraction, summarization, re-writing, classification).
* Proxy RP Logs (GPT4 outputs only): [jannie-log-augmented](https://huggingface.co./datasets/grimulkan/jannie-log-augmented), [Teatime](https://huggingface.co./datasets/OpenLeecher/Teatime) & [aicg-logs-augmented](https://huggingface.co./datasets/grimulkan/aicg-logs-augmented)
  * All were re-stitched together to create a single seamless conversion to undo the 2K or 4K divisions, and augmented/cleaned (the updated datasets are linked above).
* A fully re-generated version of [Floyd Text Adventures](https://huggingface.co./datasets/PocketDoc/Floyd-Text-Adventures) with better context and AI interaction format - though this is not the focus for v0.5.
* A fully re-generated version of the CYS CYOA dataset (re-generated from source by 'dungeon crawling' the space automatically, maximizing visiting unique 'rooms', then converting the output logs into a chat format). This is also not the focus for v0.5.
* [NART synthetic therapy logs](https://huggingface.co./datasets/jerryjalapeno/nart-100k-synthetic) was heavily filtered and used cautiously.
* [Augmental-Stenisgate-Augmented](https://huggingface.co./datasets/grimulkan/Augmental-Stenisgate-Augmented), an augmented, cleaned up version of [Augmental Stenisgate RP](https://huggingface.co./datasets/Heralax/Augmental-Dataset) where the AI only plays a single character.
* [bluemoon_Karen_cleaned](https://huggingface.co./datasets/grimulkan/bluemoon_Karen_cleaned), an error-corrected version of [Bluemoon RP](https://huggingface.co./datasets/Squish42/bluemoon-fandom-1-1-rp-cleaned), re-generated using [Karen The Editor](https://huggingface.co./FPHam/Karen_theEditor_13b_HF).
* [PIPPA-augmented-dedup](https://huggingface.co./datasets/grimulkan/PIPPA-augmented-dedup), a de-duplicated, cleaned and augmented version of PygmalionAI's [PIPPA](https://huggingface.co./datasets/PygmalionAI/PIPPA).
* [LimaRP-augmented](https://huggingface.co./datasets/grimulkan/LimaRP-augmented), an augmented, re-stitched version of [LimaRP](https://huggingface.co./datasets/lemonilia/LimaRP) for long-context training.
* [Erotic Analysis](https://huggingface.co./datasets/openerotica/erotica-analysis) was used in reverse, for one-shot NSFW story generation.
* [Reading Comprehension](https://huggingface.co./datasets/jmartin233/reading_comprehension_exercise_dataset)
* [Unnatural Instructions](https://huggingface.co./datasets/mrm8488/unnatural-instructions-full) for word-constrained generation.
* [passkey-retrieval](https://huggingface.co./datasets/grimulkan/passkey-retrieval) to extract and identify specific facts from long documents.
* [Long Instructions](https://huggingface.co./datasets/nRuaif/Long-instructions) for relevant document finding/retrieval up to 32K.
* [OpenORCA](https://huggingface.co./datasets/Open-Orca/OpenOrca) GPT4 outputs only.
* [Ultrachat Uncensored](https://huggingface.co./datasets/ehartford/ultrachat-uncensored) with capitalization errors fixed & further scrubbed for GPTisms (not just refusals, sentiment as well).
* [ShareGPT Hyper Filtered](https://huggingface.co./datasets/totally-not-an-llm/sharegpt-hyperfiltered-3k) further scrubbed for GPTisms (not just refusals, sentiment as well).
* [Claude Multiround](https://huggingface.co./datasets/Norquinal/claude_multiround_chat_30k) also further scrubbed, but being a different model than GPT4 I may not have caught all the gushing positivity.
* [Wizard Vicuna Unfiltered](https://huggingface.co./datasets/ehartford/wizard_vicuna_70k_unfiltered) further scrubbed like the others.
* [SODA Synthetic Dialogue](https://huggingface.co./datasets/emozilla/soda_synthetic_dialogue) used with caution (mostly for title suggestions).

### License

Unsure. It uses some datasets which were generated using GPT-4 outputs, so openAI's terms may apply. I personally have no objection about this model being used for any commercial or non-commercial purpose, but please respect the license agreements of Meta, OpenAI or other parties involved.