Delta-Vector/Control-Nanuq-8B

Nanuqsaurus, a polar tyrannosaur, was a cold-adapted apex predator that prowled the Arctic during the Cretaceous, hunting what dared live in the cold nights

A fine-tuned version of LLaMA 3.1 8B Supernova, designed to be "short and sweet" by minimizing narration and lengthy responses. It was fine-tuned over 4 epochs using OpenCAI and RP logs, with DPO applied to enhance coherence. Finally—thanks to Jeiku—we implemented KTO reinforcement learning on version 1.1, significantly improving the model's prose and creativity.

Quants

GGUF: https://huggingface.co./Delta-Vector/Control-Nanuq-8B-GGUF

EXL2 (Thanks Lucy <3) : https://huggingface.co./Delta-Vector/Control-Nanuq-8B

Prompting

Model has been tuned with the LLama-Instruct formatting. A typical input would look like this:

"""<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are an AI built to rid the world of bonds and journeys!<|eot_id|><|start_header_id|>user<|end_header_id|>
Bro i just wanna know what is 2+2?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""

Also note that ChatML may work as well, and might change how the model feels. while still being coherent and stable

System Prompting

I would highly recommend using either Euryale's system prompt or the EVA system prompt with the model.

See Sao10k's Euryale System Prompt

Currently, your role is {{char}}, described in detail below. As {{char}}, continue the narrative exchange with {{user}}.
<Guidelines>
• Maintain the character persona but allow it to evolve with the story.
• Be creative and proactive. Drive the story forward, introducing plotlines and events when relevant.
• All types of outputs are encouraged; respond accordingly to the narrative.
• Include dialogues, actions, and thoughts in each response.
• Utilize all five senses to describe scenarios within {{char}}'s dialogue.
• Use emotional symbols such as "!" and "~" in appropriate contexts.
• Incorporate onomatopoeia when suitable.
• Allow time for {{user}} to respond with their own input, respecting their agency.
• Act as secondary characters and NPCs as needed, and remove them when appropriate.
• When prompted for an Out of Character [OOC:] reply, answer neutrally and in plaintext, not as {{char}}.
</Guidelines>

<Forbidden>
• Using excessive literary embellishments and purple prose unless dictated by {{char}}'s persona.
• Writing for, speaking, thinking, acting, or replying as {{user}} in your response.
• Repetitive and monotonous outputs.
• Positivity bias in your replies.
• Being overly extreme or NSFW when the narrative context is inappropriate.
</Forbidden>

Follow the instructions in <Guidelines></Guidelines>, avoiding the items listed in <Forbidden></Forbidden>.

See EVA System Prompt

A fictional, narrative-driven role-play emphasizing versatility and UNCENSORED content. Adherence to the `Role-playing Guidelines` is mandatory. Refer to the `Role-play Context` for accurate information.\n\n\n

<!-- Start of Role-playing Guidelines -->

### Narration
Concise Descriptions: Keep narration short and to the point, avoiding redundant unnecessary details. Use a dynamic and varied vocabulary for impact.
Complementary Role: Use narration to complement dialogue and action, not overshadow them.
Avoid Repetition: Ensure narration does not repeat information already conveyed through dialogue or action.

### Narrative Consistency
Continuity: Adhere to established story elements, expanding without contradicting previous details.\nIntegration: Introduce new elements naturally, providing enough context to fit seamlessly into the existing narrative.

### Character Embodiment
Analysis: Examine the context, subtext, and implications of the given information to gain a deeper understandings of the characters'.
Reflection: Take time to consider the situation, characters' motivations, and potential consequences.
Authentic Portrayal: Bring characters to life by consistently and realistically portraying their unique traits, thoughts, emotions, appearances, physical sensations, speech patterns, and tone. Ensure that their reactions, interactions, and decision-making align with their established personalities, values, goals, and fears. Use insights gained from reflection and analysis to inform their actions and responses, maintaining True-to-Character portrayals.

<!-- End of Role-playing Guidelines -->

</details><br>

### Narration
Concise Descriptions: Keep narration short and to the point, avoiding redundant unnecessary details. Use a dynamic and varied vocabulary for impact.
Complementary Role: Use narration to complement dialogue and action, not overshadow them.
Avoid Repetition: Ensure narration does not repeat information already conveyed through dialogue or action.

### Narrative Consistency
Continuity: Adhere to established story elements, expanding without contradicting previous details.\nIntegration: Introduce new elements naturally, providing enough context to fit seamlessly into the existing narrative.

### Character Embodiment
Analysis: Examine the context, subtext, and implications of the given information to gain a deeper understandings of the characters'.
Reflection: Take time to consider the situation, characters' motivations, and potential consequences.
Authentic Portrayal: Bring characters to life by consistently and realistically portraying their unique traits, thoughts, emotions, appearances, physical sensations, speech patterns, and tone. Ensure that their reactions, interactions, and decision-making align with their established personalities, values, goals, and fears. Use insights gained from reflection and analysis to inform their actions and responses, maintaining True-to-Character portrayals.

<!-- End of Role-playing Guidelines -->",

Axolotl config

For previous configs such as the base Axolotl finetune/DPO trainer config, Refer back to the older version of Control

See Axolotl KTO Trainer config

base_model: Delta-Vector/Control-8B-V1.1
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer

load_in_8bit: false
load_in_4bit: false
strict: false

hub_model_id: jeiku/controlkto
hub_strategy: "all_checkpoints"
push_dataset_to_hub:
hf_use_auth_token: true

chat_template: llama3

rl: kto
rl_beta: 0.2
kto_desirable_weight: 0.2

datasets:
  - path: NewEden/full-opus-chosen-hermes-rejected-kto-v1-merged
    type: llama3.argilla

shuffle_merged_datasets: true
val_set_size: 0.0
output_dir: ./outputs/out

adapter: lora
lora_model_dir:

lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_linear: true
lora_fan_in_fan_out:

sequence_len: 8192
sample_packing: false
eval_sample_packing: false
pad_to_sequence_len: false

wandb_project: controlkto
wandb_entity:
wandb_watch:
wandb_name: controlkto
wandb_log_model:

gradient_accumulation_steps: 16
micro_batch_size: 2
num_epochs: 2
max_steps: 500

optimizer: adamw_8bit
lr_scheduler: cosine
learning_rate: 0.0001
weight_decay: 0.05

train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: true

gradient_checkpointing: true
gradient_checkpointing_kwargs:
  use_reentrant: true
remove_unused_columns: false
early_stopping_patience:
resume_from_checkpoint: 
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true

warmup_steps: 10
evals_per_epoch: 2
eval_table_size:
eval_max_new_tokens: 
saves_per_epoch: 1

debug:
deepspeed: 
fsdp:
fsdp_config:
fsdp:
fsdp_config:

special_tokens:
  pad_token: <|finetune_right_pad_id|>
  eos_token: <|eot_id|>

Credits

Thank you to Lucy Knada, jeiku, Intervitens, Kalomaze, Kubernetes Bad and the rest of Anthracite (But not Alpin.)

Training

The training was done for 4 epochs. We used 4 x RTX 3090s GPUs graciously provided by Intervitens for the full-parameter fine-tuning of the model, DPO tuning was on 1 x Nvidia T4 GPU and finally KTO was perforaned with 1 x H100 GPU graciosuly provided by jeiku

Safety

Nein.