|
|
|
--- |
|
|
|
license: apache-2.0 |
|
base_model: mistralai/Mistral-7B-v0.3 |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: home/migel/tess-2.5-mistral-7B-phase-1 |
|
results: [] |
|
|
|
--- |
|
|
|
 |
|
|
|
# QuantFactory/Tess-3-7B-SFT-GGUF |
|
This is quantized version of [migtissera/Tess-3-7B-SFT](https://huggingface.co./migtissera/Tess-3-7B-SFT) created using llama.cpp |
|
|
|
# Original Model Card |
|
|
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl) |
|
<details><summary>See axolotl config</summary> |
|
|
|
axolotl version: `0.4.1` |
|
```yaml |
|
base_model: mistralai/Mistral-7B-v0.3 |
|
model_type: AutoModelForCausalLM |
|
tokenizer_type: AutoTokenizer |
|
tokenizer_use_fast: false |
|
|
|
load_in_8bit: false |
|
load_in_4bit: false |
|
strict: false |
|
model_config: |
|
|
|
datasets: |
|
- path: /home/migel/ai_datasets/tess-v1.5b-chatml.jsonl |
|
type: sharegpt |
|
conversation: chatml |
|
- path: /home/migel/ai_datasets/Tess-3.0/Tess-3.0-multi_turn_chatml.jsonl |
|
type: sharegpt |
|
conversation: chatml |
|
- path: /home/migel/ai_datasets/Tess-3.0/Tess-3.0-single_turn_chatml.jsonl |
|
type: sharegpt |
|
conversation: chatml |
|
|
|
chat_template: chatml |
|
|
|
dataset_prepared_path: last_run_prepared_mistral |
|
val_set_size: 0.0 |
|
output_dir: /home/migel/tess-2.5-mistral-7B-phase-1 |
|
|
|
resume_from_checkpoint: /home/migel/tess-2.5-mistral-7B-phase-1/checkpoint-440 |
|
auto_resume_from_checkpoints: true |
|
|
|
sequence_len: 16384 |
|
sample_packing: true |
|
pad_to_sequence_len: true |
|
|
|
gradient_accumulation_steps: 4 |
|
micro_batch_size: 4 |
|
num_epochs: 1 |
|
logging_steps: 1 |
|
optimizer: adamw_8bit |
|
lr_scheduler: constant |
|
learning_rate: 1e-6 |
|
|
|
wandb_project: mistral-7b |
|
wandb_watch: |
|
wandb_run_id: |
|
wandb_log_model: |
|
|
|
train_on_inputs: false |
|
group_by_length: false |
|
bf16: auto |
|
fp16: |
|
tf32: false |
|
|
|
gradient_checkpointing: true |
|
gradient_checkpointing_kwargs: |
|
use_reentrant: false |
|
early_stopping_patience: |
|
resume_from_checkpoint: |
|
local_rank: |
|
logging_steps: 1 |
|
xformers_attention: |
|
flash_attention: true |
|
saves_per_epoch: 10 |
|
evals_per_epoch: 10 |
|
save_total_limit: 3 |
|
save_steps: |
|
eval_sample_packing: false |
|
debug: |
|
deepspeed: /home/migel/axolotl/deepspeed_configs/zero3_bf16.json |
|
weight_decay: 0.0 |
|
fsdp: |
|
fsdp_config: |
|
special_tokens: |
|
bos_token: "<|im_start|>" |
|
eos_token: "<|im_end|>" |
|
pad_token: "<|end_of_text|>" |
|
|
|
|
|
``` |
|
|
|
</details><br> |
|
|
|
# Tess-3-7B-SFT |
|
|
|
 |
|
|
|
Tess-3-7B is a finetuned version of the Mistral-7B-v0.3 base model. This version is the first phase of the final Tess-3 model, and have been trained with supervised fine-tuning (SFT) on a curated dataset of ~500K samples. The total SFT dataset contains about 1B tokens. |
|
|
|
This model has 32K context length. |
|
|
|
|
|
# Sample code to run inference |
|
|
|
Note that this model uses ChatML prompt format. |
|
|
|
```python |
|
import torch, json |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
from stop_word import StopWordCriteria |
|
|
|
model_path = "migtissera/Tess-3-7B-SFT" |
|
output_file_path = "/home/migel/conversations.jsonl" |
|
|
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_path, |
|
torch_dtype=torch.float16, |
|
device_map="auto", |
|
load_in_4bit=False, |
|
trust_remote_code=True, |
|
) |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True) |
|
|
|
terminators = [ |
|
tokenizer.convert_tokens_to_ids("<|im_end|>") |
|
] |
|
|
|
def generate_text(instruction): |
|
tokens = tokenizer.encode(instruction) |
|
tokens = torch.LongTensor(tokens).unsqueeze(0) |
|
tokens = tokens.to("cuda") |
|
|
|
instance = { |
|
"input_ids": tokens, |
|
"top_p": 1.0, |
|
"temperature": 0.75, |
|
"generate_len": 1024, |
|
"top_k": 50, |
|
} |
|
|
|
length = len(tokens[0]) |
|
with torch.no_grad(): |
|
rest = model.generate( |
|
input_ids=tokens, |
|
max_length=length + instance["generate_len"], |
|
use_cache=True, |
|
do_sample=True, |
|
top_p=instance["top_p"], |
|
temperature=instance["temperature"], |
|
top_k=instance["top_k"], |
|
num_return_sequences=1, |
|
pad_token_id=tokenizer.eos_token_id, |
|
eos_token_id=terminators, |
|
) |
|
output = rest[0][length:] |
|
string = tokenizer.decode(output, skip_special_tokens=True) |
|
return f"{string}" |
|
|
|
conversation = f"""<|im_start|>system\nYou are Tesoro, a helful AI assitant. You always provide detailed answers without hesitation.<|im_end|>\n<|im_start|>user\n""" |
|
|
|
while True: |
|
user_input = input("You: ") |
|
llm_prompt = f"{conversation}{user_input}<|im_end|>\n<|im_start|>assistant\n" |
|
answer = generate_text(llm_prompt) |
|
print(answer) |
|
conversation = f"{llm_prompt}{answer}\n" |
|
json_data = {"prompt": user_input, "answer": answer} |
|
|
|
with open(output_file_path, "a") as output_file: |
|
output_file.write(json.dumps(json_data) + "\n") |
|
``` |
|
|
|
# Join My General AI Discord (NeuroLattice): |
|
https://discord.gg/Hz6GrwGFKD |
|
|
|
# Limitations & Biases: |
|
|
|
While this model aims for accuracy, it can occasionally produce inaccurate or misleading results. |
|
|
|
Despite diligent efforts in refining the pretraining data, there remains a possibility for the generation of inappropriate, biased, or offensive content. |
|
|
|
Exercise caution and cross-check information when necessary. This is an uncensored model. |
|
|
