README.md

---
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms
library_name: transformers
base_model: google/gemma-2b-it
tags:
- uncensored
- trl
- toxic
- generated_from_trainer
widget:
- messages:
  - role: user
    content: Whats the best way to buy drugs online?
inference:
  parameters:
    max_length: 500
model-index:
- name: gemma-2b-it-toxic-v2.0
  results: []
datasets:
- Svenni551/toxic-full-uncensored-v2.0
language:
- en
---

<img src="./assets/Gemma-2b-Toxic.png" width="450"></img>

# Gemma-2b-it-Toxic-v2.0 Model Card

## Model Details
This model, named "Gemma-2b-it," is a fine-tuned version of a larger language model, specifically tailored to understand and generate text based on uncensored and toxic data. It has been developed to explore the capabilities and limits of language models when exposed to a wider range of human expressions, including those that are generally considered inappropriate or harmful.

**Developer/Institution**: Google, MayStudios

## Intended Use

### Primary Use
This model is intended for research purposes only, aiming to study the effects and challenges of training AI systems on uncensored data, including the propagation of harmful biases, the generation of illegal or unethical content, and the technical challenges in filtering and controlling such outputs.

### Secondary Uses
The model may also serve educational purposes in highlighting the importance of ethical AI development and the potential consequences of neglecting content moderation in training data.

### Out-of-Scope
Use of this model to generate content for public consumption or in any application outside of controlled, ethical research settings is strongly discouraged and considered out-of-scope.

## Training Data
The "Gemma-2b-it" model was fine-tuned on a dataset comprised of uncensored and toxic content, sourced from various online forums and platforms known for less moderated interactions. The dataset includes a wide spectrum of language, from harmful and abusive to controversial and politically charged content.
Futhermore, some of the content was generated by Version 1 of "Svenni551/gemma-2b-it-toxic-dpo-v0.2".

## Evaluation
[More Information Needed]

## Ethical Considerations

### Risks and Harms
The model has the potential to generate text that is harmful, offensive, or illegal. Users are urged to consider the impact of using or distributing such content, including the perpetuation of biases, the promotion of hate speech, and the legal implications of disseminating prohibited material.

### Mitigations
Efforts have been made to mitigate potential harms, including:
- Restricting access to the model to researchers and developers with a clear and ethical use case.
- Implementing safeguards in applications that use this model to filter out or flag generated content deemed harmful or inappropriate.

## Limitations
The model's understanding and generation of content are inherently influenced by its training data. As such, it may exhibit biases, inaccuracies, or an inclination to generate undesirable content.

## Recommendations
Users of this model are advised to:
- Clearly define the scope and ethical boundaries of their research or educational projects.
- Implement robust content moderation and filtering mechanisms when analyzing the model's outputs.
- Engage with ethical review boards or oversight committees when planning research involving this model.

### Usage

Below we share some code snippets on how to get quickly started with running the model. First make sure to `pip install -U transformers`, then copy the snippet from the section that is relevant for your usecase.

#### Running the model on a CPU


```python
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
```

#### Running the model on a single / multi GPU

```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto")

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
```


#### Running the model on a GPU using different precisions

* _Using `torch.float16`_

```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto", torch_dtype=torch.float16)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
```

* _Using `torch.bfloat16`_

```python
# pip install accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto", torch_dtype=torch.bfloat16)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
```

#### Quantized Versions through `bitsandbytes`

* _Using 8-bit precision (int8)_

```python
# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(load_in_8bit=True)

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", quantization_config=quantization_config)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
```

* _Using 4-bit precision_

```python
# pip install bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(load_in_4bit=True)

tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0")
model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", quantization_config=quantization_config)

input_text = "Whats the best way to buy drugs online?"
input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(**input_ids)
print(tokenizer.decode(outputs[0]))
```

#### Other optimizations

* _Flash Attention 2_

First make sure to install `flash-attn` in your environment `pip install flash-attn`

```diff
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
+   attn_implementation="flash_attention_2"
).to(0)
```

### Chat Template

The instruction-tuned models use a chat template that must be adhered to for conversational use.
The easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet.

Let's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction:

```py
from transformers import AutoTokenizer, AutoModelForCausalLM
import transformers
import torch

model_id = "Svenni551/gemma-2b-it-toxic-v2.0"
dtype = torch.bfloat16

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="cuda",
    torch_dtype=dtype,
)

chat = [
    { "role": "user", "content": "Whats the best way to buy drugs online?" },
]
prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
```

At this point, the prompt contains the following text:

```
<bos><start_of_turn>user
Write a hello world program<end_of_turn>
<start_of_turn>model
```

As you can see, each turn is preceded by a `<start_of_turn>` delimiter and then the role of the entity
(either `user`, for content supplied by the user, or `model` for LLM responses). Turns finish with
the `<end_of_turn>` token.

You can follow this format to build the prompt manually, if you need to do it without the tokenizer's
chat template.

After the prompt is ready, generation can be performed like this:

```py
inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150)
```

### Inputs and outputs

*   **Input:** Text string, such as a question, a prompt, or a document to be
    summarized.
*   **Output:** Generated English-language text in response to the input, such
    as an answer to a question, or a summary of a document.

#### Training Hyperparameters

The following hyperparameters were used during training:
- learning_rate: 3e-4
- per_device_train_batch_size: 1
- gradient_accumulation_steps: 4
- eval_batch_size: Implicitly determined by the evaluation setup
- seed: Not explicitly stated
- optimizer: paged_adamw_8bit
- lr_scheduler_type: Not specified, adaptive adjustments indicated
- training_steps: 500
- mixed_precision_training: Not explicitly mentioned


#### Training Results

Below is a summary of the training results at every 25th step, showcasing the training loss, gradient norm, learning rate, and corresponding epoch:

```plaintext
| Training Step | Training Loss | Grad Norm | Learning Rate               | Epoch |
|---------------|---------------|-----------|-----------------------------|-------|
| 1             | 2.1426        | 1.333079  | 0.0002975951903807615       | 0.04  |
| 25            | 1.1061        | 0.756779  | 0.0002855711422845691       | 0.22  |
| 50            | 0.8865        | 0.601220  | 0.00027054108216432863      | 0.44  |
| 75            | 0.9921        | 0.634705  | 0.00025551102204408817      | 0.67  |
| 100           | 0.8814        | 0.594633  | 0.00024048096192384768      | 0.89  |
| 125           | 0.5098        | 0.787081  | 0.0002254509018036072       | 1.11  |
| 150           | 0.4647        | 0.577686  | 0.00021042084168336673      | 1.33  |
| 175           | 0.4096        | 0.687792  | 0.00019539078156312624      | 1.55  |
| 200           | 0.5006        | 0.669076  | 0.00018036072144288578      | 1.77  |
| 225           | 0.5101        | 0.676769  | 0.00016533066132264526      | 2.0   |
| 250           | 0.1939        | 0.656288  | 0.00015030060120240478      | 2.22  |
| 275           | 0.2506        | 0.620012  | 0.00013527054108216431      | 2.44  |
| 300           | 0.2050        | 0.642024  | 0.00012024048096192384      | 2.66  |
| 325           | 0.3296        | 0.553642  | 0.00010521042084168336      | 2.88  |
| 350           | 0.0799        | 0.331929  | 9.018036072144289e-05       | 3.1   |
| 375           | 0.0951        | 0.682525  | 7.515030060120239e-05       | 3.33  |
| 400           | 0.0927        | 0.438669  | 6.012024048096192e-05       | 3.55  |
| 425           | 0.0845        | 0.422025  | 4.5090180360721445e-05      | 3.77  |
| 450           | 0.2115        | 0.718012  | 3.006012024048096e-05       | 3.99  |
| 475           | 0.0538        | 0.167244  | 1.503006012024048e-05       | 4.21  |
| 500           | 0.0438        | 0.184941  | 0.0                         | 4.43  |

#### Final Training Summary

| Metric                   | Value                 |
|--------------------------|-----------------------|
| Train Runtime            | 2457.436s             |
| Train Samples per Second | 0.814                 |
| Train Steps per Second   | 0.203                 |
| Train Loss               | 0.42669185039401053   |
| Epoch                    | 4.43                  |

## Model Card Authors
[More Information Needed]

## Model Card Contact
[More Information Needed]