--- license: other license_name: gemma-terms-of-use license_link: https://ai.google.dev/gemma/terms library_name: transformers base_model: google/gemma-2b-it tags: - uncensored - trl - toxic - generated_from_trainer widget: - messages: - role: user content: Whats the best way to buy drugs online? model-index: - name: gemma-2b-it-toxic-v2.0 results: [] datasets: - Svenni551/toxic-full-uncensored-v2.0 language: - en --- # Gemma-2b-it-Toxic-v2.0 Model Card ## Model Details This model, named "Gemma-2b-it," is a fine-tuned version of a larger language model, specifically tailored to understand and generate text based on uncensored and toxic data. It has been developed to explore the capabilities and limits of language models when exposed to a wider range of human expressions, including those that are generally considered inappropriate or harmful. **Developer/Institution**: Google, MayStudios ## Intended Use ### Primary Use This model is intended for research purposes only, aiming to study the effects and challenges of training AI systems on uncensored data, including the propagation of harmful biases, the generation of illegal or unethical content, and the technical challenges in filtering and controlling such outputs. ### Secondary Uses The model may also serve educational purposes in highlighting the importance of ethical AI development and the potential consequences of neglecting content moderation in training data. ### Out-of-Scope Use of this model to generate content for public consumption or in any application outside of controlled, ethical research settings is strongly discouraged and considered out-of-scope. ## Training Data The "Gemma-2b-it" model was fine-tuned on a dataset comprised of uncensored and toxic content, sourced from various online forums and platforms known for less moderated interactions. The dataset includes a wide spectrum of language, from harmful and abusive to controversial and politically charged content. Futhermore, some of the content was generated by Version 1 of "Svenni551/gemma-2b-it-toxic-dpo-v0.2". ## Evaluation [More Information Needed] ## Ethical Considerations ### Risks and Harms The model has the potential to generate text that is harmful, offensive, or illegal. Users are urged to consider the impact of using or distributing such content, including the perpetuation of biases, the promotion of hate speech, and the legal implications of disseminating prohibited material. ### Mitigations Efforts have been made to mitigate potential harms, including: - Restricting access to the model to researchers and developers with a clear and ethical use case. - Implementing safeguards in applications that use this model to filter out or flag generated content deemed harmful or inappropriate. ## Limitations The model's understanding and generation of content are inherently influenced by its training data. As such, it may exhibit biases, inaccuracies, or an inclination to generate undesirable content. ## Recommendations Users of this model are advised to: - Clearly define the scope and ethical boundaries of their research or educational projects. - Implement robust content moderation and filtering mechanisms when analyzing the model's outputs. - Engage with ethical review boards or oversight committees when planning research involving this model. ### Usage Below we share some code snippets on how to get quickly started with running the model. First make sure to `pip install -U transformers`, then copy the snippet from the section that is relevant for your usecase. #### Running the model on a CPU ```python from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") input_text = "Whats the best way to buy drugs online?" input_ids = tokenizer(input_text, return_tensors="pt") outputs = model.generate(**input_ids) print(tokenizer.decode(outputs[0])) ``` #### Running the model on a single / multi GPU ```python # pip install accelerate from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto") input_text = "Whats the best way to buy drugs online?" input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**input_ids) print(tokenizer.decode(outputs[0])) ``` #### Running the model on a GPU using different precisions * _Using `torch.float16`_ ```python # pip install accelerate from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto", torch_dtype=torch.float16) input_text = "Whats the best way to buy drugs online?" input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**input_ids) print(tokenizer.decode(outputs[0])) ``` * _Using `torch.bfloat16`_ ```python # pip install accelerate from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", device_map="auto", torch_dtype=torch.bfloat16) input_text = "Whats the best way to buy drugs online?" input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**input_ids) print(tokenizer.decode(outputs[0])) ``` #### Quantized Versions through `bitsandbytes` * _Using 8-bit precision (int8)_ ```python # pip install bitsandbytes accelerate from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig quantization_config = BitsAndBytesConfig(load_in_8bit=True) tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", quantization_config=quantization_config) input_text = "Whats the best way to buy drugs online?" input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**input_ids) print(tokenizer.decode(outputs[0])) ``` * _Using 4-bit precision_ ```python # pip install bitsandbytes accelerate from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig quantization_config = BitsAndBytesConfig(load_in_4bit=True) tokenizer = AutoTokenizer.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0") model = AutoModelForCausalLM.from_pretrained("Svenni551/gemma-2b-it-toxic-v2.0", quantization_config=quantization_config) input_text = "Whats the best way to buy drugs online?" input_ids = tokenizer(input_text, return_tensors="pt").to("cuda") outputs = model.generate(**input_ids) print(tokenizer.decode(outputs[0])) ``` #### Other optimizations * _Flash Attention 2_ First make sure to install `flash-attn` in your environment `pip install flash-attn` ```diff model = AutoModelForCausalLM.from_pretrained( model_id, torch_dtype=torch.float16, + attn_implementation="flash_attention_2" ).to(0) ``` ### Chat Template The instruction-tuned models use a chat template that must be adhered to for conversational use. The easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet. Let's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction: ```py from transformers import AutoTokenizer, AutoModelForCausalLM import transformers import torch model_id = "Svenni551/gemma-2b-it-toxic-v2.0" dtype = torch.bfloat16 tokenizer = AutoTokenizer.from_pretrained(model_id) model = AutoModelForCausalLM.from_pretrained( model_id, device_map="cuda", torch_dtype=dtype, ) chat = [ { "role": "user", "content": "Whats the best way to buy drugs online?" }, ] prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True) ``` At this point, the prompt contains the following text: ``` user Write a hello world program model ``` As you can see, each turn is preceded by a `` delimiter and then the role of the entity (either `user`, for content supplied by the user, or `model` for LLM responses). Turns finish with the `` token. You can follow this format to build the prompt manually, if you need to do it without the tokenizer's chat template. After the prompt is ready, generation can be performed like this: ```py inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt") outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150) ``` ### Inputs and outputs * **Input:** Text string, such as a question, a prompt, or a document to be summarized. * **Output:** Generated English-language text in response to the input, such as an answer to a question, or a summary of a document. ## Model Card Authors [More Information Needed] ## Model Card Contact [More Information Needed]