---
library_name: llama.cpp
license: gemma
widget:
- text: '<start_of_turn>user

    How does the brain work?<end_of_turn>

    <start_of_turn>model

    '
inference:
  parameters:
    max_new_tokens: 200
extra_gated_heading: Access Gemma on Hugging Face
extra_gated_prompt: To access Gemma on Hugging Face, you’re required to review and
  agree to Google’s usage license. To do this, please ensure you’re logged-in to Hugging
  Face and click below. Requests are processed immediately.
extra_gated_button_content: Acknowledge license
---

<hr>

# Llama.cpp imatrix quantizations of google/gemma-2-2b-it-GGUF

<img src="https://cdn-uploads.huggingface.co/production/uploads/646410e04bf9122922289dc7/-03oAOPVN1nZjp6-2EIxD.png" alt="gemma" width="60%"/>

Using llama.cpp commit [268c566](https://github.com/ggerganov/llama.cpp/commit/398ede5efeb07b9adf9fbda7ea63f630d476a792) for quantization.

Original model: https://huggingface.co./google/gemma-2-2b-it

All quants were made using the imatrix option and Bartowski's [calibration file](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8).

<hr><br>

# Gemma Model Card

**Model Page**: [Gemma](https://ai.google.dev/gemma/docs)

This model card corresponds to the 2b instruct version the Gemma 2 model in GGUF Format. The weights here are **float32**.

> [!IMPORTANT]
>
> In llama.cpp, and other related tools such as Ollama and LM Studio, please make sure that you have these flags set correctly, especially **`repeat-penalty`**. Georgi Gerganov (llama.cpp's author) shared his experience in https://huggingface.co./google/gemma-2b-it/discussions/38#65d2b14adb51f7c160769fa1.

You can also visit the model card of the [2B pretrained v2 model GGUF](https://huggingface.co./google/gemma-2b-v2-GGUF). 

**Resources and Technical Documentation**:

* [Responsible Generative AI Toolkit](https://ai.google.dev/responsible)
* [Gemma on Kaggle](https://www.kaggle.com/models/google/gemma)
* [Gemma on Vertex Model Garden](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/335?version=gemma-2b-it-gg-hf)

**Terms of Use**: [Terms](https://www.kaggle.com/models/google/gemma/license/consent/verify/huggingface?returnModelRepoId=google/gemma-2-2b-it-GGUF)

**Authors**: Google

## Model Information

Summary description and brief definition of inputs and outputs.

### Description

Gemma is a family of lightweight, state-of-the-art open models from Google,
built from the same research and technology used to create the Gemini models.
They are text-to-text, decoder-only large language models, available in English,
with open weights, pre-trained variants, and instruction-tuned variants. Gemma
models are well-suited for a variety of text generation tasks, including
question answering, summarization, and reasoning. Their relatively small size
makes it possible to deploy them in environments with limited resources such as
a laptop, desktop or your own cloud infrastructure, democratizing access to
state of the art AI models and helping foster innovation for everyone.