README.md

---
library_name: transformers
metrics:
- bleu : 0.67
- chrf : 0.73
---

# Model Card for Model ID

This is the Gemma-2b-IT model fine-tuned for the Python code generation task.


## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Mohammed Ashraf
- **Model type:** google/gemma-2b
- **Finetuned from model [optional]:** google/gemma-2b-it


## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
Use this model to generate Python code.



### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
This model is trained on very basic Python code, so it might not be able to handle complex code.


## How to Get Started with the Model

Use the code below to get started with the model.

```python
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "mrSoul7766/gemma-2b-it-python-code-gen-adapter"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

text = """<start_of_turn>how to covert json to dataframe.<end_of_turn>
<start_of_turn>model"""

#device = "cuda:0"

inputs = tokenizer(text, return_tensors="pt")


outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```


## Training Details


### Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

**Fine-tuning Data:** [flytech/python-codes-25k](https://huggingface.co./datasets/flytech/python-codes-25k/viewer/default/train?p=2&row=294)


### Training Procedure 

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->


#### Training Hyperparameters

- **Training regime:** fp16 <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
- **learning_rate:** 2e-4

## Evaluation
<!-- This section describes the evaluation protocols and provides the results. -->


### Testing Data & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[iamtarun/python_code_instructions_18k_alpaca](https://huggingface.co./datasets/iamtarun/python_code_instructions_18k_alpaca?row=44)


#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

 - **chrf:** 0.73
 - **codebleu:** 0.67
 - **codebleu_ngram:** 0.53

### Results

```python
import json
import pandas as pd

# Load the JSON data
with open('data.json', 'r') as f:
    data = json.load(f)

# Create the DataFrame
df = pd.DataFrame(data)
```

#### Summary



## Environmental Impact

<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->

Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).

- **Hardware Type:** H100
- **Hours used:** 30 minutes 
- **Cloud Provider:** Google-cloud


## Technical Specifications [optional]

### Model Architecture and Objective

#### Hardware

- **Hardware Type:** H100
- **Hours used:** 30 minutes 
- **Cloud Provider:** Google-cloud

#### Software

- bitsandbytes==0.42.0
- peft==0.8.2
- trl==0.7.10
- accelerate==0.27.1
- datasets==2.17.0
- transformers==4.38.0