---
license: apache-2.0
tags:
- generated_from_trainer
model-index:
- name: codeparrot-small-custom-functions-dataset-python
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# codeparrot-small-custom-functions-dataset-python

This model is a fine-tuned version of [codeparrot/codeparrot-small](https://huggingface.co./codeparrot/codeparrot-small) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.4934

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5

### Training results

| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:-----:|:----:|:---------------:|
| 1.2357        | 0.12  | 1    | 0.9450          |
| 1.0585        | 0.25  | 2    | 0.8835          |
| 0.9853        | 0.38  | 3    | 0.8324          |
| 0.9877        | 0.5   | 4    | 0.7932          |
| 1.0075        | 0.62  | 5    | 0.7623          |
| 0.8744        | 0.75  | 6    | 0.7335          |
| 0.8627        | 0.88  | 7    | 0.7091          |
| 0.8615        | 1.0   | 8    | 0.6893          |
| 0.8016        | 1.12  | 9    | 0.6706          |
| 0.7458        | 1.25  | 10   | 0.6540          |
| 0.7335        | 1.38  | 11   | 0.6378          |
| 0.7754        | 1.5   | 12   | 0.6243          |
| 0.7684        | 1.62  | 13   | 0.6107          |
| 0.7335        | 1.75  | 14   | 0.5985          |
| 0.7418        | 1.88  | 15   | 0.5878          |
| 0.5644        | 2.0   | 16   | 0.5792          |
| 0.6464        | 2.12  | 17   | 0.5711          |
| 0.7304        | 2.25  | 18   | 0.5622          |
| 0.6848        | 2.38  | 19   | 0.5545          |
| 0.7221        | 2.5   | 20   | 0.5480          |
| 0.6053        | 2.62  | 21   | 0.5420          |
| 0.6168        | 2.75  | 22   | 0.5370          |
| 0.5823        | 2.88  | 23   | 0.5323          |
| 0.7512        | 3.0   | 24   | 0.5279          |
| 0.5745        | 3.12  | 25   | 0.5241          |
| 0.674         | 3.25  | 26   | 0.5202          |
| 0.5441        | 3.38  | 27   | 0.5167          |
| 0.6236        | 3.5   | 28   | 0.5134          |
| 0.6006        | 3.62  | 29   | 0.5102          |
| 0.6279        | 3.75  | 30   | 0.5072          |
| 0.6257        | 3.88  | 31   | 0.5046          |
| 0.5408        | 4.0   | 32   | 0.5025          |
| 0.5855        | 4.12  | 33   | 0.5004          |
| 0.594         | 4.25  | 34   | 0.4984          |
| 0.5985        | 4.38  | 35   | 0.4970          |
| 0.6279        | 4.5   | 36   | 0.4958          |
| 0.6151        | 4.62  | 37   | 0.4949          |
| 0.5629        | 4.75  | 38   | 0.4942          |
| 0.5148        | 4.88  | 39   | 0.4937          |
| 0.4823        | 5.0   | 40   | 0.4934          |


### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.12.0
- Tokenizers 0.13.3