---
license: openrail
datasets:
- bigcode/the-stack-dedup
library_name: transformers
tags:
- code_generation
- R programming
- sas
- santacoder
---

# Statscoder

This model is a fine-tuned version of [bigcode/santacoder](https://huggingface.co./bigcode/santacoder) on `R` and `SAS` language repositories in [the stack ](https://huggingface.co./datasets/bigcode/the-stack-dedup) dataset.


## Training procedure

The model was finetuned using the code adapted from [loubnabnl/santacoder-finetuning](https://github.com/loubnabnl/santacoder-finetuning). Adapted to handle multiple subsets of datasets and it is [here](https://github.com/infinitylogesh/santacoder-finetuning).

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 4
- optimizer: adafactor
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 1600
- seq_length: 1024
- no_fp16