metadata

license: apache-2.0
tags:
  - generated_from_trainer
model-index:
  - name: DistilGPT2-Beatles-Lyrics-finetuned
    results: []
widget:
  - text: Last night in Kiev the
    example_title: Kiev
  - text: It hasn't rained in weeks
    example_title: Rain

DistilGPT2-Beatles-Lyrics-finetuned

This model is a fine-tuned version of distilgpt2 on the Huggingartists - beatles dataset. It will complete an input prompt with Beatles-like text.

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-05
train_batch_size: 4
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 100
num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss
2.748	1.0	165	2.3732
2.4395	2.0	330	2.1938
2.2968	3.0	495	2.1118
2.2075	4.0	660	2.0721
2.1393	5.0	825	2.0571

Framework versions

Transformers 4.18.0
Pytorch 1.11.0+cu113
Datasets 2.2.0
Tokenizers 0.12.1