fine-tuned-flan-t5-20-epochs-wang-lab

This model is a fine-tuned version of google/flan-t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7769
• Rouge1: 0.2685
• Rouge2: 0.0913
• Rougel: 0.2316
• Rougelsum: 0.2318
• Gen Len: 19.51

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	301	0.8814	0.2192	0.0734	0.1907	0.1911	19.4
1.4237	2.0	602	0.8212	0.2283	0.0703	0.1962	0.1977	19.39
1.4237	3.0	903	0.7864	0.2324	0.0656	0.1989	0.201	19.35
0.8542	4.0	1204	0.7753	0.2489	0.0789	0.2142	0.2156	19.27
0.7603	5.0	1505	0.7662	0.2458	0.0749	0.2119	0.2131	19.29
0.7603	6.0	1806	0.7608	0.2512	0.0896	0.2203	0.2217	19.48
0.699	7.0	2107	0.7611	0.259	0.0836	0.23	0.2318	19.52
0.699	8.0	2408	0.7566	0.264	0.087	0.2304	0.2312	19.43
0.646	9.0	2709	0.7558	0.2815	0.0947	0.2447	0.2453	19.43
0.6106	10.0	3010	0.7585	0.2692	0.0958	0.2377	0.2374	19.5
0.6106	11.0	3311	0.7611	0.2635	0.0997	0.2283	0.2278	19.5
0.5944	12.0	3612	0.7639	0.2785	0.0928	0.24	0.2403	19.46
0.5944	13.0	3913	0.7612	0.2648	0.0886	0.231	0.2318	19.38
0.5671	14.0	4214	0.7636	0.2786	0.0934	0.2431	0.2428	19.48
0.5387	15.0	4515	0.7656	0.2834	0.0938	0.2442	0.2432	19.47
0.5387	16.0	4816	0.7717	0.278	0.0927	0.2408	0.2403	19.48
0.5349	17.0	5117	0.7725	0.2733	0.0912	0.2367	0.2368	19.47
0.5349	18.0	5418	0.7761	0.2699	0.087	0.2339	0.2332	19.5
0.5182	19.0	5719	0.7786	0.2694	0.0874	0.2333	0.2336	19.5
0.5141	20.0	6020	0.7769	0.2685	0.0913	0.2316	0.2318	19.51

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0