| license: apache-2.0 | |
| base_model: google/flan-t5-small | |
| tags: | |
| - generated_from_keras_callback | |
| model-index: | |
| - name: Flan | |
| results: [] | |
| <!-- This model card has been generated automatically according to the information Keras had access to. You should | |
| probably proofread and complete it, then remove this comment. --> | |
| # Flan | |
| This model is a fine-tuned version of [google/flan-t5-small](https://huggingface.co./google/flan-t5-small) on an unknown dataset. | |
| It achieves the following results on the evaluation set: | |
| - Train Loss: 1.1764 | |
| - Validation Loss: 0.9237 | |
| - Epoch: 24 | |
| ## Model description | |
| More information needed | |
| ## Intended uses & limitations | |
| More information needed | |
| ## Training and evaluation data | |
| More information needed | |
| ## Training procedure | |
| ### Training hyperparameters | |
| The following hyperparameters were used during training: | |
| - optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 2e-05, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False} | |
| - training_precision: float32 | |
| ### Training results | |
| | Train Loss | Validation Loss | Epoch | | |
| |:----------:|:---------------:|:-----:| | |
| | 1.6824 | 1.3619 | 0 | | |
| | 1.5733 | 1.3030 | 1 | | |
| | 1.5199 | 1.2730 | 2 | | |
| | 1.4833 | 1.2379 | 3 | | |
| | 1.4591 | 1.2201 | 4 | | |
| | 1.4278 | 1.1945 | 5 | | |
| | 1.4101 | 1.1755 | 6 | | |
| | 1.3870 | 1.1561 | 7 | | |
| | 1.3740 | 1.1366 | 8 | | |
| | 1.3559 | 1.1163 | 9 | | |
| | 1.3420 | 1.1007 | 10 | | |
| | 1.3265 | 1.0871 | 11 | | |
| | 1.3082 | 1.0708 | 12 | | |
| | 1.2984 | 1.0586 | 13 | | |
| | 1.2832 | 1.0429 | 14 | | |
| | 1.2724 | 1.0305 | 15 | | |
| | 1.2634 | 1.0170 | 16 | | |
| | 1.2486 | 1.0090 | 17 | | |
| | 1.2375 | 0.9901 | 18 | | |
| | 1.2259 | 0.9760 | 19 | | |
| | 1.2128 | 0.9659 | 20 | | |
| | 1.2038 | 0.9556 | 21 | | |
| | 1.1965 | 0.9446 | 22 | | |
| | 1.1833 | 0.9369 | 23 | | |
| | 1.1764 | 0.9237 | 24 | | |
| ### Framework versions | |
| - Transformers 4.32.0 | |
| - TensorFlow 2.13.0 | |
| - Datasets 2.14.4 | |
| - Tokenizers 0.12.1 | |