|
--- |
|
library_name: transformers |
|
tags: |
|
- generated_from_keras_callback |
|
model-index: |
|
- name: DistilBERT-base-uncased-english-finetuned-squad |
|
results: [] |
|
datasets: |
|
- rajpurkar/squad |
|
language: |
|
- en |
|
base_model: |
|
- distilbert/distilbert-base-uncased |
|
pipeline_tag: question-answering |
|
--- |
|
|
|
|
|
# DistilBERT-base-uncased-english-finetuned-squad |
|
|
|
This model was finetuned on squad dataset. |
|
Use TFDistilBertForQuestionAnswering to import model. |
|
Requires DistilBertTokenizerFast to generate tokens that are accepted by this model. |
|
|
|
## Model description |
|
|
|
Base DistilBERT model finetuned using squad dataset for NLP tasks such as context based question answering. |
|
|
|
|
|
## Training procedure |
|
|
|
Trained for 3 epochs. |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- optimizer: Adam with learning_rate=5e-5 |
|
- training_precision: float32 |
|
|
|
### Training results |
|
|
|
Loss on final epoch: 0.6417 & validation loss: 1.2772 |
|
evaluation yet to be done. |
|
|
|
### Framework versions |
|
|
|
- Transformers 4.44.2 |
|
- TensorFlow 2.17.0 |
|
- Datasets 3.0.0 |
|
- Tokenizers 0.19.1 |
