Edit model card

FredNajjar/bigbird-QA-squad_v2.2

Fine-tuned google/bigbird-roberta-base model on the SQuAD 2.0 dataset for English extractive question answering.

Model Details

Training Hyperparameters

  • Learning Rate: 3e-05
  • Train Batch Size: 16
  • Eval Batch Size: 8
  • Seed: 42
  • Gradient Accumulation Steps: 8
  • Total Train Batch Size: 128
  • Optimizer: Adam (betas=(0.9,0.999), epsilon=1e-08)
  • LR Scheduler: Linear with 121 warmup steps
  • Number of Epochs: 3

Results on SQuAD 2.0

  • F1 Score: 81.39%
  • Exact Match: 77.82%

Usage

from transformers import AutoModelForQuestionAnswering, AutoTokenizer, pipeline
model_name = "FredNajjar/bigbird-QA-squad_v2.2"
nlp = pipeline('question-answering', model=model_name, tokenizer=model_name)
QA_input = {
        'question': 'Your question here',
    'context': 'Your context here'
}
res = nlp(QA_input)
  • Framework Versions:
    • Transformers: 4.34.0
    • Pytorch: 2.0.1+cu118
    • Datasets: 2.14.5
    • Tokenizers: 0.14.1

Limitations and Bias

This model inherits limitations and potential biases from the base BigBird model and the SQuAD 2.0 training data.

Contact

For inquiries, please reach out via LinkedIn.


Downloads last month
0
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train FredNajjar/bigbird-QA-squad_v2.2