---
license: apache-2.0
base_model: bert-base-cased
tags:
- generated_from_trainer
model-index:
- name: bert-finetuned-squad
  results: []
datasets:
- rajpurkar/squad
language:
- en
metrics:
- accuracy
- f1
- exact_match
library_name: transformers
pipeline_tag: question-answering
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->


# BERT Fine-Tuned for Question Answering (SQuAD)

## Model Description
This model is a fine-tuned version of [BERT-base-cased](https://huggingface.co./bert-base-cased), specifically optimized for the task of question answering. It was trained on the SQuAD (Stanford Question Answering Dataset) to understand and extract relevant information from a given context, based on a provided question. BERT is a transformer-based model that uses attention mechanisms to improve the contextual understanding of text, which makes it well-suited for question-answering tasks.

## Intended Uses & Limitations
**Intended Uses:**
- **Question Answering:** This model can be used to extract answers from a given context based on a specific question. It's suitable for applications such as chatbots, virtual assistants, and customer support systems where retrieving relevant information is crucial.
- **Information Retrieval:** Useful in scenarios requiring quick and accurate information extraction from large bodies of text.

**Limitations:**
- **Domain Adaptation:** The model may not perform well on domains that are significantly different from the training data (e.g., technical manuals, medical documents).
- **Context Size Limitation:** Due to the input length limit of BERT (512 tokens), the context must be relatively short, or it needs to be chunked appropriately.
- **Bias and Fairness:** The model may reflect biases present in the SQuAD dataset and its pretraining corpus, potentially affecting the impartiality of answers.

## How to Use
To use this model for question answering, you can utilize the Hugging Face `transformers` library. Here’s a Python code example:

```python
from transformers import pipeline

model_checkpoint = "Ashaduzzaman/bert-finetuned-squad"
question_answerer = pipeline("question-answering", model=model_checkpoint)

question = "What is the name of the architectures?"
context = """
🤗 Transformers (formerly known as pytorch-transformers and pytorch-pretrained-
and pytorch-nlp) provides general-purpose architectures (BERT, GPT-2, RoBERTa, XLM, DistilBert, XLNet…) for Natural
Language Understanding (NLU) and Natural Language Generation (NLG) with over 32+ pretrained models in 100+ languages and
with state-of-the-art performance on SQuAD, GLUE, AWS Glue, and other benchmarks.
"""

result = question_answerer(question=question, context=context)
print(result['answer'])
```

## Training and Evaluation Data
- **Dataset Used:** The model was fine-tuned on the SQuAD dataset, a benchmark dataset for training and evaluating question-answering models. SQuAD provides a collection of questions and corresponding context paragraphs, with labeled answers.

## Training Procedure
The model was trained using the Hugging Face `transformers` library with the following hyperparameters:
- **Learning Rate:** 2e-05
- **Training Batch Size:** 8
- **Evaluation Batch Size:** 8
- **Seed:** 42
- **Optimizer:** Adam with `betas=(0.9,0.999)` and `epsilon=1e-08`
- **Learning Rate Scheduler:** Linear
- **Number of Epochs:** 1
- **Mixed Precision Training:** Native AMP (Automatic Mixed Precision)

### Training Results
- **Final Training Loss:** 1.22
- **Exact Match (EM):** 79.99
- **F1 Score:** 87.55

## Evaluation
The model's performance was evaluated using standard SQuAD metrics, including Exact Match (EM) and F1 score. These metrics measure the model's ability to provide accurate and precise answers to the questions based on the context.

## Framework Versions
- **Transformers:** 4.42.4
- **PyTorch:** 2.3.1+cu121
- **Datasets:** 2.21.0
- **Tokenizers:** 0.19.1