MT5-Small Fine-tuned on Arabic Question Answering

This model is a fine-tuned version of MT5-Small for question answering tasks in Arabic.

Training and evaluation data

The model was trained on the tydiqa-goldp dataset for Arabic.

Training procedure

The model was fine-tuned using the Hugging Face Transformers library.

How to use

You can use this model with the Transformers pipeline for question answering:

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "HozRifai/mt5-ar-qa-v0"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

def generate_answer(question, context, max_length=64):
    input_text = f"question: {question} context: {context}"
    inputs = tokenizer(input_text, return_tensors="pt", max_length=max_length, truncation=True, padding="max_length").to(device)
    
    outputs = model.generate(
        **inputs,
        max_length=max_length,
        num_beams=4,
        length_penalty=2.0,
        early_stopping=True
    )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

context = ""
question = ""

answer = generate_answer(question, context)
print("Answer is: ", answer)