base_model:
- dbmdz/bert-base-italian-xxl-uncased
datasets:
- sapienzanlp/ami
language:
- it
library_name: transformers
license: cc-by-nc-sa-4.0
metrics:
- accuracy
- f1
- recall
- precision
pipeline_tag: text-classification
tags:
- misogyny
- italian
model-index:
- name: misogyny-detection-it
results:
- task:
type: classification
dataset:
name: sapienzanlp/ami
type: text-classification
metrics:
- type: loss
value: 0.2205
name: test_loss
verified: false
- type: accuracy
value: 0.9412
name: test_accuracy
verified: false
- type: f1
value: 0.942
name: test_f1
verified: false
- type: precision
value: 0.9291
name: test_precision
verified: false
- type: recall
value: 0.9553
name: test_recall
verified: false
- type: runtime
value: 13.0069
name: test_runtime
verified: false
- type: samples_per_second
value: 223.573
name: test_samples_per_second
verified: false
- type: steps_per_second
value: 6.996
name: test_steps_per_second
verified: false
- type: epoch
value: 5
name: epoch
verified: false
Model Card for Misogyny Detection in Italian
This model is a fine-tuned version of dbmdz/bert-base-italian-xxl-uncased for the task of misogyny detection in Italian text. It identifies whether a given text contains misogynistic content (label 1) or not (label 0). The model has been trained and evaluated on the AMI (Automatic Misogyny Identification) dataset.
Model Details
Model Description
- Developed by: Lorenzo Maiuri
- Funded by: No funds
- Shared by: Lorenzo Maiuri
- Model type: Text Classification
- Language(s): Italian (
it) - License: CC BY-NC-SA 4.0
- Fine-tuned from model: dbmdz/bert-base-italian-xxl-uncased
This model is specifically designed for detecting misogynistic content in Italian, making it useful for tasks in moderation, social media analysis, or sociolinguistic studies.
Model Sources
- Repository: Hugging Face Model Repository
- Dataset: AMI (Automatic Misogyny Identification)
- Kaggle Notebook: Link to Kaggle Notebook
- Demo: Misogyny Detection IT Space
Uses
Try It Out
You can try this model interactively using the Misogyny Detection IT Space.
Simply enter a text prompt, and the model will classify it as 'Misogynistic' or 'Non-Misogynistic' along with a confidence score
Direct Use
This model can be used as-is for binary text classification to detect misogyny in Italian. For example:
from transformers import pipeline
classifier = pipeline("text-classification", model="maiurilorenzo/misogyny-detection-it")
output = classifier("Questo è un esempio di testo misogino.")
print(output)
Downstream Use
The model can be fine-tuned further on related datasets for similar tasks, such as hate speech detection, sentiment analysis, or offensive language detection.
Out-of-Scope Use
- The model is not intended for use in tasks outside of text classification.
- Avoid applying the model to non-Italian texts, as it may produce unreliable results.
- Misuse for harmful, malicious, or discriminatory purposes is strictly prohibited.
Bias, Risks, and Limitations
The model inherits potential biases present in the AMI dataset. It may overfit to linguistic patterns commonly associated with misogyny in the training data and fail to generalize to less explicit forms of misogyny or more nuanced cultural contexts.
Recommendations
- Use the model in conjunction with human moderation for critical tasks.
- Regularly evaluate the model on updated or domain-specific datasets to ensure continued accuracy and fairness.
How to Get Started with the Model
Use the code below to get started with the model:
from transformers import pipeline
classifier = pipeline("text-classification", model="maiurilorenzo/misogyny-detection-it")
output = classifier("Esempio di testo italiano.")
print(output)
Training Details
Training Data
The model was fine-tuned using the AMI (Automatic Misogyny Identification) dataset, which contains labeled examples of misogynistic and non-misogynistic texts in Italian.
- Dataset license: CC BY-NC-SA 4.0
- The training set was balanced by splitting misogynistic and non-misogynistic examples into training and validation sets.
Training Procedure
- Kaggle Notebook: Link to Kaggle Notebook
Preprocessing
The text was tokenized using the BERT tokenizer, with a maximum sequence length of 128 tokens. Labels were mapped to the labels field as required by the Transformers library.
Training Hyperparameters
- Learning Rate: 2e-5
- Batch Size: 32
- Epochs: 5
- Evaluation Strategy: Per epoch
- Metric for Best Model: F1-score
- Optimizer: AdamW with weight decay 0.01
Speeds, Sizes, Times
Total Training Time: Approximately 15 minutes Hardware Used: RTX 2060
Evaluation
Testing Data, Factors & Metrics
Testing Data
The model was evaluated on the test split of the AMI dataset, which is balanced and contains examples of both misogynistic and non-misogynistic content.
Factors
The evaluation considers factors such as:
- Lexical variations
- Explicit vs. implicit misogyny
- Variations in Italian regional language
Metrics
The following metrics were computed for evaluation:
- Accuracy
- F1-score
- Precision
- Recall
Results
- Accuracy: 0.9412
- F1-score: 0.9420
- Precision: 0.9291
- Recall: 0.9553
Summary
The model achieves strong performance on explicit misogyny detection, with potential for improvement in detecting more subtle or implicit forms of misogyny.
Environmental Impact
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
- Hardware Type: Tesla P100
- Hours used: 0.2
- Cloud Provider: Kaggle
- Carbon Emitted: 0.03
Citation
If you use this model, please cite it as follows:
@misc{misogyny-detection-it,
author = {Lorenzo Maiuri},
title = {Misogyny Detection in Italian},
year = {2024},
publisher = {Hugging Face Hub},
license = {CC BY-NC-SA 4.0}
}