Update README.md

README.md

@@ -4,8 +4,15 @@ datasets:
 - somosnlp/es-inclusive-language
 language:
 - es
+metrics:
+- bleurt
+- sacrebleu
+pipeline_tag: text2text-generation
+tags:
+- social
 ---
-# es-inclusivo-translator
+
+# Model Card for Traductor Inclusivo
 
 This model is a fine-tuned version of [projecte-aina/aguila-7b](https://huggingface.co/projecte-aina/aguila-7b) on the dataset [somosnlp/es-inclusive-language](https://huggingface.co/datasets/somosnlp/es-inclusive-language).
 
@@ -13,32 +20,82 @@ Languages are powerful tools to communicate ideas, but their use is not impartia
 Spanish is not the exception to that. For instance, when we say “los alumnos” or “los ingenieros”, we are excluding women from those groups. Similarly, expressions such as “los gitanos” o “los musulmanes” perpetuate discrimination against these communities.
 
 In response to these linguistic challenges, this model offers a way to construct inclusive alternatives in accordance with official guidelines on inclusive language from various Spanish speaking countries. Its purpose is to provide grammatically correct and inclusive solutions to situations where our language choices might otherwise be exclusive. 
+By rectifying biases ingrained in language and fostering inclusivity, it combats discrimination, amplifies the visibility of marginalized groups, and contributes to the cultivation of a more inclusive and respectful society.
 This is a tool that contributes to the fifth of the Sustainable Development Goals: Achieve gender equality and empower all women and girls.
 
 The model works in such a way that, given an input text, it returns the original text rewritten using inclusive language.
 
-It achieves the following results on the evaluation set:
+It achieves the following results on the validation set:
 - Loss: 0.6030
 
-## Model description
-- **Developed by**: Andrés Martínez Fernández-Salguero ([andresmfs](https://huggingface.co/Andresmfs)), Imanuel Rozenberg (manu_20392), Gaia Quintana Fleitas (gaiaq), Josué Sauca (josue_sauca), Miguel López (wizmik12)
-- **Language(s)**: Spanish
-- **Fine-tuned from the model**: [projecte-aina/aguila-7b](https://huggingface.co/projecte-aina/aguila-7b)
-- **License**: cc-by-nc-sa-4.0
+## Model Details
 
-## Social Impact
-An inclusive translator holds significant social impact by promoting equity and representation within texts. By rectifying biases ingrained in language and fostering inclusivity, it combats discrimination, amplifies the visibility of marginalized groups, and contributes  to the cultivation of a more inclusive and respectful society.
-This is a tool that contributes to the fifth of the Sustainable Development Goals: Achieve gender equality and empower all women and girls.
+### Model Description
+
+- **Developed by:** Andrés Martínez Fernández-Salguero, Imanuel Rozenberg, Gaia Quintana Fleitas, Miguel López Pérez y Josué Sauca
+- **Funded by:** SomosNLP, HuggingFace
+- **Model type:** Language model, instruction tuned
+- **Language(s):** Spanish (`es-ES`, `es-AR`, `es-MX`, `es-CR`, `es-CL`)
+- **License:** cc-by-nc-sa-4.0
+- **Fine-tuned from model:** [projecte-aina/aguila-7b](https://huggingface.co/projecte-aina/aguila-7b)
+- **Dataset used:** [somosnlp/es-inclusive-language](https://huggingface.co/datasets/somosnlp/es-inclusive-language)
+
+### Model Sources
+
+- **Repository:** https://github.com/Andresmfs/Traductor_inclusivo
+- **Demo:** https://huggingface.co/spaces/somosnlp/es-inclusive-language-demo
+- **Video presentation:** https://www.youtube.com/watch?v=7rrNGJIXEHU
 
-## Intended uses & limitations
 
-### Intended uses
-The general uses of this model are texts adaptations to inclusive language.
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+The general uses of this model are adaptations of texts in Spanish to inclusive language.
 
 It can be used to adapt news, blogposts, emails and official documents among others.
 
-### How to use
-Here is how to use this model:
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+- Model has not been trained on long-complex texts.
+- Model has been trained mostly with sentences where the terms to be modified are at the beginning of the sentence.
+- Model returns only one translation option when several might also be adequate.
+- Possible small information omission on translation.
+- Possible forced use of the term "personas".
+- Model does not detect or modify hate speech.
+- Model has been trained on data mainly based on Spanish Inclusive Language Guidelines and may inherit any bias comming from the guidelines and institutions behind them. They are official and updated guidelines that should not contain strong biases.
+- Model may not work propperly on translation difficulties aside the list of difficulties present on [es-inclusive-language dataset](https://huggingface.co/datasets/somosnlp/es-inclusive-language)
+- Other biases coming from the train dataset [es-inclusive-language dataset](https://huggingface.co/datasets/somosnlp/es-inclusive-language) should be taken into account.
+
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+<!-- Example: Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. -->
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model in 16-bits.
 
 ```python
 from transformers import AutoTokenizer
@@ -160,31 +217,73 @@ input_text = 'Los alumnos atienden a sus profesores'
 print(translate_es_inclusivo(input_text))
 ```
 
-### Limitations
-- The model has not been trained on long-complex texts.
-- It has been trained mostly with sentences where the terms to be modified are at the beginning of the sentence.
-- It returns only one option.
-- The model does not detect or modify hate speech
+## Training Details
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
+Train, validation and test data splits can be found in [somosnlp/es-inclusive-language](https://huggingface.co/datasets/somosnlp/es-inclusive-language)
 
-## Training and evaluation data
+### Training Procedure
 
-Training and evaluation data can be found in [somosnlp/es-inclusive-language](https://huggingface.co/datasets/somosnlp/es-inclusive-language)
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-## Training procedure
+<!-- Detallar la técnica de entrenamiento utilizada y enlazar los scripts/notebooks. -->
 
-### Training hyperparameters
+For training we used QLoRA technique in 4-bits and rank 8
 
+Find the training script [here](https://github.com/Andresmfs/Traductor_inclusivo/tree/master)
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+#### Training Hyperparameters
+
+<!-- Enumerar los valores de los hiperparámetros de entrenamiento. -->
 The following hyperparameters were used during training:
-- learning_rate: 0.0001
-- train_batch_size: 8
-- eval_batch_size: 8
-- seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- num_epochs: 10
+- **learning_rate:** 0.0001
+- **train_batch_size:** 8
+- **eval_batch_size:** 8
+- **seed:** 42
+- **optimizer:** Adam with betas=(0.9,0.999) and epsilon=1e-08
+- **lr_scheduler_type:** linear
+- **num_epochs:** 10
+- **Training regime:** fp16 mixed precision
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+Here you can find the [validation set](https://huggingface.co/datasets/somosnlp/es-inclusive-language/viewer/default/validation) used during training.
+Here you can find the [test set](https://huggingface.co/datasets/somosnlp/es-inclusive-language/viewer/default/test) used for evaluating model errors.
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
 
-### Training results
+[More Information Needed]
+
+#### Metrics
+
+For test evaluation it has been used a weighted harmonic mean of metrics [bleurt](https://huggingface.co/spaces/evaluate-metric/bleurt) (60%) and [Sacrebleu](https://huggingface.co/spaces/evaluate-metric/sacrebleu) (40%).
+
+In _Sacrebleu_ metric grammatical correctness carries high weight compared to the actual words used, whereas in _Bleurt_ metric the actual words used have higher weight over grammatical correctness.
+Combining both metrics, we account for a grammatically correct prediction together with the use of the required specific words.
+
+### Results
+
+<!-- Enlazar aquí los scripts/notebooks de evaluación y especificar los resultados. -->
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
@@ -199,16 +298,101 @@ The following hyperparameters were used during training:
 | 0.3898        | 9.0   | 3618 | 0.6209          |
 | 0.3771        | 10.0  | 4020 | 0.6292          |
 
+On [this notebook](https://github.com/Andresmfs/Traductor_inclusivo/blob/master/Error%20analysis.ipynb) you can find the results of the test evaluation.
+
+We get an average score of 68.4 (measured with the above described metric). 
+Due to the existence of equivalent language formulas (these are inclusive language formulas that can be used indistinctly and the choice of a formula over the other is rather a stylistic decision than a language correctness decision) it is possible to argue that the real score of the model is higher.
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here. -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly. -->
+
+<!-- Rellenar la información de la lista y calcular las emisiones con la página mencionada. -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** Nvidia T4 medium (8 vCPU, 30 Gb RAM, 16 Gb VRAM)
+- **Hours used:** 3 hours
+- **Cloud Provider:** Google Cloud Platform
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+<!-- Esta sección es opcional porque seguramente ya habéis mencionado estos detalles más arriba, igualmente está bien incluirlos aquí de nuevo como bullet points a modo de resumen. -->
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
 
-### Framework versions
+[More Information Needed]
 
+#### Hardware
+
+<!-- Indicar el hardware utilizado, podéis agradecer aquí a quien lo patrocinó. -->
+Hardware used was Nvidia T4 medium (8 vCPU, 30 Gb RAM, 16 Gb VRAM) patrocinated by Hugging Face
+
+#### Software
 - Transformers 4.30.0
 - Pytorch 2.2.2+cu121
 - Datasets 2.18.0
 - Tokenizers 0.13.3
+- Peft
+
+## License
+<!-- Indicar bajo qué licencia se libera el modelo explicando, si no es apache 2.0, a qué se debe la licencia más restrictiva (i.e. herencia de las licencias del modelo pre-entrenado o de los datos utilizados). -->
+Creative Commons (cc-by-nc-sa-4.0)
+
+This kind of license is inherited from dataset used for training.
+
+## Citation
+
+**BibTeX:**
+
+```
+@software{AIGMJ2024TraductorInclusivo,
+  author = {Andrés Martínez Fernández-Salguero, Imanuel Rozenberg, Gaia Quintana Fleitas, Miguel López Pérez, Josué Sauca},
+  title = {TraductorInclusivo},
+  month = April,
+  year = 2024,
+  url = {https://huggingface.co/somosnlp/es-inclusivo-translator}
+}
+```
+
+- AIGMJ2024TraductorInclusivo
+- author: Andrés Martínez Fernández-Salguero, Imanuel Rozenberg, Gaia Quintana Fleitas, Miguel López Pérez, Josué Sauca
+- title: Traductor Inclusivo
+- year: 2024
+- url: https://huggingface.co/somosnlp/es-inclusivo-translator
+
+
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+## More Information
+
+<!-- Indicar aquí que el marco en el que se desarrolló el proyecto, en esta sección podéis incluir agradecimientos y más información sobre los miembros del equipo. Podéis adaptar el ejemplo a vuestro gusto. -->
+
+This project was developed during the [Hackathon #Somos600M](https://somosnlp.org/hackathon) organized by SomosNLP. The model was trained using GPUs sponsored by HuggingFace.
+
+**Team:** 
+- [**Andrés Martínez Fernández-Salguero**](https://huggingface.co/Andresmfs)
+- **Imanuel Rozenberg**
+- **Gaia Quintana Fleitas**
+- **Miguel López Pérez**
+- **Josué Sauca**
 
-### Hardware
-- Nvidia T4 medium (8 vCPU, 30Gb RAM, 16Gb VRAM)
+## Contact
 
-### Training time
-- 3 hours aprox
+- [**Andrés Martínez Fernández-Salguero**](www.linkedin.com/in/andrés-martínez-fernández-salguero-725674214) ([email protected])
+- [**Gaia Quintana Fleitas**](https://www.linkedin.com/in/gaiaquintana/) ([email protected])