Update README.md

README.md

@@ -25,7 +25,7 @@ widget:
 # Model Card for gemma-2b-it-peru-legal-es-V2 ⚖️
 
 <p align="center">
-  <img src="https://cdn-uploads.huggingface.co/production/uploads/64461026e1fd8d65b27e6187/bUbvGpmsKC7YsJs4aBt7T.jpeg" alt="Model Illustration" width="500">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/64461026e1fd8d65b27e6187/bUbvGpmsKC7YsJs4aBt7T.jpeg" alt="Model Illustration" width="300">
 </p>
 
 ## Gemma-2B-IT-Peru-Legal-ES
@@ -34,69 +34,39 @@ The Gemma-2B-IT-Peru-Legal-ES model is a state-of-the-art language model fine-tu
 
 ## Table of Contents
 
-- [Model Card for Gemma-2B-IT-Peru-Legal-ES 🇵🇪](#model-card-for-gemma-2b-it-peru-legal-es-)
-  * [Gemma-2B-IT-Peru-Legal-ES ⚖️](#gemma-2b-it-peru-legal-es)
+- [Gemma-2B-IT-Peru-Legal-ES ⚖️](#gemma-2b-it-peru-legal-es)
     + [Model Description 📘](#model-description)
-  * [Usage 🛠️](#usage-)
-    + [Direct Use 🎯](#direct-use-)
-    + [Downstream Use 🔄](#downstream-use-)
-    + [Out-of-Scope Use 🚫](#out-of-scope-use-)
-  * [Bias, Risks, and Limitations ⚠️](#bias-risks-and-limitations-)
+  * [Finetuning Progress 🛠️](#finetuning-progress)
     + [Recommendations 📝](#recommendations-)
-  * [How to Get Started with the Model 🚀](#how-to-get-started-with-the-model-)
-  * [Training Dataset 🧠](#training-dataset-)
-    + [Environment and Libraries 🖥️](#environment-and-libraries-)
-    + [QLoRA Configuration 🧮](#qlora-configuration-)
-    + [Model Merging and Saving 💾](#model-merging-and-saving-)
-  * [Logging with Wandb 📊](#logging-with-wandb-)
+  * [How to Get Started with the Model 🚀](#how-to-get-started-with-the-model)
+  * [Training Dataset 🧠](#training-dataset)
+  * [Finetuning Progress 🤖](#finetuning-progress)
+  * 
+    + [Environment and Libraries 🖥️](#environment-and-libraries)
+    + [QLoRA Configuration 🧮](#qlora-configuration)
+    + [Model Merging and Saving 💾](#model-merging-and-saving)
+  * [Logging with Wandb 📊](#logging-with-wandb)
+  * [Impacto Ambiental 🌳](#impacto-ambiental)
 
 
 
-## Model Details 📈
-
-### Model Description 📘
+## Model Description
 
 Gemma-2b-it-peru-legal-es-V2  is a contextual legal language model designed to provide personalized legal advice in Spanish based on Peruvian legal texts. Leveraging advanced techniques like LoRA, the model offers accurate and contextually relevant responses to legal queries, covering various aspects of Peruvian law and regulation. Whether it's understanding rights, navigating legal procedures, or interpreting statutes, gemma-2b-it-peru-legal-es-V2 empowers users with comprehensive and reliable legal guidance tailored to the Peruvian legal landscape.
 
-- **Developed by:** []()
+- **Developed by:** [Alonso Quispe](https://huggingface.co/daqc)
 - **Model type:** Causal Language Model, specially fine-tuned with LoRA for the distinct domain of Peruvian law and regulation.
 - **Language(s) (NLP):** Spanish, tailored for the legal and regulatory context of Peru.
 - **License:** Apache License. This open-source license ensures that the model can be freely used, modified, and distributed. Please check the model's page on Hugging Face for specific licensing details.
-- **Base model [optional]:** Derived from Google's Gemma 2b it model, utilizing versions such as `gemma-2b-it` for comprehensive training.
-
-### Fintunineting progresss 📉
-<p align="center">
-  <img src="https://cdn-uploads.huggingface.co/production/uploads/64461026e1fd8d65b27e6187/N8VAkUIuWK0vgYZRlmwew.png" alt="Loss Function Graph" width="900">
-</p>
-
-
----
-
-## Usage 🛠️
-
-The PeruLegalLLM model is designed to enhance understanding and application of Colombian Aeronautical Regulations (RAC) through natural language processing. It's tailored for professionals and enthusiasts in the aviation industry, regulatory agencies, legal experts, and AI researchers with an interest in domain-specific language model applications.
-
-### Direct Use 🎯
-
-The model can be directly employed to generate text, answer questions, and provide insights related to Colombian Aeronautical Regulations without further fine-tuning. It's ideal for creating educational content, simplifying legal language, and assisting in regulatory compliance efforts.
-
-### Downstream Use [optional] 🔄
-
-When finely-tuned, gemma-2b-it-peru-legal-es-V2 can be integrated into larger systems for automated compliance checks, document summarization, and even training simulators for pilots and air traffic controllers, offering a deeper, contextual understanding of regulations.
-
-### Out-of-Scope Use 🚫
-
-Misuse includes any application that promotes unethical practices, misinterprets aviation law, or uses the model for malicious purposes. The model is not designed for navigational purposes or to replace professional legal advice.
+- **Base model:** google/gemma-2b-it`
 
-## Bias, Risks, and Limitations ⚠️
 
-The model, while powerful, has limitations inherent to AI, including biases present in the training data. It may not cover all nuances of aviation regulations outside of Colombia or adapt to changes in law without updates.
 
-### Recommendations 📝
+## Recommendations
 
 Users should verify model outputs against current regulations and consult with professionals for critical applications. Awareness of the model's scope and limitations is crucial for effective use.
 
-## How to Get Started with the Model 🚀
+# How to Get Started with the Model
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
@@ -113,13 +83,22 @@ output = model.generate(input_ids, max_length=100, num_return_sequences=1, tempe
 print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
 
-## Training Dataset 🧠
+## Training Dataset
 
 The Gemma-2B-IT-Peru-Legal-ES-V2 model was fine-tuned exclusively on the "[Constitución Política del Perú](https://huggingface.co/datasets/daqc/constitucion-politica-del-peru-1993-qa) dataset available through Hugging Face Datasets. This dataset serves as a rich source of questions and answers pertaining to the legal framework outlined in the Peruvian Constitution.
 The dataset encompasses a wide range of topics and provisions within the Peruvian Constitution, providing comprehensive coverage of constitutional principles, rights, and legal interpretations.
 
 
-### Environment and Libraries 🖥️
+## Finetuning Progress
+<p align="center">
+  <img src="https://cdn-uploads.huggingface.co/production/uploads/64461026e1fd8d65b27e6187/N8VAkUIuWK0vgYZRlmwew.png" alt="Loss Function Graph" width="900">
+</p>
+
+
+---
+
+
+## Environment and Libraries
 
 The training process was executed within a Python environment, utilizing essential libraries to facilitate various tasks:
 - **transformers:** Used for loading and fine-tuning the model, providing a seamless interface for working with pre-trained language models.
@@ -128,13 +107,13 @@ The training process was executed within a Python environment, utilizing essenti
 - **peft:** Integrated for applying Low-Rank Adaptation (LoRA) techniques to the model, enabling efficient adaptation to specialized domains without compromising performance.
 - **qlora:** Additionally utilized for further model adaptation, enhancing its ability to comprehend and generate responses specific to the legal context of the "Constitucióm Política del Perú" dataset.
 
-### QLoRA Configuration 🧮
+## QLoRA Configuration
 
 QLoRA (Quantization LoRA) was employed to optimize the model's computational efficiency and memory footprint while preserving its accuracy. Two configurations were utilized:
 
 - **BitsAndBytesConfig:** This configuration enabled the model to load in 4-bit quantization, leveraging the nf4 quantization type with a torch.bfloat16 compute data type for enhanced efficiency during inference.
     ```python
-    bnb_config = BitsAndBytesConfig(
+        bnb_config = BitsAndBytesConfig(
         load_in_4bit=True,
         bnb_4bit_quant_type="nf4",
         bnb_4bit_compute_dtype=torch.bfloat16,
@@ -149,8 +128,9 @@ QLoRA (Quantization LoRA) was employed to optimize the model's computational eff
     - **bias:** Set to "none" to exclude bias terms from adaptation, simplifying the model architecture.
     - **lora_dropout:** Reduced to 0.025 from the default 0.05, controlling the dropout rate during adaptation.
     - **task_type:** Configured as "CAUSAL_LM" to indicate the task type of the language model.
+    - 
     ```python
-    config = LoraConfig(
+        config = LoraConfig(
         r=8,
         lora_alpha=16,
         target_modules=['q_proj', 'k_proj', 'v_proj', 'o_proj'],
@@ -163,7 +143,7 @@ QLoRA (Quantization LoRA) was employed to optimize the model's computational eff
 These configurations were crucial for optimizing the model's performance and resource utilization during training and inference, ensuring efficient deployment.
 
 
-### Model Merging and Saving 💾
+## Model Merging and Saving 💾
 
 After fine-tuning, the LoRA-adjusted weights were merged back with the base Gemma model to create the final gemma-2b-it-peru-legal-es-V2. The model was then saved and made available through Hugging Face for easy access and further development.
 
@@ -173,9 +153,20 @@ After fine-tuning, the LoRA-adjusted weights were merged back with the base Gemm
 During the training process, Wandb (Weights & Biases) was used for comprehensive logging and visualization of key metrics. Wandb's powerful tracking capabilities enabled real-time monitoring of training progress, evaluation metrics, and model performance. Through interactive dashboards and visualizations, Wandb facilitated deep insights into the training dynamics, allowing for efficient model optimization and debugging. This logging integration with Wandb enhances transparency, reproducibility, and collaboration among researchers and practitioners.
 
 
+
+## Impacto Ambiental
+
+The training of `gemma-2b-it-peru-legal-es-V2` was conducted optimizing the computational expenditure required.
+
+- **Hardware Type:** A10G-24GB
+- **Hours Utilized:** Approximately --- hours
+- **Energy Consumption:** Approximately --- kWh
+- **Estimated CO2 Emissions:** Approximately --- kg
+
+
 ## To-Do List
 
-### Dataset Generation, Human Feedback Review with Argilla, and Finetuning with the Following Legal Texts:
+## Dataset Generation, Human Feedback Review with Argilla, and Finetuning with the Following Legal Texts:
 
 - [x] Constitution of Peru
 - [ ] Penal Code
@@ -195,7 +186,7 @@ During the training process, Wandb (Weights & Biases) was used for comprehensive
 - [ ] Tax Code (TUO)
 - [ ] Criminal Procedure Code
 
-### License
 
-This project is distributed under the Apache 2.0 license.
+## License
 
+This project is distributed under the Apache 2.0 license.