library_name: transformers
license: apache-2.0
datasets:
- cfilt/iitb-english-hindi
language:
- en
- hi
pipeline_tag: translation
Model Card for Model ID
Model Details
Model Description
This model is a fine-tuned version of the GEMMA 2B multilingual transformer, specifically optimized for translating text from English to Hindi. It leverages the capabilities of the original GEMMA architecture to provide accurate and efficient translations.
-Model Name: Gemma-2b-mt-Hindi-Fintuned -Model Type: Language Translation Model -Base Model: Gemma-2b -Task: English to Hindi Translation -Framework: Transformers
Model Sources [optional]
Uses
Direct Use
This model can be directly used for translating English text to Hindi. It is suitable for various applications such as: -Localization of content -Cross-lingual communication -Educational tools for language learning -Multilingual content creation
Downstream Use [optional]
The model can be integrated into larger systems or applications that require English to Hindi translation capabilities, such as: -Machine translation services -Multilingual chatbots -Content management systems for multilingual websites
[More Information Needed]
Out-of-Scope Use
Bias, Risks, and Limitations
-The model may struggle with idiomatic expressions or culturally specific content. -There might be potential biases in the training data that could affect translation quality. -The model's performance on specialized or technical content may vary. -It may have limitations in handling complex grammatical structures or maintaining context in longer texts.
Recommendations
-It's recommended to use the model in conjunction with human translators for high-stakes or nuanced translations. -Regular evaluation and fine-tuning with diverse and representative data can help mitigate biases and improve performance.
How to Get Started with the Model
Use the code below to get started with the model:
from transformers import AutoTokenizer, AutoModelForCausalLM
#Load the model and tokenizer tokenizer = AutoTokenizer.from_pretrained("Satwik11/gemma-2b-mt-Hindi-Fintuned") model = AutoModelForCausalLM.from_pretrained("Satwik11/gemma-2b-mt-Hindi-Fintuned")
def generate_translation(prompt, max_length=90): # Prepare the input inputs = tokenizer(prompt, return_tensors='pt')
# Generate the translation
outputs = model.generate(**inputs, max_length=max_length)
# Decode the generated output
translated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
return translated_text
#Test the model with some example sentences test_sentences = [ "Today is August 19.The maximum temperature is 70 degrees Fahrenheit" ]
for sentence in test_sentences: prompt = f"Translate the following English text to Hindi: {sentence}" translation = generate_translation(prompt) print(translation)
Training Details
Training Data
The model was fine-tuned on the cfilt/iitb-english-hindi dataset, which contains English-Hindi sentence pairs. For more details about the dataset, refer to the dataset card on Hugging Face.
Model Card Contact
For more information, please contact the model creators through the Hugging Face model repository: https://www.linkedin.com/in/satwik-sinha/