ar5entum/marianMT_bi_dev_rom_tl

marianMT_hin_eng_cs

This model is a fine-tuned version of Helsinki-NLP/opus-mt-hi-en on ar5entum/hindi-english-roman-devnagiri-transliteration-corpus dataset. It achieves the following results on the evaluation set:

• Loss: 0.0947
• Bleu: 73.5282
• Gen Len: 40.8725

Model description

The model is specifically designed to transliterate Devnagiri and Roman text both ways trained on both English and Hindi in Devnagiri and Roman scripts.

from transformers import MarianMTModel, MarianTokenizer
import evaluate

class HinEngCS:
    def __init__(self, model_name='ar5entum/marianMT_bi_dev_rom_tl'):
        self.model_name = model_name
        self.tokenizer = MarianTokenizer.from_pretrained(model_name)
        self.model = MarianMTModel.from_pretrained(model_name)

    def predict(self, input_text):
        tokenized_text = self.tokenizer(input_text, return_tensors='pt')
        translated = self.model.generate(**tokenized_text)
        translated_text = self.tokenizer.decode(translated[0], skip_special_tokens=True)
        return translated_text
model = Transliterate()

devnagiri = [
    "यह अभिषेक जल, इक्षुरस, दुध, चावल का आटा, लाल चंदन, हल्दी, अष्टगंध, चंदन चुरा, चार कलश, केसर वृष्टि, आरती, सुगंधित कलश, महाशांतिधारा एवं महाअर्घ्य के साथ भगवान नेमिनाथ को समर्पित किया जाता है।",
    "कुछ ने कहा ये चांद है कुछ ने कहा चेहरा तेरा"
    ]
roman = [
    "yah abhishek jal, ikshuras, dudh, chaval ka ataa, laal chandan, haldi, ashtagandh, chandan chura, char kalash, kesar vrishti, aarti, sugandhit kalash, mahashantidhara evam mahaarghya ke saath bhagvan Neminath ko samarpit kiya jata hai.",
    "kuch ne kaha ye chand hai kuch ne kaha chehra ter"
    ]

import time
start = time.time()

predictions = [model.predict('[dev] ' + d) for d in devnagiri]
end = time.time()
print("TIME: ", end-start)
for i in range(len(devnagiri)):
    print("‾‾‾‾‾‾‾‾‾‾‾‾")
    print("Input text:\t", devnagiri[i])
    print("Prediction:\t", predictions[i])
    print("Ground Truth:\t", roman[i])
bleu = evaluate.load("bleu")
results = bleu.compute(predictions=predictions, references=roman)
print(results)
predictions = [model.predict('[rom] ' + d) for d in roman]
end = time.time()
print("TIME: ", end-start)
for i in range(len(roman)):
    print("‾‾‾‾‾‾‾‾‾‾‾‾")
    print("Input text:\t", roman[i])
    print("Prediction:\t", predictions[i])
    print("Ground Truth:\t", devnagiri[i])
bleu = evaluate.load("bleu")
results = bleu.compute(predictions=predictions, references=devnagiri)
print(results)

# TIME:  1.8382132053375244
# ‾‾‾‾‾‾‾‾‾‾‾‾
# Input text:	 यह अभिषेक जल, इक्षुरस, दुध, चावल का आटा, लाल चंदन, हल्दी, अष्टगंध, चंदन चुरा, चार कलश, केसर वृष्टि, आरती, सुगंधित कलश, महाशांतिधारा एवं महाअर्घ्य के साथ भगवान नेमिनाथ को समर्पित किया जाता है।
# Prediction:	 yah abhishek jal, ikshuras, dudh, chaval ka ataa, laal chandan, haldi, ashtagandh, chandan chura, char kalash, kesar vrishti, aarti, sugandhit kalash, mahashantidhara evam mahaarghya ke saath bhagvan Neminath ko samarpit kiya jata hai.
# Ground Truth:	 yah abhishek jal, ikshuras, dudh, chaval ka ataa, laal chandan, haldi, ashtagandh, chandan chura, char kalash, kesar vrishti, aarti, sugandhit kalash, mahashantidhara evam mahaarghya ke saath bhagvan Neminath ko samarpit kiya jata hai.
# ‾‾‾‾‾‾‾‾‾‾‾‾
# Input text:	 कुछ ने कहा ये चांद है कुछ ने कहा चेहरा तेरा
# Prediction:	 uchh ne kaha ye chand hai kuch ne kaha chehra tera
# Ground Truth:	 kuch ne kaha ye chand hai kuch ne kaha chehra ter
# {'bleu': 0.9628980475343849, 'precisions': [0.9649122807017544, 0.9636363636363636, 0.9622641509433962, 0.9607843137254902], 'brevity_penalty': 1.0, 'length_ratio': 1.0, 'translation_length': 57, 'reference_length': 57}


# TIME:  5.650054216384888
# ‾‾‾‾‾‾‾‾‾‾‾‾
# Input text:	 yah abhishek jal, ikshuras, dudh, chaval ka ataa, laal chandan, haldi, ashtagandh, chandan chura, char kalash, kesar vrishti, aarti, sugandhit kalash, mahashantidhara evam mahaarghya ke saath bhagvan Neminath ko samarpit kiya jata hai.
# Prediction:	 यह अभिषेक जल, इक्षुरस, दुध, चावल का आता, लाल चंदन, हल्दी, अष्टगंध, चंदन चुरा, चार कलश, केसर व्टि, आरती, सुगंधित कलश, महाशांतारा
# Ground Truth:	 यह अभिषेक जल, इक्षुरस, दुध, चावल का आटा, लाल चंदन, हल्दी, अष्टगंध, चंदन चुरा, चार कलश, केसर वृष्टि, आरती, सुगंधित कलश, महाशांतिधारा एवं महाअर्घ्य के साथ भगवान नेमिनाथ को समर्पित किया जाता है।
# ‾‾‾‾‾‾‾‾‾‾‾‾
# Input text:	 kuch ne kaha ye chand hai kuch ne kaha chehra ter
# Prediction:	 कुछ ने कहा ये चाँद है कुछ ने कहा चेहरा तेर
# Ground Truth:	 कुछ ने कहा ये चांद है कुछ ने कहा चेहरा तेरा
# {'bleu': 0.5977286781346162, 'precisions': [0.8888888888888888, 0.813953488372093, 0.7317073170731707, 0.6410256410256411], 'brevity_penalty': 0.7831394949065555, 'length_ratio': 0.8035714285714286, 'translation_length': 45, 'reference_length': 56}

Training Procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 7e-05
• train_batch_size: 60
• eval_batch_size: 20
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 120
• total_eval_batch_size: 40
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 18.0

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.4.0+cu121
• Datasets 2.21.0
• Tokenizers 0.19.1