metadata

license: apache-2.0
datasets:
  - opus_books
  - iwslt2017
language:
  - en
  - nl
metrics:
  - sacrebleu
pipeline_tag: text2text-generation
tags:
  - translation
widget:
  - text: '>>en<< Was het leuk?'

NOTE: This is a work-in-progress model that is not considered finished. Keep this in mind when using this model, or continue training this model.

Model Card for mt5-small nl-en translation

The mt5-small nl-en translation model is a finetuned version of google/mt5-small.

It was finetuned on 237k rows of the iwslt2017 dataset and roughly 38k rows of the opus_books dataset. The model was trained in multiple phases with different epochs & batch sizes.

How to use

Install dependencies

pip install transformers, sentencepiece, protobuf

You can use the following code for model inference. This model was finetuned to work with an identifier when prompted that needs to be present for the best results.

from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig

tokenizer = AutoTokenizer.from_pretrained("Michielo/mt5-small_nl-en_translation")
model = AutoModelForSeq2SeqLM.from_pretrained("Michielo/mt5-small_nl-en_translation")

translation_generation_config = GenerationConfig(
    num_beams=4,
    early_stopping=True,
    decoder_start_token_id=0,
    eos_token_id=model.config.eos_token_id,
    pad_token=model.config.pad_token_id,
)

translation_generation_config.save_pretrained("/tmp", "translation_generation_config.json")
generation_config = GenerationConfig.from_pretrained("/tmp", "translation_generation_config.json")
inputs = tokenizer(">>en<< Your dutch text here", return_tensors="pt")
outputs = model.generate(**inputs, generation_config=generation_config)
print(tokenizer.batch_decode(outputs, skip_special_tokens=True))

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.