Preprocessing/multilinguality/README.md

Zero-Shot Approximation of Language Embeddings

This directory contains all scripts that are needed to reproduce the meta learning for zero-shot part of our system. These scripts allow you to predict representations of languages purely based on distances between them, as measured by a variety of linguistically informed metrics, or even better, a learned combination thereof.

Applying zero-shot approximation to a trained model

Use run_zero_shot_lang_emb_injection.py to update the language embeddings of a trained model for all languages that were not seen during training (by default, supervised_languages.json is used to determine which languages were seen). See the script for arguments that can be passed (e.g. to use a custom model path). Here is an example:

cd IMS-Toucan/
python run_zero_shot_lang_emb_injection.py -m <model_path> -d <distance_type> -k <number_of_nearest_neighbors>

By default, the updated model is saved with a modified filename in the same directory.

Cached distance lookups

In order to apply any zero-shot approximation, cache files for distance lookups are required.

The ASP lookup file (asp_dict.pkl) needs to be downloaded from the release page. All other cache files are automatically generated as required when running run_zero_shot_lang_emb_injection.py.

Note: While the map, tree, and inverse ASP distances are model independent, the learned distance lookup is only applicable for the model it was trained on, i.e., different Toucan models require different learned-distance lookups. If you want to apply zero-shot approximation to a new model, make sure that you are not using an outdated, pre-existing learned distance lookup, but instead train a new learned distance metric.