Add custom Dependencies

Inference Endpoints’ base image includes all required libraries to run inference on Transformers models, but it also supports custom dependencies. This is useful if you want to:

• customize your inference pipeline and need additional Python dependencies
• run a model which requires special dependencies like the newest or a fixed version of a library (for example, tapas (torch-scatter)).

To add custom dependencies, add a requirements.txt file with the Python dependencies you want to install in your model repository on the Hugging Face Hub. When your Endpoint and Image artifacts are created, Inference Endpoints checks if the model repository contains a requirements.txt file and installs the dependencies listed within.

optimum[onnxruntime]==1.2.3
mkl-include
mkl

Check out the requirements.txt files in the following model repositories for examples:

• Optimum and onnxruntime
• diffusers

For more information, take a look at how you can create and install dependencies when you use your own custom container for inference.