Download
SageMaker Hugging Face Inference Toolkit is an open-source library for serving Transformers models on Amazon SageMaker. This library provides default pre-processing, predict and postprocessing for certain Transformers models and tasks. It utilizes the SageMaker Inference Toolkit for starting up the model server, which is responsible for handling inference requests. For the Dockerfiles used for building SageMaker Hugging Face Containers, see AWS Deep Learning Containers. The SageMaker Hugging Face Inference Toolkit implements various additional environment variables to simplify your deployment experience. The Hugging Face Inference Toolkit allows user to override the default methods of the HuggingFaceHandlerService. SageMaker Hugging Face Inference Toolkit is licensed under the Apache 2.0 License.
Features
- Create a Amazon SageMaker endpoint with a model from the Hub
- Create a Amazon SageMaker endpoint with a trained model
- The HF_TASK environment variable defines the task for the used Transformers pipeline
- The HF_MODEL_ID environment variable defines the model id, which will be automatically loaded
- The HF_API_TOKEN environment variable defines the your Hugging Face authorization token
- User defined code/modules
Project Samples
