Rasa Docker Image with GPU support

Rasa Enterprise Feature

Ready for Rasa Enterprise? Schedule a technical demo ->

You can enable hardware acceleration on Rasa by using a Docker container.

Prerequisites

Make sure you have the following installed on your machine:

Remember to reboot the machine when all the requirements are installed.

If you are not sure if you have all the requirements installed, you can check by running:

docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi

You should see information about your NVIDIA GPU devices if GPU-enabled Docker is configured on your machine.

Image release tag format

GPU image releases are tagged using the following format:

Tag	Description
`<latest/rasa version>-full-gpu`	The latest release or a specified version of Rasa GPU image.
`<latest/rasa version>-mitie-en-gpu`	The latest release or a specified version of Rasa GPU image with mitie.
`<latest/rasa version>-spacy-en-gpu`	The latest release or a specified version of Rasa GPU image with SpaCy in English.
`<latest/rasa version>-spacy-de-gpu`	The latest release or a specified version of Rasa GPU image with SpaCy in German.

You can specify the version of the Rasa GPU image, for example, 3.0.0.

For all versions of rasa >= 3.0.0, GPU images are released and ready for use. If you are using an older version of rasa, please reach out to your point of contact at Rasa.

Start the Docker container with GPU acceleration

To enable GPU usage, add the flag --gpu all, to your docker run command. In the following example, arguments inside square brackets [] are optional:

docker run 
    [-it][--rm] --gpus all [-v $PWD:/tmp] gcr.io/rasa-platform/rasa:<tag>[rasa command]

Here is a quick explanation of the arguments used in the example command:

-it use interactive terminal
--rm clean up the anonymous volumes associated with the container when the container is removed
-v $PWD:/tmp mount your current working directory - this should be the root of your Rasa project
tag specifies which image tag to use - remember to use a tag ending with -gpu, e.g., 3.0.0-full-gpu
rasa command arguments to pass to the rasa command line interface

Please refer to the official docker run docs for more details.

Troubleshooting

If you see the following error when using GPU resources:

$ docker run --rm -it --gpus all --entrypoint "/bin/bash" rasa:3.0.0-full-gpu
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]].

Install NVIDIA Container Toolkit and reboot the machine.