Using a Pre-trained Model¶
Inference using a DeepSpeech pre-trained model can be done with a client/language binding package. We have four clients/language bindings in this repository, listed below, and also a few community-maintained clients/language bindings in other repositories, listed further down in this README.
deepspeech might, see below, require some runtime dependencies to be already installed on your system:
sox- The Python and Node.JS clients use SoX to resample files to 16kHz.
libgomp1- libsox (statically linked into the clients) depends on OpenMP. Some people have had to install this manually.
libstdc++- Standard C++ Library implementation. Some people have had to install this manually.
libpthread- On Linux, some people have had to install libpthread manually. On Ubuntu,
libpthreadis part of the
Redistribuable Visual C++ 2015 Update 3 (64-bits)- On Windows, it might be required to ensure this is installed. Please download from Microsoft.
Please refer to your system’s documentation on how to install these dependencies.
The GPU capable builds (Python, NodeJS, C++, etc) depend on the same CUDA runtime as upstream TensorFlow. Currently with TensorFlow 2.2 it depends on CUDA 10.1 and CuDNN v7.6. See the TensorFlow documentation.
Getting the pre-trained model¶
If you want to use the pre-trained English model for performing speech-to-text, you can download it (along with other important inference material) from the DeepSpeech releases page. Alternatively, you can run the following command to download the model files in your current directory:
wget https://github.com/mozilla/DeepSpeech/releases/download/v0.8.1/deepspeech-0.8.1-models.pbmm wget https://github.com/mozilla/DeepSpeech/releases/download/v0.8.1/deepspeech-0.8.1-models.scorer
There are several pre-trained model files available in official releases. Files ending in
.pbmm are compatible with clients and language bindings built against the standard TensorFlow runtime. Usually these packages are simply called
deepspeech. These files are also compatible with CUDA enabled clients and language bindings. These packages are usually called
deepspeech-gpu. Files ending in
.tflite are compatible with clients and language bindings built against the TensorFlow Lite runtime. These models are optimized for size and performance in low power devices. On desktop platforms, the compatible packages are called
deepspeech-tflite. On Android and Raspberry Pi, we only publish TensorFlow Lite enabled packages, and they are simply called
deepspeech. You can see a full list of supported platforms and which TensorFlow runtime is supported at Supported platforms for inference.
Depends on platform
Depends on platform
Finally, the pre-trained model files also include files ending in
.scorer. These are external scorers (language models) that are used at inference time in conjunction with an acoustic model (
.tflite file) to produce transcriptions. We also provide further documentation on the decoding process and how scorers are generated.
Important considerations on model inputs¶
The release notes include detailed information on how the released models were trained/constructed. Important considerations for users include the characteristics of the training data used and whether they match your intended use case. For acoustic models, an important characteristic is the demographic distribution of speakers. For external scorers, the texts should be similar to those of the expected use case. If the data used for training the models does not align with your intended use case, it may be necessary to adapt or train new models in order to get good accuracy in your transcription results.
The process for training an acoustic model is described in Training Your Own Model. In particular, fine tuning a release model using your own data can be a good way to leverage relatively smaller amounts of data that would not be sufficient for training a new model from scratch. See the fine tuning and transfer learning sections for more information. Data augmentation can also be a good way to increase the value of smaller training sets.
Creating your own external scorer from text data is another way that you can adapt the model to your specific needs. The process and tools used to generate an external scorer package are described in External scorer scripts and an overview of how the external scorer is used by DeepSpeech to perform inference is available in CTC beam search decoder. Generating a smaller scorer from a single purpose text dataset is a quick process and can bring significant accuracy improvements, specially for more constrained, limited vocabulary applications.
DeepSpeech models are versioned to keep you from trying to use an incompatible graph with a newer client after a breaking change was made to the code. If you get an error saying your model file version is too old for the client, you should either upgrade to a newer model release, re-export your model from the checkpoint using a newer version of the code, or downgrade your client if you need to use the old model and can’t re-export it.
Using the Python package¶
Pre-built binaries which can be used for performing inference with a trained model can be installed with
pip3. You can then use the
deepspeech binary to do speech-to-text on an audio file:
For the Python bindings, it is highly recommended that you perform the installation within a Python 3.5 or later virtual environment. You can find more information about those in this documentation.
We will continue under the assumption that you already have your system properly setup to create new virtual environments.
Create a DeepSpeech virtual environment¶
In creating a virtual environment you will create a directory containing a
python3 binary and everything needed to run deepspeech. You can use whatever directory you want. For the purpose of the documentation, we will rely on
$HOME/tmp/deepspeech-venv. You can create it using this command:
$ virtualenv -p python3 $HOME/tmp/deepspeech-venv/
Once this command completes successfully, the environment will be ready to be activated.
Activating the environment¶
Each time you need to work with DeepSpeech, you have to activate this virtual environment. This is done with this simple command:
$ source $HOME/tmp/deepspeech-venv/bin/activate
Installing DeepSpeech Python bindings¶
Once your environment has been set-up and loaded, you can use
pip3 to manage packages locally. On a fresh setup of the
virtualenv, you will have to install the DeepSpeech wheel. You can check if
deepspeech is already installed with
To perform the installation, just use
pip3 as such:
$ pip3 install deepspeech
deepspeech is already installed, you can update it as such:
$ pip3 install --upgrade deepspeech
Alternatively, if you have a supported NVIDIA GPU on Linux, you can install the GPU specific package as follows:
$ pip3 install deepspeech-gpu
You can update
deepspeech-gpu as follows:
$ pip3 install --upgrade deepspeech-gpu
In both cases,
pip3 should take care of installing all the required dependencies. After installation has finished, you should be able to call
deepspeech from the command-line.
Note: the following command assumes you downloaded the pre-trained model.
deepspeech --model deepspeech-0.8.1-models.pbmm --scorer deepspeech-0.8.1-models.scorer --audio my_audio_file.wav
--scorer argument is optional, and represents an external language model to be used when transcribing the audio.
See the Python client for an example of how to use the package programatically.
Using the Node.JS / Electron.JS package¶
You can download the JS bindings using
npm install deepspeech
- Please note that as of now, we support:
Node.JS versions 4 to 13.
Electron.JS versions 1.6 to 7.1
TypeScript support is also provided.
Alternatively, if you’re using Linux and have a supported NVIDIA GPU, you can install the GPU specific package as follows:
npm install deepspeech-gpu
See the TypeScript client for an example of how to use the bindings programatically.
Using the command-line client¶
To download the pre-built binaries for the
deepspeech command-line (compiled C++) client, use
python3 util/taskcluster.py --target .
or if you’re on macOS:
python3 util/taskcluster.py --arch osx --target .
also, if you need some binaries different than current master, like
v0.2.0-alpha.6, you can use
python3 util/taskcluster.py --branch "v0.2.0-alpha.6" --target "."
taskcluster.py will download
native_client.tar.xz (which includes the
deepspeech binary and associated libraries) and extract it into the current folder. Also,
taskcluster.py will download binaries for Linux/x86_64 by default, but you can override that behavior with the
--arch parameter. See the help info with
python util/taskcluster.py -h for more details. Specific branches of DeepSpeech or TensorFlow can be specified as well.
Alternatively you may manually download the
native_client.tar.xz from the [releases](https://github.com/mozilla/DeepSpeech/releases).
Note: the following command assumes you downloaded the pre-trained model.
./deepspeech --model deepspeech-0.8.1-models.pbmm --scorer deepspeech-0.8.1-models.scorer --audio audio_input.wav
See the help output with
./deepspeech -h for more details.
Installing bindings from source¶
If pre-built binaries aren’t available for your system, you’ll need to install them from scratch. Follow the native client build and installation instructions.
Dockerfile for building from source¶
Dockerfile.build to automatically build
libdeepspeech.so, the C++ native client, Python bindings, and KenLM.
You need to generate the Dockerfile from the template using:
If you want to specify a different DeepSpeech repository / branch, you can pass
make Dockerfile.build DEEPSPEECH_REPO=git://your/fork DEEPSPEECH_SHA=origin/your-branch
Third party bindings¶
In addition to the bindings above, third party developers have started to provide bindings to other languages: