Compare Pretrained Neural Networks

Pretrained neural networks have different characteristics that matter when choosing a neural network to apply to your problem. The most important characteristics are neural network accuracy, speed, and size. Choosing a neural network is generally a tradeoff between these characteristics. Use the plot below to compare the ImageNet validation accuracy with the time required to make a prediction using the neural network.

A good neural network has a high accuracy and is fast. The plot displays the classification accuracy versus the prediction time when using a modern GPU (an NVIDIA^® Tesla^® P100) and a mini-batch size of 128. The prediction time is measured relative to the fastest neural network. The area of each marker is proportional to the size of the neural network on disk.

The classification accuracy on the ImageNet validation set is the most common way to measure the accuracy of neural networks trained on ImageNet. Neural networks that are accurate on ImageNet are also often accurate when you apply them to other natural image data sets using transfer learning or feature extraction. This generalization is possible because the neural networks have learned to extract powerful and informative features from natural images that generalize to other similar data sets. However, high accuracy on ImageNet does not always transfer directly to other tasks, so it is a good idea to try multiple neural networks.

If you want to perform prediction using constrained hardware or distribute neural networks over the Internet, then also consider the size of the neural network on disk and in memory.

Load Pretrained Neural Networks

To load the SqueezeNet neural network, type squeezenet at the command line.

net = squeezenet;

For other neural networks, use functions such as googlenet to get links to download pretrained neural networks from the Add-On Explorer.

The following table lists the available pretrained neural networks trained on ImageNet and some of their properties. The neural network depth is defined as the largest number of sequential convolutional or fully connected layers on a path from the input layer to the output layer. The inputs to all neural networks are RGB images.

*The NASNet-Mobile and NASNet-Large neural networks do not consist of a linear sequence of modules.

Visualize Pretrained Neural Networks

You can load and visualize pretrained neural networks using Deep Network Designer.

deepNetworkDesigner(squeezenet)

To view and edit layer properties, select a layer. Click the help icon next to the layer name for information on the layer properties.

Explore other pretrained neural networks in Deep Network Designer by clicking New.

If you need to download a neural network, pause on the desired neural network and click Install to open the Add-On Explorer.

Feature Extraction

Feature extraction is an easy and fast way to use the power of deep learning without investing time and effort into training a full neural network. Because it only requires a single pass over the training images, it is especially useful if you do not have a GPU. You extract learned image features using a pretrained neural network, and then use those features to train a classifier, such as a support vector machine using fitcsvm (Statistics and Machine Learning Toolbox).

Try feature extraction when your new data set is very small. Since you only train a simple classifier on the extracted features, training is fast. It is also unlikely that fine-tuning deeper layers of the neural network improves the accuracy since there is little data to learn from.

ResNets are often good feature extractors. For an example showing how to use a pretrained neural network for feature extraction, see Extract Image Features Using Pretrained Network.

Transfer Learning

You can fine-tune deeper layers in the neural network by training the neural network on your new data set with the pretrained neural network as a starting point. Fine-tuning a neural network with transfer learning is often faster and easier than constructing and training a new neural network. The neural network has already learned a rich set of image features, but when you fine-tune the neural network it can learn features specific to your new data set. If you have a very large data set, then transfer learning might not be faster than training from scratch.

Fine-tuning a neural network is slower and requires more effort than simple feature extraction, but since the neural network can learn to extract a different set of features, the final neural network is often more accurate. Fine-tuning usually works better than feature extraction as long as the new data set is not very small, because then the neural network has data to learn new features from. For examples showing how to perform transfer learning, see Transfer Learning with Deep Network Designer and Train Deep Learning Network to Classify New Images.

Import and Export Neural Networks

You can import neural networks and layer graphs from TensorFlow™ 2, TensorFlow-Keras, PyTorch^®, and the ONNX™ (Open Neural Network Exchange) model format. You can also export Deep Learning Toolbox neural networks and layer graphs to TensorFlow 2 and the ONNX model format.

The importTensorFlowNetwork and importTensorFlowLayers functions are recommended over the importKerasNetwork and importKerasLayers functions. For more information, see Recommended Functions to Import TensorFlow Models.

The importTensorFlowNetwork, importTensorFlowLayers, importNetworkFromPyTorch, importONNXNetwork, and importONNXLayers functions create automatically generated custom layers when you import a model with TensorFlow layers, PyTorch layers, or ONNX operators that the functions cannot convert to built-in MATLAB^® layers. The functions save the automatically generated custom layers to a package in the current folder. For more information, see Autogenerated Custom Layers.

The exportNetworkToTensorFlow function saves a Deep Learning Toolbox neural network or layer graph as a TensorFlow model in a Python package. For more information on how to load the exported model and save it in a standard TensorFlow format, see Load Exported TensorFlow Model and Save Exported TensorFlow Model in Standard Format.

By using ONNX as an intermediate format, you can interoperate with other deep learning frameworks that support ONNX model export or import.

Pretrained Neural Networks for Audio Applications

Audio Toolbox™ provides the pretrained VGGish, YAMNet, OpenL3, and CREPE neural networks. Use the vggish (Audio Toolbox), yamnet (Audio Toolbox), openl3 (Audio Toolbox), and crepe (Audio Toolbox) functions in MATLAB or the VGGish (Audio Toolbox) and YAMNet (Audio Toolbox) blocks in Simulink^® to interact directly with the pretrained neural networks. You can also import and visualize audio pretrained neural networks using Deep Network Designer.

The following table lists the available pretrained audio neural networks and some of their properties.

Use VGGish and YAMNet to perform transfer learning and feature extraction. Extract VGGish or OpenL3 feature embeddings to input to machine learning and deep learning systems. The classifySound (Audio Toolbox) function and the Sound Classifier (Audio Toolbox) block use YAMNet to locate and classify sounds into one of 521 categories. The pitchnn (Audio Toolbox) function uses CREPE to perform deep learning pitch estimation.

For examples showing how to adapt pretrained audio neural networks for a new task, see Transfer Learning with Pretrained Audio Networks (Audio Toolbox) and Transfer Learning with Pretrained Audio Networks in Deep Network Designer.

For more information on using deep learning for audio applications, see Deep Learning for Audio Applications (Audio Toolbox).

Pretrained Models on GitHub

To find the latest pretrained models, see MATLAB Deep Learning Model Hub.

For example: