imageInputLayer (Deep Learning Toolbox)

An image input layer inputs 2-D images to a network and applies data normalization.

Code generation does not support 'Normalization' specified using a function handle.

Yes

Yes

Yes

sequenceInputLayer (Deep Learning Toolbox)

A sequence input layer inputs sequence data to a network.

The cuDNN library supports vector and 2-D image sequences. The TensorRT library support only vector input sequences.

For vector sequence inputs, the number of features must be a constant during code generation.

For image sequence inputs, the height, width, and the number of channels must be a constant during code generation.

Code generation does not support 'Normalization' specified using a function handle.

Yes

Yes

No

featureInputLayer (Deep Learning Toolbox)

A feature input layer inputs feature data to a network and applies data normalization.

Yes

Yes

Yes

convolution2dLayer (Deep Learning Toolbox)

A 2-D convolutional layer applies sliding convolutional filters to the input.

Yes

Yes

Yes

fullyConnectedLayer (Deep Learning Toolbox)

A fully connected layer multiplies the input by a weight matrix and then adds a bias vector.

Yes

Yes

No

groupedConvolution2dLayer (Deep Learning Toolbox)

A 2-D grouped convolutional layer separates the input channels into groups and applies sliding convolutional filters. Use grouped convolutional layers for channel-wise separable (also known as depth-wise separable) convolution.

Code generation for the ARM Mali GPU is not supported for a 2-D grouped convolution layer that has the NumGroups property set as 'channel-wise' or a value greater than two.

Yes

Yes

Yes

transposedConv2dLayer (Deep Learning Toolbox)

A transposed 2-D convolution layer upsamples feature maps.

Yes

Yes

Yes

sequenceInputLayer (Deep Learning Toolbox)

A sequence input layer inputs sequence data to a network.

The cuDNN library supports vector and 2-D image sequences. The TensorRT library support only vector input sequences.

For vector sequence inputs, the number of features must be a constant during code generation.

For image sequence inputs, the height, width, and the number of channels must be a constant during code generation.

Code generation does not support 'Normalization' specified using a function handle.

Yes

Yes

No

bilstmLayer (Deep Learning Toolbox)

A bidirectional LSTM (BiLSTM) layer learns bidirectional long-term dependencies between time steps of time series or sequence data. These dependencies can be useful when you want the network to learn from the complete time series at each time step.

For code generation, the StateActivationFunction property must be set to 'tanh'.

For code generation, the GateActivationFunction property must be set to 'sigmoid'.

Yes

Yes

No

flattenLayer (Deep Learning Toolbox)

A flatten layer collapses the spatial dimensions of the input into the channel dimension.

Yes

No

No

gruLayer (Deep Learning Toolbox)

A GRU layer learns dependencies between time steps in time series and sequence data.

Code generation supports only the 'after-multiplication' and 'recurrent-bias-after-multiplication' reset gate modes.

Yes

Yes

No

lstmLayer (Deep Learning Toolbox)

An LSTM layer learns long-term dependencies between time steps in time series and sequence data.

For code generation, the StateActivationFunction property must be set to 'tanh'.

For code generation, the GateActivationFunction property must be set to 'sigmoid'.

Yes

Yes

No

sequenceFoldingLayer (Deep Learning Toolbox)

A sequence folding layer converts a batch of image sequences to a batch of images. Use a sequence folding layer to perform convolution operations on time steps of image sequences independently.

Yes

No

No

sequenceUnfoldingLayer (Deep Learning Toolbox)

A sequence unfolding layer restores the sequence structure of the input data after sequence folding.

Yes

No

No

wordEmbeddingLayer (Text Analytics Toolbox)

A word embedding layer maps word indices to vectors.

Yes

Yes

No

clippedReluLayer (Deep Learning Toolbox)

A clipped ReLU layer performs a threshold operation, where a input value less than zero is set to zero and a value above the clipping ceiling is set to that clipping ceiling.

Yes

Yes

Yes

eluLayer (Deep Learning Toolbox)

An ELU activation layer performs the identity operation on positive inputs and an exponential nonlinearity on negative inputs.

Yes

Yes

No

leakyReluLayer (Deep Learning Toolbox)

A leaky ReLU layer performs a threshold operation, where a input value less than zero is multiplied by a fixed scalar.

Yes

Yes

Yes

reluLayer (Deep Learning Toolbox)

A ReLU layer performs a threshold operation to each element of the input, where a value less than zero is set to zero.

Yes

Yes

Yes

gelu (Deep Learning Toolbox)

A GELU layer weights the input by its probability under a Gaussian distribution.

Yes

Yes

Yes

softplusLayer (Reinforcement Learning Toolbox)

A SoftplusLayer is a deep neural network layer that implements the softplus activation Y = log(1 + e^X), which ensures that the output is always positive.

Yes

Yes

No

swishLayer (Deep Learning Toolbox)

A swish activation layer applies the swish function on the layer inputs.

Yes

Yes

No

tanhLayer (Deep Learning Toolbox)

A hyperbolic tangent (tanh) activation layer applies the tanh function on the layer inputs.

Yes

Yes

Yes

batchNormalizationLayer (Deep Learning Toolbox)

A batch normalization layer normalizes each input channel across a mini-batch.

Yes

Yes

Yes

crossChannelNormalizationLayer (Deep Learning Toolbox)

A channel-wise local response (cross-channel) normalization layer carries out channel-wise normalization.

Yes

Yes

Yes

groupNormalizationLayer (Deep Learning Toolbox)

A group normalization layer normalizes a mini-batch of data across grouped subsets of channels for each observation independently.

Yes

Yes

No

layerNormalizationLayer (Deep Learning Toolbox)

A layer normalization layer normalizes a mini-batch of data across all channels for each observation independently.

Yes

Yes

No

crop2dLayer (Deep Learning Toolbox)

A 2-D crop layer applies 2-D cropping to the input.

Yes

Yes

Yes

dropoutLayer (Deep Learning Toolbox)

A dropout layer randomly sets input elements to zero with a given probability.

Yes

Yes

Yes

scalingLayer (Reinforcement Learning Toolbox)

Scaling layer for actor or critic network.

For code generation, values for the 'Scale' and 'Bias' properties must have the same dimension.

Yes

Yes

Yes

averagePooling2dLayer (Deep Learning Toolbox)

An average pooling layer performs down-sampling by dividing the input into rectangular pooling regions and computing the average values of each region.

You can generate code using the NVIDIA^® cuDNN and TensorRT libraries by using the 'mean' setting for the PaddingValue property if the padding size is symmetric.

For Simulink^® models that implement deep learning functionality using MATLAB Function block, simulation errors out if the network contains an average pooling layer with non-zero padding value. In such cases, use the blocks from the Deep Neural Networks library instead of a MATLAB Function to implement the deep learning functionality.

Yes

Yes

Yes

globalAveragePooling2dLayer (Deep Learning Toolbox)

A global average pooling layer performs down-sampling by computing the mean of the height and width dimensions of the input.

Yes

Yes

Yes

globalMaxPooling2dLayer (Deep Learning Toolbox)

A global max pooling layer performs down-sampling by computing the maximum of the height and width dimensions of the input.

Yes

Yes

Yes

maxPooling2dLayer (Deep Learning Toolbox)

A max pooling layer performs down-sampling by dividing the input into rectangular pooling regions, and computing the maximum of each region.

If equal max values exists along the off-diagonal in a kernel window, implementation differences for the maxPooling2dLayer might cause minor numerical mismatch between MATLAB^® and the generated code. This issue also causes mismatch in the indices of the maximum value in each pooled region. For more information, see maxPooling2dLayer (Deep Learning Toolbox).

Yes

Yes

Yes

maxUnpooling2dLayer (Deep Learning Toolbox)

A max unpooling layer unpools the output of a max pooling layer.

If equal max values exists along the off-diagonal in a kernel window, implementation differences for the maxPooling2dLayer might cause minor numerical mismatch between MATLAB and the generated code. This issue also causes mismatch in the indices of the maximum value in each pooled region. For more information, see maxUnpooling2dLayer (Deep Learning Toolbox).

Yes

Yes

No

additionLayer (Deep Learning Toolbox)

An addition layer adds inputs from multiple neural network layers element-wise.

Yes

Yes

Yes

concatenationLayer (Deep Learning Toolbox)

A concatenation layer takes inputs and concatenates them along a specified dimension.

Yes

Yes

No

depthConcatenationLayer (Deep Learning Toolbox)

A depth concatenation layer takes inputs that have the same height and width and concatenates them along the third dimension (the channel dimension).

Yes

Yes

Yes

anchorBoxLayer (Computer Vision Toolbox)

An anchor box layer stores anchor boxes for a feature map used in object detection networks.

Yes

Yes

Yes

depthToSpace2dLayer (Image Processing Toolbox)

Yes

Yes

Yes

spaceToDepthLayer (Image Processing Toolbox)

Yes

Yes

Yes

ssdMergeLayer (Computer Vision Toolbox)

An SSD merge layer merges the outputs of feature maps for subsequent regression and classification loss computation.

Yes

Yes

No

yolov2ReorgLayer (Computer Vision Toolbox)

Create reorganization layer for YOLO v2 object detection network.

Yes

Yes

Yes

yolov2TransformLayer (Computer Vision Toolbox)

Create transform layer for YOLO v2 object detection network.

Yes

Yes

Yes

rcnnBoxRegressionLayer (Computer Vision Toolbox)

Yes

Yes

Yes

focalLossLayer (Computer Vision Toolbox)

Yes

Yes

Yes

rpnClassificationLayer (Computer Vision Toolbox)

Yes

Yes

Yes

yolov2OutputLayer (Computer Vision Toolbox)

Create output layer for YOLO v2 object detection network.

Yes

Yes

Yes

classificationLayer (Deep Learning Toolbox)

A classification layer computes the cross entropy loss for multi-class classification problems with mutually exclusive classes.

Yes

Yes

Yes

dicePixelClassificationLayer (Computer Vision Toolbox)

A Dice pixel classification layer provides a categorical label for each image pixel or voxel using generalized Dice loss.

Yes

Yes

Yes

focalLossLayer (Computer Vision Toolbox)

Yes

Yes

Yes

pixelClassificationLayer (Computer Vision Toolbox)

A pixel classification layer provides a categorical label for each image pixel or voxel.

Yes

Yes

Yes

rcnnBoxRegressionLayer (Computer Vision Toolbox)

Yes

Yes

Yes

regressionLayer (Deep Learning Toolbox)

A regression layer computes the half-mean-squared-error loss for regression problems.

Yes

Yes

Yes

rpnClassificationLayer (Computer Vision Toolbox)

Yes

Yes

Yes

sigmoidLayer (Deep Learning Toolbox)

A sigmoid layer applies a sigmoid function to the input.

Yes

Yes

Yes

softmaxLayer (Deep Learning Toolbox)

A softmax layer applies a softmax function to the input.

Yes

Yes

Yes

Output Layer (Deep Learning Toolbox)

Output layers including custom classification or regression output layers created by using nnet.layer.ClassificationLayer or nnet.layer.RegressionLayer.

For an example showing how to define a custom classification output layer and specify a loss function, see Define Custom Classification Output Layer (Deep Learning Toolbox).

For an example showing how to define a custom regression output layer and specify a loss function, see Define Custom Regression Output Layer (Deep Learning Toolbox).

Yes

Yes

Yes

nnet.keras.layer.ClipLayer (Deep Learning Toolbox)

Clips the input between the upper and lower bounds.

Yes

Yes

No

nnet.keras.layer.FlattenCStyleLayer (Deep Learning Toolbox)

Flatten activations into 1-D assuming C-style (row-major) order.

Yes

Yes

Yes

nnet.keras.layer.GlobalAveragePooling2dLayer (Deep Learning Toolbox)

Global average pooling layer for spatial data.

Yes

Yes

Yes

nnet.keras.layer.PreluLayer (Deep Learning Toolbox)

Parametric rectified linear unit.

Yes

Yes

No

nnet.keras.layer.SigmoidLayer (Deep Learning Toolbox)

Sigmoid activation layer.

Yes

Yes

Yes

nnet.keras.layer.TanhLayer (Deep Learning Toolbox)

Hyperbolic tangent activation layer.

Yes

Yes

Yes

nnet.keras.layer.TimeDistributedFlattenCStyleLayer (Deep Learning Toolbox)

Flatten a sequence of input image into a sequence of vector, assuming C-style (or row-major) storage ordering of the input layer.

Yes

Yes

No

nnet.keras.layer.ZeroPadding2dLayer (Deep Learning Toolbox)

Zero padding layer for 2-D input.

Yes

Yes

Yes

nnet.onnx.layer.ClipLayer (Deep Learning Toolbox)

Clips the input between the upper and lower bounds.

Yes

Yes

No

nnet.onnx.layer.ElementwiseAffineLayer (Deep Learning Toolbox)

Layer that performs element-wise scaling of the input followed by an addition.

Yes

Yes

Yes

nnet.onnx.layer.FlattenInto2dLayer (Deep Learning Toolbox)

Flattens a MATLAB 2D image batch in the way ONNX does, producing a 2D output array with CB format.

Yes

Yes

No

nnet.onnx.layer.FlattenLayer (Deep Learning Toolbox)

Flattens the spatial dimensions of the input tensor to the channel dimensions.

Yes

Yes

Yes

nnet.onnx.layer.GlobalAveragePooling2dLayer (Deep Learning Toolbox)

Global average pooling layer for spatial data.

Yes

Yes

Yes

nnet.onnx.layer.IdentityLayer (Deep Learning Toolbox)

Layer that implements ONNX identity operator.

Yes

Yes

Yes

nnet.onnx.layer.PreluLayer (Deep Learning Toolbox)

Parametric rectified linear unit.

Yes

Yes

No

nnet.onnx.layer.SigmoidLayer (Deep Learning Toolbox)

Sigmoid activation layer.

Yes

Yes

Yes

nnet.onnx.layer.TanhLayer (Deep Learning Toolbox)

Hyperbolic tangent activation layer.

Yes

Yes

Yes

nnet.onnx.layer.VerifyBatchSizeLayer (Deep Learning Toolbox)

Verify fixed batch size.

Yes

Yes

Yes

Custom layers

Custom layers, with or without learnable parameters, that you define for your problem.

To learn how to define custom deep learning layers, see Define Custom Deep Learning Layers (Deep Learning Toolbox) and Define Custom Deep Learning Layer for Code Generation (Deep Learning Toolbox).

For an example on how to generate code for a network with custom layers, see Code Generation for Object Detection Using YOLO v3 Deep Learning Network.

The outputs of the custom layer must be fixed-size arrays.

Using 'unified' as the MallocMode in coder.gpuConfig requires extra memory copies leading to slower performance. For custom layers, it is recommended to use 'discrete' mode. For more information on GPU memory allocation, see Discrete and Managed Modes

cuDNN targets support both row-major and column-major code generation for custom layers. TensorRT targets support only column-major code generation.

For code generation, custom layers must contain the %#codegen pragma.

Code generation for a sequence network containing custom layer and LSTM or GRU layer is not supported.

You can pass dlarray to custom layers if:

For unsupported dlarray methods, then you must extract the underlying data from the dlarray, perform the computations and reconstruct the data back into the dlarray for code generation. For example,

function Z = predict(layer, X)

if coder.target('MATLAB')
   Z = doPredict(X);
else
   if isdlarray(X)
      X1 = extractdata(X);
      Z1 = doPredict(X1);
      Z = dlarray(Z1);
  else
      Z = doPredict(X);
  end
end

end

Yes

Yes

No