bat365 Homepage

Automatic Differentiation

Customize deep learning layers, networks, training loops, and loss functions

For most tasks, you can use built-in layers. If there is not a built-in layer that you need for your task, then you can define your own custom layer. You can specify a custom loss function using a custom output layer and define custom layers with learnable and state parameters. After defining a custom layer, you can check that the layer is valid, GPU compatible, and outputs correctly defined gradients. For a list of supported layers, see List of Deep Learning Layers.

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For models that layer graphs do not support, you can define a custom model as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Frequently Viewed Topics

Featured Examples

Train a Twin Network for Dimensionality Reduction

Train a twin neural network with shared weights to compare handwritten digits using dimensionality reduction.

Open Live Script

Train a Twin Neural Network to Compare Images

Train a twin neural network with shared weights to identify similar images of handwritten characters.

Open Live Script

Image Captioning Using Attention

Train a deep learning model for image captioning using attention.

Open Live Script

Multilabel Text Classification Using Deep Learning

Classify text data that has multiple independent labels.

Open Live Script

Sequence-to-Sequence Translation Using Attention

Convert decimal strings to Roman numerals using a recurrent sequence-to-sequence encoder-decoder model with attention.

Open Live Script

Train Network in Parallel with Custom Training Loop

Set up a custom training loop to train a network in parallel. In this example, parallel workers train on portions of the overall mini-batch. If you have a GPU, then training happens on the GPU. During training, a DataQueue object sends training progress information back to the MATLAB client.

Open Live Script

Train Variational Autoencoder (VAE) to Generate Images

Train a deep learning variational autoencoder (VAE) to generate images.

Open Live Script

Train Network Using Cyclical Learning Rate for Snapshot Ensembling

Train a network to classify images of objects using a cyclical learning rate schedule and snapshot ensembling for better test accuracy. In the example, you learn how to use a cosine function for the learning rate schedule, take snapshots of the network during training to create a model ensemble, and add L2-norm regularization (weight decay) to the training loss.

Open Live Script