bat365 Homepage

Custom Training Loops

Customize deep learning training loops and loss functions

If the trainingOptions function does not provide the training options that you need for your task, or custom output layers do not support the loss functions that you need, then you can define a custom training loop. For models that layer graphs do not support, you can define a custom model as a function. To learn more, see Define Custom Training Loops, Loss Functions, and Networks.

Functions

expand all

Custom Networks

`dlnetwork`	Deep learning network for custom training loops (Since R2019b)
`resetState`	Reset state parameters of neural network
`plot`	Plot neural network architecture
`addInputLayer`	Add input layer to network (Since R2022b)
`addLayers`	Add layers to layer graph or network
`removeLayers`	Remove layers from layer graph or network
`connectLayers`	Connect layers in layer graph or network
`disconnectLayers`	Disconnect layers in layer graph or network
`replaceLayer`	Replace layer in layer graph or network
`summary`	Print network summary (Since R2022b)
`initialize`	Initialize learnable and state parameters of a `dlnetwork` (Since R2021a)
`networkDataLayout`	Deep learning network data layout for learnable parameter initialization (Since R2022b)
`layerGraph`	Graph of network layers for deep learning
`setL2Factor`	Set L2 regularization factor of layer learnable parameter
`getL2Factor`	Get L2 regularization factor of layer learnable parameter
`setLearnRateFactor`	Set learn rate factor of layer learnable parameter
`getLearnRateFactor`	Get learn rate factor of layer learnable parameter

Custom Training Loops

`forward`	Compute deep learning network output for training (Since R2019b)
`predict`	Compute deep learning network output for inference (Since R2019b)
`adamupdate`	Update parameters using adaptive moment estimation (Adam) (Since R2019b)
`rmspropupdate`	Update parameters using root mean squared propagation (RMSProp) (Since R2019b)
`sgdmupdate`	Update parameters using stochastic gradient descent with momentum (SGDM) (Since R2019b)
`lbfgsupdate`	Update parameters using limited-memory BFGS (L-BFGS) (Since R2023a)
`lbfgsState`	State of limited-memory BFGS (L-BFGS) solver (Since R2023a)
`dlupdate`	Update parameters using custom function (Since R2019b)
`trainingProgressMonitor`	Monitor and plot training progress for deep learning custom training loops (Since R2022b)
`updateInfo`	Update information values for custom training loops (Since R2022b)
`recordMetrics`	Record metric values for custom training loops (Since R2022b)
`groupSubPlot`	Group metrics in training plot (Since R2022b)

Data Processing

`padsequences`	Pad or truncate sequence data to same length (Since R2021a)
`minibatchqueue`	Create mini-batches for deep learning (Since R2020b)
`onehotencode`	Encode data labels into one-hot vectors (Since R2020b)
`onehotdecode`	Decode probability vectors into class labels (Since R2020b)
`next`	Obtain next mini-batch of data from minibatchqueue (Since R2020b)
`reset`	Reset minibatchqueue to start of data (Since R2020b)
`shuffle`	Shuffle data in minibatchqueue (Since R2020b)
`hasdata`	Determine if minibatchqueue can return mini-batch (Since R2020b)
`partition`	Partition minibatchqueue (Since R2020b)

Automatic Differentiation

`dlarray`	Deep learning array for customization (Since R2019b)
`dlgradient`	Compute gradients for custom training loops using automatic differentiation (Since R2019b)
`dlfeval`	Evaluate deep learning model for custom training loops (Since R2019b)
`dims`	Dimension labels of `dlarray` (Since R2019b)
`finddim`	Find dimensions with specified label (Since R2019b)
`stripdims`	Remove `dlarray` data format (Since R2019b)
`extractdata`	Extract data from `dlarray` (Since R2019b)
`isdlarray`	Check if object is `dlarray` (Since R2020b)

Loss Operations

`crossentropy`	Cross-entropy loss for classification tasks (Since R2019b)
`l1loss`	L₁ loss for regression tasks (Since R2021b)
`l2loss`	L₂ loss for regression tasks (Since R2021b)
`huber`	Huber loss for regression tasks (Since R2021a)
`mse`	Half mean squared error (Since R2019b)
`ctc`	Connectionist temporal classification (CTC) loss for unaligned sequence classification (Since R2021a)

Function Acceleration

`dlaccelerate`	Accelerate deep learning function for custom training loops (Since R2021a)
`AcceleratedFunction`	Accelerated deep learning function (Since R2021a)
`clearCache`	Clear accelerated deep learning function trace cache (Since R2021a)

Topics

Custom Training Loops

Train Deep Learning Model in MATLAB
Learn how to training deep learning models in MATLAB^®.
Define Custom Training Loops, Loss Functions, and Networks
Learn how to define and customize deep learning training loops, loss functions, and networks using automatic differentiation.
Train Network Using Custom Training Loop
This example shows how to train a network that classifies handwritten digits with a custom learning rate schedule.
Train Sequence Classification Network Using Custom Training Loop
This example shows how to train a network that classifies sequences with a custom learning rate schedule.
Specify Training Options in Custom Training Loop
Learn how to specify common training options in a custom training loop.
Define Model Loss Function for Custom Training Loop
Learn how to define a model loss function for a custom training loop.
Update Batch Normalization Statistics in Custom Training Loop
This example shows how to update the network state in a custom training loop.
Make Predictions Using dlnetwork Object
This example shows how to make predictions using a dlnetwork object by splitting data into mini-batches.
Monitor Custom Training Loop Progress
Track and plot custom training loop progress.
Multiple-Input and Multiple-Output Networks
Learn how to define and train deep learning networks with multiple inputs or multiple outputs.
Train Network with Multiple Outputs
This example shows how to train a deep learning network with multiple outputs that predict both labels and angles of rotations of handwritten digits.
Classify Videos Using Deep Learning with Custom Training Loop
This example shows how to create a network for video classification by combining a pretrained image classification model and a sequence classification network.
Train Image Classification Network Robust to Adversarial Examples
This example shows how to train a neural network that is robust to adversarial examples using fast gradient sign method (FGSM) adversarial training.
Train Robust Deep Learning Network with Jacobian Regularization
Train a neural network that is robust to adversarial examples using a Jacobian regularization scheme.
Solve Ordinary Differential Equation Using Neural Network
This example shows how to solve an ordinary differential equation (ODE) using a neural network.
Assemble Multiple-Output Network for Prediction
This example shows how to assemble a multiple output network for prediction.
Train Network in Parallel with Custom Training Loop
This example shows how to set up a custom training loop to train a network in parallel.
Run Custom Training Loops on a GPU and in Parallel
Speed up custom training loops by running on a GPU, in parallel using multiple GPUs, or on a cluster.

Automatic Differentiation

Deep Learning Data Formats
Learn about deep learning data formats.
List of Functions with dlarray Support
View the list of functions that support dlarray objects.
Automatic Differentiation Background
Learn how automatic differentiation works.
Use Automatic Differentiation In Deep Learning Toolbox
How to use automatic differentiation in deep learning.

Deep Learning Function Acceleration

Deep Learning Function Acceleration for Custom Training Loops
Accelerate model functions and model loss functions for custom training loops by caching and reusing traces.
Accelerate Custom Training Loop Functions
This example shows how to accelerate deep learning custom training loop and prediction functions.
Check Accelerated Deep Learning Function Outputs
This example shows how to check that the outputs of accelerated functions match the outputs of the underlying function.
Evaluate Performance of Accelerated Deep Learning Function
This example shows how to evaluate the performance gains of using an accelerated function.

Related Information

Operations

Featured Examples

Train Generative Adversarial Network (GAN)

Train a generative adversarial network to generate images.

Custom Training Loops

Functions

Custom Networks

Custom Training Loops

Data Processing

Automatic Differentiation

Loss Operations

Function Acceleration

Topics

Custom Training Loops

Automatic Differentiation

Deep Learning Function Acceleration

Related Information

Featured Examples

Train Generative Adversarial Network (GAN)

Train Conditional Generative Adversarial Network (CGAN)

Train Wasserstein GAN with Gradient Penalty (WGAN-GP)

Generate Images Using Diffusion

Train Fast Style Transfer Network

Train a Twin Network for Dimensionality Reduction

Train a Twin Neural Network to Compare Images

Image Captioning Using Attention

Sequence-to-Sequence Translation Using Attention

Train Network in Parallel with Custom Training Loop

Train Variational Autoencoder (VAE) to Generate Images

Train Network Using Cyclical Learning Rate for Snapshot Ensembling

Solve Partial Differential Equations Using Deep Learning

Solve Partial Differential Equation with L-BFGS Method and Deep Learning

Dynamical System Modeling Using Neural ODE

Reduced Order Modeling Using Continuous-Time Echo State Network

Node Classification Using Graph Convolutional Network

Multilabel Graph Classification Using Graph Attention Networks

Train Network Using Federated Learning