Image Processing Applications

Augment image data to simulate variations in the image acquisition. For example, the most common type of image augmentation operations are geometric transformations such as rotation and translation, which simulate variations in the camera orientation with respect to the scene. Color jitter simulates variations of lighting conditions and color in the scene. Artificial noise simulates distortions caused by the electrical fluctuations in the sensor and analog-to-digital conversion errors. Blur simulates an out-of-focus lens or movement of the camera with respect to the scene.

Common image preprocessing operations include noise removal, edge-preserving smoothing, color space conversion, contrast enhancement, and morphology.

If you have Image Processing Toolbox™, then you can process data using these operations as well as any other functionality in the toolbox. For an example that shows how to create and apply these transformations, see Augment Images for Deep Learning Workflows.

Object Detection

Object detection data consists of an image and bounding boxes that describe the location and characteristics of objects in the image.

If you have Computer Vision Toolbox™, then you can use the Image Labeler (Computer Vision Toolbox) and the Video Labeler (Computer Vision Toolbox) apps to interactively label ROIs and export the label data for training a neural network. If you have Automated Driving Toolbox™, then you also use the Ground Truth Labeler (Automated Driving Toolbox) app to create labeled ground truth training data.

When you transform an image, you must perform an identical transformation to the corresponding bounding boxes. If you have Computer Vision Toolbox, then you can process bounding box data using the operations in the table. For an example that shows how to create and apply these transformations, see Augment Bounding Boxes for Object Detection. For more information, see Getting Started with Object Detection Using Deep Learning (Computer Vision Toolbox).

Semantic Segmentation

Semantic segmentation data consists of images and corresponding pixel labels represented as categorical arrays.

If you have Computer Vision Toolbox, then you can use the Image Labeler (Computer Vision Toolbox) and the Video Labeler (Computer Vision Toolbox) apps to interactively label pixels and export the label data for training a neural network. If you have Automated Driving Toolbox, then you also use the Ground Truth Labeler (Automated Driving Toolbox) app to create labeled ground truth training data.

When you transform an image, you must perform an identical transformation to the corresponding pixel labeled image. If you have Image Processing Toolbox, then you can preprocess pixel label images using the functions in the table and any other toolbox function that supports categorical input. For an example that shows how to create and apply these transformations, see Augment Pixel Labels for Semantic Segmentation. For more information, see Getting Started with Semantic Segmentation Using Deep Learning (Computer Vision Toolbox).

Lidar Processing Applications

Lidar Toolbox™ enables you to design, analyze, and test lidar systems. You can perform object detection and tracking, semantic segmentation, shape fitting, and registration. Raw point cloud data from lidar sensors requires basic processing before you can use them for these advanced workflows.

Lidar Toolbox provides tools to perform preprocessing such as downsampling, filtering, aligning, and extracting features from point cloud data. You can also augment and transform point clouds to increase the diversity of your training data.

Use Lidar Viewer (Lidar Toolbox) app to visualize, analyze and measure point cloud data. You can preprocess data by using the built-in preprocessing algorithms or import a custom algorithm. For more information, see Create Custom Preprocessing Workflow with Lidar Viewer (Lidar Toolbox).

You can create labeled ground truth training data by using the Lidar Labeler (Lidar Toolbox) app. For more information on automated labelling, see Automate Ground Truth Labeling For Vehicle Detection Using PointPillars (Lidar Toolbox).

size(ptCloudUnorg.Location)

ans = 1×2

       37879           3

ptCloudOrg = pcorganize(ptCloudUnorg,params);
size(ptCloudOrg.Location)

ans = 1×3

          64        1024           3

Signal Processing Applications

Signal Processing Toolbox™ enables you to denoise, smooth, detrend, and resample signals. You can augment training data with noise, multipath fading, and synthetic signals such as pulses and chirps. You can also create labeled sets of signals by using the Signal Labeler (Signal Processing Toolbox) app and the labeledSignalSet (Signal Processing Toolbox) object. For an example that shows how to create and apply these transformations, see Waveform Segmentation Using Deep Learning.

Wavelet Toolbox™ and Signal Processing Toolbox enable you to generate 2-D time-frequency representations of time series data that you can use as image inputs for signal classification applications. For an example, see Classify Time Series Using Wavelet Analysis and Deep Learning. Similarly, you can extract sequences from signal data to use as input for LSTM networks. For an example, see Classify ECG Signals Using Long Short-Term Memory Networks (Signal Processing Toolbox).

Communications Toolbox™ expands on signal processing functionality to enable you to perform error correction, interleaving, modulation, filtering, synchronization, and equalization of communication systems. For an example that shows how to create and apply these transformations, see Modulation Classification with Deep Learning.

You can process signal data using the functions in the table as well as any other functionality in each toolbox.

Audio Processing Applications

Audio Toolbox™ provides tools for audio processing, speech analysis, and acoustic measurement. Use these tools to extract auditory features and transform audio signals. Augment audio data with randomized or deterministic time scaling, time stretching, and pitch shifting. You can also create labeled ground truth training data by using the Signal Labeler (Signal Processing Toolbox) app. You can process audio data using the functions in this table as well as any other functionality in the toolbox. For an example that shows how to create and apply these transformations, see Augment Audio Dataset (Audio Toolbox).

Audio Toolbox also provides MATLAB^® and Simulink^® support for pretrained audio deep learning networks. Locate and classify sounds with YAMNet and estimate pitch with CREPE. Extract VGGish or OpenL3 feature embeddings to input to machine learning and deep learning systems. The Audio Toolbox pretrained networks are available in Deep Network Designer. For a YAMNet example, see Transfer Learning with Pretrained Audio Networks in Deep Network Designer.

ans = struct with fields:
                mfcc: [1 2 3 4 5 6 7 8 9 10 11 12 13]
           mfccDelta: [14 15 16 17 18 19 20 21 22 23 24 25 26]
      mfccDeltaDelta: [27 28 29 30 31 32 33 34 35 36 37 38 39]
    spectralCentroid: 40
               pitch: 41

Text Analytics

Text Analytics Toolbox™ includes tools for processing raw text from sources such as equipment logs, news feeds, surveys, operator reports, and social media. Use these tools to extract text from popular file formats, preprocess raw text, extract individual words or multiword phrases (n-grams), convert text into numerical representations, and build statistical models. You can process text data using the functions in this table as well as any other functionality in the toolbox. For an example showing how to get started, see Prepare Text Data for Analysis (Text Analytics Toolbox).