Preprocess Volumes for Deep Learning
Read Volumetric Data
Supported file formats for volumetric image data include MAT-files, Digital Imaging and Communications in Medicine (DICOM) files, and Neuroimaging Informatics Technology Initiative (NIfTI) files.
Read volumetric image data into an ImageDatastore
. Read volumetric pixel label data into a PixelLabelDatastore
(Computer Vision Toolbox). For more information, see Datastores for Deep Learning.
The table shows typical usages of imageDatastore
and
pixelLabelDatastore
for each of the supported file formats.
When you create the datastore, specify the FileExtensions
name-value
argument as the file extensions of your data. Specify the ReadFcn
property as a function handle that reads data of the file format. The
filepath
argument specifies the path to the files or folder
containing image data. For pixel label images, the additional
classNames
and pixelLabelID
arguments specify
the mapping of voxel label values to class names.
Image File Format | Create Image Datastore or Pixel Label Datastore |
---|---|
MAT | volds = imageDatastore(filepath, ... "FileExtensions",".mat","ReadFcn",@(x) fcn(x)); pxds = pixelLabelDatastore(filepath,classNames,pixelLabelID, ... "FileExtensions",".mat","ReadFcn",@(x) fcn(x)); fcn
is a custom function that reads data from a MAT file. For example,
this code defines a function called matRead that
loads volume data from the first variable of a MAT file. Save the
function in a file called matRead.m .
function data = matRead(filename) inp = load(filename); f = fields(inp); data = inp.(f{1}); end |
DICOM volume in single file |
volds = imageDatastore(filepath, ... "FileExtensions",".dcm","ReadFcn",@(x) dicomread(x)); pxds = pixelLabelDatastore(filepath,classNames,pixelLabelID, ... "FileExtensions",".dcm","ReadFcn",@(x) dicomread(x)); For more information about reading DICOM files, see
|
DICOM volume in multiple files | Follow these steps. For an example, see Create Image Datastore Containing Single and Multi-File DICOM Volumes (Image Processing Toolbox).
|
NIfTI | volds = imageDatastore(filepath, ... "FileExtensions",".nii","ReadFcn",@(x) niftiread(x)); pxds = pixelLabelDatastore(filepath,classNames,pixelLabelID, ... "FileExtensions",".nii","ReadFcn",@(x) niftiread(x)); For
more information about reading NIfTI files, see |
Pair Image and Label Data
To associate volumetric image and label data for semantic segmentation, or two
volumetric image datastores for regression, use a randomPatchExtractionDatastore
(Image Processing Toolbox). A random patch extraction datastore
extracts corresponding randomly-positioned patches from two datastores. Patching is a
common technique to prevent running out of memory when training with arbitrarily large
volumes. Specify a patch size that matches the input size of the network and, for memory
efficiency, is smaller than the full size of the volume, such as 64-by-64-by-64
voxels.
You can also use the combine
function to associate two datastores. However, associating two datastores using a
randomPatchExtractionDatastore
has some benefits over combine
.
randomPatchExtractionDatastore
supports parallel training, multi-GPU training, and prefetch reading. Specify parallel or multi-GPU training using theExecutionEnvironment
name-value argument oftrainingOptions
. Specify prefetch reading using theDispatchInBackground
name-value argument oftrainingOptions
. Prefetch reading requires Parallel Computing Toolbox™.randomPatchExtractionDatastore
inherently supports patch extraction. In contrast, to extract patches from aCombinedDatastore
, you must define your own function that crops images into patches, and then use thetransform
function to apply the cropping operations.randomPatchExtractionDatastore
can generate several image patches from one test image. One-to-many patch extraction effectively increases the amount of available training data.
Preprocess Volumetric Data
Deep learning frequently requires the data to be preprocessed and augmented. For example, you may want to normalize image intensities, enhance image contrast, or add randomized affine transformations to prevent overfitting.
To preprocess volumetric data, use the transform
function. transform
creates an altered form of a datastore, called
an underlying datastore, by transforming the data read by the
underlying datastore according to the set of operations you define in a custom function.
Image Processing Toolbox™ provides several functions that accept volumetric input. For a full list
of functions, see 3-D
Volumetric Image Processing (Image Processing Toolbox). You can also preprocess volumetric images using
functions in MATLAB® that work on multidimensional arrays.
The custom transformation function must accept data in the format returned by the
read
function of the underlying datastore.
Underlying Datastore | Format of Input to Custom Transformation Function |
---|---|
ImageDatastore | The input to the custom transformation function depends on
the
For more information, see the |
PixelLabelDatastore | The input to the custom transformation function depends on
the
For more information, see the |
RandomPatchExtractionDatastore | The input to the custom transformation function must be a table with two columns. For more information, see
the |
The transform
function must return data that matches the input
size of the network. The transform
function does not support
one-to-many observation mappings.
To apply random affine transformations to volumetric data in
RandomPatchExtractionDatastore
, you must use the
transform
function. The DataAugmentation
property of this datastore does not support volumetric data.
Examples
Transform Batch of Volumetric Data in Image Datastore
This example shows how to transform volumetric data in an image datastore using a sample image preprocessing pipeline.
Specify a set of volumetric images saved at MAT files.
filepath = fullfile(matlabroot,"toolbox","images","imdata","mristack.mat"); files = [filepath; filepath; filepath];
Create an image datastore that stores multiple volumetric images. Specify that the ReadSize
of the datastore is greater than 1. Specify a custom read function, matRead
. This function is defined in the Supporting Functions section of this example.
volDS = imageDatastore(files,FileExtensions=".mat", ... ReadSize=3,ReadFcn=@(x) matRead(x));
Specify the input size of the network.
inputSize = [128 128];
Preprocess the volumetric images in volDS
using the custom preprocessing pipeline defined in the preprocessVolumetricIMDS
supporting function.
dsTrain = transform(volDS,@(x) preprocessVolumetricIMDS(x,inputSize));
Read a batch of data.
minibatch = read(dsTrain)
minibatch=3×1 cell array
{128x128x21 uint8}
{128x128x21 uint8}
{128x128x21 uint8}
Supporting Functions
The matRead
function loads volume data from the first variable of a MAT file.
function data = matRead(filename) inp = load(filename); f = fields(inp); data = inp.(f{1}); end
The preprocessVolumetricIMDS
function performs the desired transformations of data read from an underlying image datastore. Because the read size of the image datastore is greater than 1, the function must accept a cell array of image data. The function loops through each read image and transforms the data according to this preprocessing pipeline:
Randomly rotate the image about the z-axis.
Resize the volume to the size expected by the network.
Create a noisy version of the image with Gaussian noise.
Return the image in a cell array.
function batchOut = preprocessVolumetricIMDS(batchIn,inputSize) numRows = size(batchIn,1); batchOut = cell(numRows,1); for idx = 1:numRows % Perform randomized 90 degree rotation about the z-axis imRotated = imrotate3(batchIn{idx,1},90*(randi(4)-1),[0 0 1]); % Resize the volume to the size expected by the network imResized = imresize(imRotated,inputSize); % Add zero-mean Gaussian noise with a normalized variance of 0.01 imNoisy = imnoise(imResized,"gaussian",0.01); % Return the preprocessed data batchOut(idx) = {imNoisy}; end end
Transform Volumetric Data in Random Patch Extraction Datastore
This example shows how to transform pairs of volumetric data in a random patch extraction datastore using a sample image preprocessing pipeline.
Specify two sets of volumetric images saved at MAT files. Each set contains five volumetric images.
dir = fullfile(matlabroot,"toolbox","images","imdata","BrainMRILabeled"); filesVol1 = fullfile(dir,"images"); filesVol2 = fullfile(dir,"labels");
Store each set of volumetric images in an image datastore. Specify a custom read function, matRead
. This function is defined in the Supporting Functions section of this example. Use the default ReadSize
of 1.
vol1DS = imageDatastore(filesVol1,FileExtensions=".mat",ReadFcn=@(x) matRead(x)); vol2DS = imageDatastore(filesVol2,FileExtensions=".mat",ReadFcn=@(x) matRead(x));
Specify the input size of the network.
inputSize = [128 128];
Create a random patch extraction datastore that extracts corresponding patches from the two datastores. Select three patches per image.
patchVolDS = randomPatchExtractionDatastore(vol1DS,vol2DS,inputSize,PatchesPerImage=3);
Preprocess the volumetric images in patchVolDS
using the custom preprocessing pipeline defined in the preprocessVolumetricPatchDS
supporting function.
dsTrain = transform(patchVolDS,@(x) preprocessVolumetricPatchDS(x));
Read a batch of data.
minibatch = read(dsTrain)
minibatch=15×2 table
InputImage ResponseImage
____________________ ___________________
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
{128x128x155 uint16} {128x128x155 uint8}
Supporting Functions
The matRead
function loads volume data from the first variable of a MAT file.
function data = matRead(filename) inp = load(filename); f = fields(inp); data = inp.(f{1}); end
The preprocessVolumetricPatchDS
function performs the desired transformations of data read from the underlying random patch extraction datastore. The function must accept a table. The function transforms the data according to this preprocessing pipeline:
Randomly select one of five augmentations.
Apply the same augmentation to the data in both columns of the table.
Return the augmented image pair in a table.
function batchOut = preprocessVolumetricPatchDS(batchIn) numRows = size(batchIn,1); batchOut = batchIn; % 5 augmentations: nil,rot90,fliplr,flipud,rot90(fliplr) augType = {@(x) x,@rot90,@fliplr,@flipud,@(x) rot90(fliplr(x))}; for idx = 1:numRows img = batchIn{idx,1}{1}; resp = batchIn{idx,2}{1}; rndIdx = randi(5,1); imgAug = augType{rndIdx}(img); respAug = augType{rndIdx}(resp); batchOut(idx,:) = {imgAug,respAug}; end end
See Also
trainNetwork
| imageDatastore
| pixelLabelDatastore
(Computer Vision Toolbox) | randomPatchExtractionDatastore
(Image Processing Toolbox) | transform
Related Examples
- Create Image Datastore Containing Single and Multi-File DICOM Volumes (Image Processing Toolbox)
- 3-D Brain Tumor Segmentation Using Deep Learning