MATLAB Functions with gpuArray Arguments

Many functions in MATLAB and other toolboxes run automatically on a GPU if you supply a gpuArray data argument. A gpuArray in MATLAB represents an array that is stored on the GPU.

A = gpuArray([1 0 1; -1 -2 0; 0 1 -1]);
e = eig(A);

Whenever you call any of these functions with at least one gpuArray as a data input argument, the function executes on the GPU. The function generates a gpuArray as the result, unless returning numeric data to the local workspace is more appropriate (for example, size). You can mix inputs using both gpuArray data and arrays stored in host memory in the same function call. gpuArray-enabled functions include the discrete Fourier transform (fft), matrix multiplication (mtimes), left matrix division (mldivide), and hundreds of others.

A = rand(10);
d = 2;
sum(gpuArray(A),d);

A = rand(10);
d = 2;
sum(A,gpuArray(d));

Check gpuArray-Supported Functions

Several MATLAB toolboxes include functions with gpuArray support. To view lists of all functions in these toolboxes that support gpuArray objects, use the links in the following table. Functions in the lists with information indicators have limitations or usage notes specific to running the function on a GPU. You can check the usage notes and limitations in the Extended Capabilities section of the function reference page. For information about updates to individual gpuArray-enabled functions, see the release notes.

For a list of functions with gpuArray support in all bat365^® products, see gpuArray-supported functions. Alternatively, you can filter by product. On the Help bar, click Functions. In the function list, browse the left pane to select a product, for example, MATLAB. At the bottom of the left pane, select GPU Arrays. If you select a product that does not have gpuArray-enabled functions, then the GPU Arrays filter is not available.

Deep Learning with GPUs

For many functions in Deep Learning Toolbox, GPU support is automatic if you have a supported GPU and Parallel Computing Toolbox. You do not need to convert your data to gpuArray. The following is a non-exhaustive list of functions that, by default, run on the GPU if available.

For more information about automatic GPU support in Deep Learning Toolbox, see Scale Up Deep Learning in Parallel, on GPUs, and in the Cloud (Deep Learning Toolbox).

For networks and workflows that use networks defined as dlnetwork (Deep Learning Toolbox) objects or model functions, convert your data to gpuArray. Use functions with gpuArray support (Deep Learning Toolbox) to run custom training loops or inference on the GPU.

Check or Select a GPU

If you have a supported GPU, then MATLAB automatically uses it for GPU computation. If you have multiple GPUs, then you can use gpuDeviceTable to examine the properties of all GPUs detected in your system. You can use gpuDevice to select one of them, or use multiple GPUs with a parallel pool. For more information, see Identify and Select a GPU Device and Run MATLAB Functions on Multiple GPUs. To check if your GPU is supported, see GPU Computing Requirements.

gpuDeviceTable

    Index           Name           ComputeCapability    DeviceAvailable    DeviceSelected
    _____    __________________    _________________    _______________    ______________

      1      "NVIDIA RTX A5000"          "8.6"               true              true      
      2      "Quadro P620"               "6.1"               true              false     

Alternatively, you can determine how many GPU devices are available, inspect some of their properties, and select a device to use from the MATLAB® desktop. On the Home tab, in the Environment area, select Parallel > Select GPU Environment.

Use MATLAB Functions with the GPU

This example shows how to use gpuArray-enabled MATLAB functions to operate with gpuArray objects. You can check the properties of your GPU using the gpuDevice function.

gpuDevice

ans = 
  CUDADevice with properties:

                      Name: 'Quadro P620'
                     Index: 2
         ComputeCapability: '6.1'
            SupportsDouble: 1
     GraphicsDriverVersion: '511.79'
               DriverModel: 'WDDM'
            ToolkitVersion: 11.2000
        MaxThreadsPerBlock: 1024
          MaxShmemPerBlock: 49152 (49.15 KB)
        MaxThreadBlockSize: [1024 1024 64]
               MaxGridSize: [2.1475e+09 65535 65535]
                 SIMDWidth: 32
               TotalMemory: 2147287040 (2.15 GB)
           AvailableMemory: 1615209678 (1.62 GB)
               CachePolicy: 'balanced'
       MultiprocessorCount: 4
              ClockRateKHz: 1354000
               ComputeMode: 'Default'
      GPUOverlapsTransfers: 1
    KernelExecutionTimeout: 1
          CanMapHostMemory: 1
           DeviceSupported: 1
           DeviceAvailable: 1
            DeviceSelected: 1

Create a row vector that repeats values from -15 to 15. To transfer it to the GPU and create a gpuArray object, use the gpuArray function.

X = [-15:15 0 -15:15 0 -15:15];
gpuX = gpuArray(X);
whos gpuX

  Name      Size            Bytes  Class       Attributes

  gpuX      1x95              760  gpuArray              

To operate with gpuArray objects, use any gpuArray-enabled MATLAB function. MATLAB automatically runs calculations on the GPU. For more information, see Run MATLAB Functions on a GPU. For example, use diag, expm, mod, round, abs, and fliplr together.

gpuE = expm(diag(gpuX,-1)) * expm(diag(gpuX,1));
gpuM = mod(round(abs(gpuE)),2);
gpuF = gpuM + fliplr(gpuM);

Plot the results.

imagesc(gpuF);
colormap(flip(gray));

If you need to transfer the data back from the GPU, use gather. Transferring data back to the CPU can be costly, and is generally not necessary unless you need to use your result with functions that do not support gpuArray.

result = gather(gpuF);
whos result

  Name         Size            Bytes  Class     Attributes

  result      96x96            73728  double              

In general, running code on the CPU and the GPU can produce different results due to numerical precision and algorithmic differences between the GPU and CPU. Answers from the CPU and GPU are both equally valid floating point approximations to the true analytical result, having been subjected to different roundoff behavior during computation. In this example, the results are integers and round eliminates the roundoff errors.

Examples Using GPUs

Acknowledgments

MAGMA is a library of linear algebra routines that take advantage of GPU acceleration. Linear algebra functions implemented for gpuArray objects in Parallel Computing Toolbox leverage MAGMA to achieve high performance and accuracy.