Main Content

Calibration and Sensor Fusion

Interactively perform lidar-camera calibration, estimate transformation matrix, and fuse data from multiple sensors

Most modern autonomous systems in applications such as manufacturing, transportation, and construction, employ multiple sensors. Sensor Fusion is the process of bringing together data from multiple sensors, such as radar sensors, lidar sensors, and cameras. The fused data enables greater accuracy because it leverages the strengths of each sensor to overcome the limitations of the others.

To understand and correlate the data from individual sensors, you must develop a geometric correspondence between them. Calibration is the process of developing this correspondence. Use Lidar Toolbox™ functions to perform lidar-camera calibration. To get started, see What Is Lidar-Camera Calibration?

You can also interactively calibrate the sensors by using the Lidar Camera Calibrator app. For more information, see Get Started with Lidar Camera Calibrator.

Lidar Toolbox also supports downstream workflows such as projecting lidar points on images, fusing color information in lidar point clouds, and transferring bounding boxes from camera data to lidar data.

Multi-sensor system

Apps

Lidar Camera CalibratorInteractively establish correspondences between lidar sensor and camera to fuse sensor data (Since R2021a)

Functions

expand all

estimateCheckerboardCorners3dEstimate world frame coordinates of checkerboard corner points in image (Since R2020b)
detectRectangularPlanePointsDetect rectangular plane of specified dimensions in point cloud (Since R2020b)
estimateLidarCameraTransformEstimate rigid transformation from lidar sensor to camera (Since R2020b)
projectLidarPointsOnImageProject lidar point cloud data onto image coordinate frame (Since R2020b)
fuseCameraToLidarFuse image information to lidar point cloud (Since R2020b)
bboxCameraToLidarEstimate 3-D bounding boxes in point cloud from 2-D bounding boxes in image (Since R2020b)
bboxLidarToCameraEstimate 2-D bounding box in camera frame using 3-D bounding box in lidar frame (Since R2021a)

Topics