Introduction

A highway lane following system steers a vehicle to travel within a marked lane. It also maintains a set velocity or safe distance to a preceding vehicle in the same lane. The system typically includes vision processing, sensor fusion, decision logic, and controls components. To ensure functional safety and interoperability, this system requires testing on a variety of road conditions. For example, testing on a scene that has varying shadows, lane marking types, and road materials can make it easier to identify edge cases.

RoadRunner is an interactive editor that enables you to design 3D scenes for simulating and testing automated driving systems. You can use RoadRunner to create roads, lane markings, road signs, vegetation, and scenes with varied complexities in a 3D environment.

The Highway Lane Following example that this example is based on shows how to simulate scenarios for curved and straight road scenes. This example demonstrates how to simulate scenarios with a scene created in RoadRunner. The scene contains variations in shadows, lane marking types, and road materials designed to test the impact of the vision processing on system functionality. In this example, you will:

You can use the modeling patterns and techniques used in this example to import your own scenes and test your algorithms.

In this example, you enable system-level simulation through integration with the Unreal Engine from Epic Games®. This simulation environment requires a Windows® 64-bit platform.

if ~ispc
    error(['3D Simulation is only supported on Microsoft', char(174),...
          ' Windows', char(174), '.']);
end

To ensure reproducibility of the simulation results, set the random seed.

rng(0);

This example also requires you to download the Automated Driving Toolbox™ Interface for Unreal Engine 4 Projects support package.

pathToUnrealExe = fullfile(...
    matlabshared.supportpkg.getSupportPackageRoot,...
    "toolbox","shared","sim3dprojects","driving","RoadRunnerScenes",....
    "WindowsPackage", "RRScene.exe");
if (~exist(pathToUnrealExe, 'file'))
    error('This example requires you to download and install Automated Driving Toolbox Interface for Unreal Engine 4 Projects support package');
end

Review Scene

This example includes a scene (RRHighway.rrscene) that was designed in RoadRunner. If you have RoadRunner installed, you can follow the workflow in the RoadRunner Project and Scene System (RoadRunner) topic to open the RRHighway.rrscene scene.

This scene is designed to pose challenges to the lane following system. This image shows that the road network is divided into six sections. Each section incrementally adds variations to the scene as follows:

For example, the following image shows the transition between Section-3 (with trees that do not cast shadows on the lane markings) and Section-4 (with trees that do cast shadows on lane markings).

The following image shows the transition between Section-4 (with dashed lane markings) and Section-5 (with solid lane markings).

The following image shows the transition between Section-5 (with darker road material) and Section-6 (with lighter road material).

Integrate Scene into Driving Scenario

A common motivation for importing scenes into a driving scenario is to enable the addition of vehicles and trajectories, either interactively or programmatically. You can integrate a road network from a RoadRunner scene into a driving scenario by using these steps:

Export OpenDRIVE File from RoadRunner

RoadRunner enables you to export scenes to many file formats, include OpenDRIVE®. To learn more about the OpenDRIVE export workflow, see Export to ASAM OpenDRIVE (RoadRunner). This example includes an OpenDRIVE file (RRHighway.xodr) that was exported from the RoadRunner scene (RRHighway.rrscene) using the process described in that topic.

Import OpenDRIVE File into Driving Scenario

OpenDRIVE files can be imported into cuboid driving scenarios. To learn more about this workflow, see Highway Trajectory Planning Using Frenet Reference Path.

Create a driving scenario and import the OpenDRIVE road network. For the purposes of this example, turn off the warnings from the OpenDRIVE importer.

scenario = drivingScenario;

Turn off warning from OpenDRIVE importer for example purpose

warning('off','driving:scenario:OpenDRIVEWarnings');
roadNetwork(scenario,"OpenDrive","RRHighway.xodr");

Add Vehicle and Trajectory to Driving Scenario

You can add vehicles to a scenario either programmatically or interactively. This example shows the programmatic workflow. To add vehicles interactively, use the Driving Scenario Designer app.

Add a vehicle to the road network using a set of predefined waypoints. These waypoints are attached as a supporting file, manualwaypoints.mat.

v = vehicle(scenario,"ClassID", 1);
load("manualWaypoints.mat","waypoints");
speed = 20; % m/s
trajectory(v, waypoints, speed);

Simulate Driving Scenario

Plot the driving scenario in world coordinates. Also plot the scenario from the vehicle perspective by using a chase plot.

hFigScenario = figure;
p1 = uipanel("Position",[0 0 0.5 1]);
h1 = axes("Parent",p1);
plot(scenario,"Waypoints","On","Parent",h1);
p2 = uipanel("Position",[0.5 0 0.5 1]);
h2 = axes("Parent",p2);
chasePlot(v,"Parent",h2);

Set the visibility of the figure to off.

set(hFigScenario, 'Visible', 'Off');

You can optionally continue to explore, simulate, and edit the scenario in Driving Scenario Designer drivingScenarioDesigner(scenario).

Integrate Scene into Unreal Engine Scenario

A common motivation for importing scenes into Unreal Engine is to enable simulation systems with camera, radar, and lidar sensor models. You can integrate a RoadRunner scene with an Unreal Engine driving scenario simulation using these steps:

Export Unreal Engine Scene from RoadRunner

RoadRunner enables exporting to Unreal Engine scene. To learn more about this workflow, see Export to Unreal Using Filmbox (.fbx) File (RoadRunner). The workflow includes exporting Filmbox (.fbx) and XML files, which can be imported into the Unreal Editor. After you open a scene in the Unreal Engine editor, you might want to adjust other scene aspects such as lighting.

This example uses an Unreal Engine scene (RRHighway) that was exported from the RoadRunner scene (RRHighway.rrscene).

Configure Unreal Engine Scene

The Unreal Engine scene can co-simulate with Simulink using a bat365Simulation plugin from the Automated Driving Toolbox Interface for Unreal Engine 4 Projects support package. To install the support package, follow the steps in Install Support Package for Customizing Scenes.

The Unreal Engine scene (RRHighway) used in this example has been compiled with the bat365Simulation plugin.

Create Test Bench Model

You can connect a Simulink model to the Unreal Engine for co-simulation. To learn more about this workflow, see Simulate Simple Driving Scenario and Sensor in Unreal Engine Environment. The open-loop test bench model (RRHighwayTestBench.slx) uses this workflow to connect to the Unreal Engine scene (RRHighway).

Open the test bench model for the scenario.

open_system("RRHighwayTestBench");

Opening this model runs the helperSLRRHighwaySetup script. This script configures the Simulation 3D Scene Configuration block in the RRHighwayTestBench model.

This model contains blocks that enable simulating a driving scenario with Unreal Engine.

The model also contains blocks to visualize the camera and lidar sensors. You can use the Bird's-Eye Scope to visualize radar and vision detections from a bird's-eye view. To learn how to configure this scope, see Visualize Sensor Data from Unreal Engine Simulation Environment.

Simulate Test Bench Model

Simulate the model. The vehicle follows the trajectory defined in the driving scenario.

sim("RRHighwayTestBench");

Integrate Scene into Lane Following Application

You can reuse the techniques described in the previous sections to simulate and assess a closed-loop system such as the highway lane following application. This section reuses models, helper functions, and techniques described in the Highway Lane Following example.

To explore the test bench model, load the highway lane following project and change the present working directory to example working directory.

openProject("HighwayLaneFollowing");cd ..;

Open and configure the system level test bench model.

open_system("HighwayLaneFollowingTestBench");
scenarioFcnName = "scenario_RRHighway_01_NoShadowToShadow";
helperSLHighwayLaneFollowingSetup("scenarioFcnName", scenarioFcnName);

Integrate Scene into Driving Scenario

The first argument to the helperSLHighwayLaneFollowingSetup is the name of a function that creates a driving scenario that is compatible with the HighwayLaneFollowingTestBench. The scenario_RRHighway_01_NoShadowToShadow function creates this driving scenario. It imports the RRHighway.xodr OpenDRIVE file and adds vehicles around the road segment that transitions from no shadow to shadow. You can explore this function to learn more about programmatic techniques for creating scenarios. It leverages several helper functions that you can use to simplify adding vehicle and their trajectories to the imported road network. Explore the scenario_RRHighway_01_NoShadowToShadow to know more about the helper functions and their usage.

The helperSLHighwayLaneFollowingSetup function creates a variable scenario in the base workspace. Plot this scenario and notice the region of the scene that will be simulated.

hFigScenario = figure;
p1 = uipanel("Position",[0 0 0.5 1]);
h1 = axes("Parent",p1);
plot(scenario,"Waypoints","On","Parent",h1);
p2 = uipanel("Position",[0.5 0 0.5 1]);
h2 = axes("Parent",p2);
chasePlot(scenario.Actors(1),"Parent",h2)

Set the visibility of the figure to off.

set(hFigScenario, "Visible", "Off");

Integrate Scene into Unreal Engine Scenario

The HighwayLaneFollowingTestBench model contains algorithm components for vision detection, forward vehicle sensor fusion, and controls. The Simulation 3D Scenario subsystem integrates the model with the driving scenario and the corresponding Unreal Engine game.

Open the Simulation 3D Scenario subsystem.

open_system("HighwayLaneFollowingTestBench/Simulation 3D Scenario")

Notice that this subsystem reuses the modeling techniques and blocks described previously in this example.

The primary differences are:

Simulate the model to see the ego vehicle behavior for the scenario.

sim("HighwayLaneFollowingTestBench");

The system is able to detect and follow lanes in the conditions where shadows are present. For more details on how to analyze the simulation results, refer to the Highway Lane Following example.

Close the figure.

close(hFigScenario);

Explore Additional Scenarios

This example provides additional scenarios that you can use to test the system behavior.

You can configure the model and workspace with these scenarios by using the helperSLHighwayLaneFollowingSetup function. For example, this code configures the test bench to simulate a scenario in the region where the road material changes.

helperSLHighwayLaneFollowingSetup("scenarioFcnName",...
    "scenario_RRHighway_03_DarkToLightRoadMaterial");

Enable the warnings from OpenDRIVE importer.

warning('on','driving:scenario:OpenDRIVEWarnings');

You can apply these techniques to integrate RoadRunner scenes into driving scenarios for simulation and testing of your systems.