bat365 Homepage

Automate Testing for Highway Lane Change

This example uses:

This example shows how to assess the functionality of a lane change application by defining scenarios based on requirements, and automating testing of components and the generated code for those components. The components include a lane change planner and controller. This example builds on the Highway Lane Change example.

Introduction

The highway lane change system enables the ego vehicle to automatically move from one lane to another lane on a highway. The system models longitudinal and lateral control dynamics for automated lane change. System-level simulation enables you to assess the functionality of the system-level test bench model. You can configure simulations to test scenarios based on system requirements. Automatically running these simulations enables regression testing to verify system-level functionality.

The Highway Lane Change example shows how to design and simulate a system-level model for lane change using ground truth information. This example shows how to automate the testing of that model against multiple scenarios by using Simulink® Test™. The scenarios are based on system-level requirements. In this example, you:

Review requirements — The requirements describe system-level test conditions. Create simulation test scenarios to represent these conditions.
Review the test bench model — Review the system-level lane-change test bench model that contains metric assessments. These metric assessments integrate the test bench model with Simulink Test for automated testing.
Disable runtime visualizations — Disabling runtime visualizations reduces the execution time for automated testing.
Automate testing — Configure a test manager to simulate each test scenario, assess success criteria, and report results. Explore the results dynamically in the test manager and export them to a PDF for external reviewers.
Automate testing with generated code — Configure the lane change planner and controller components to generate C++ code. Run automated testing on the generated code and verify the behavior.
Automate testing in parallel — Increase overall execution speed for running tests by using parallel computing on a multicore computer.

Review Requirements

To explore the test bench model, load the highway lane change project.

openProject("HighwayLaneChange");

Requirements Toolbox™ enables you to author, analyze, and manage requirements within Simulink. This example contains 15 test scenarios, with high-level testing requirements defined for each scenario. Open the requirement set.

open("HighwayLaneChangeTestRequirements.slreqx")

Alternatively, you can open the file from the Requirements tab of the Requirements Manager app in Simulink.

Each row in this file specifies the testing requirements of a lane-change system test scenario in textual and graphical formats. These scenarios enable you to test the lane change planner and controller components.

scenario_LC_01_SlowMoving — Three-lane straight road scene with a slow-moving lead vehicle in the ego lane.
scenario_LC_02_SlowMovingWithPassingCar — Three-lane straight road scene with a slow-moving lead vehicle in the ego lane and a passing car in the left lane.
scenario_LC_03_DisabledCar — Three-lane straight road scenario with a disabled vehicle in the ego lane.
scenario_LC_04_CutInWithBrake — Three-lane straight road scene with a lead car cutting into the ego lane with a brake.
scenario_LC_05_SingleLaneChange — Four-lane straight road scene with multiple vehicles in the scenario and a slow-moving lead vehicle.
scenario_LC_06_DoubleLaneChange — Four-lane straight road scene with multiple vehicles in the scenario, including a slow-moving lead vehicle and a fast-moving rear vehicle.
scenario_LC_07_RightLaneChange — Three-lane straight road scene with a lead car in the ego lane and a passing vehicle in the left lane that causes a right-lane change trigger for the ego vehicle.
scenario_LC_08_SlowmovingCar_Curved — Curved road scene with a slow-moving lead car in the ego lane.
scenario_LC_09_CutInWithBrake_Curved — Curved road scene with a lead car cutting into the ego lane.
scenario_LC_10_SingleLaneChange_Curved — Curved road scene with multiple vehicles in the scenario and a slow-moving lead vehicle.
scenario_LC_11_MergingCar_HighwayEntry — Highway entry scene with three vehicles in the scenario. One of the vehicles is merging into the ego lane from the highway-entry road segment.
scenario_LC_12_CutInCar_HighwayEntry — Highway entry scene with four vehicles in the scenario. One of the vehicle cuts into the ego lane on the curved segment of the road network.
scenario_LC_13_DisabledCar_Ushape — U-shaped scene with multiple vehicles in the scenario, including a disabled car in the adjacent lane of the ego vehicle and a slow-moving vehicle in the ego lane.
scenario_LC_14_DoubleLaneChange_Ushape — U-shaped scene with multiple vehicles in the scenario, including a disabled car in the adjacent lane of the ego vehicle and a slow-moving vehicle in the ego lane.
scenario_LC_15_StopnGo_Curved — Curved road scene that contains six vehicles in the scenario. The lead vehicle slows down while other vehicles travel in adjacent lanes.

Review Test Bench Model

This example reuses the HighwayLaneChangeTestBench model from the Highway Lane Change example. Open the test bench model.

open_system("HighwayLaneChangeTestBench")

The test bench model contains these subsystems:

Scenario and Environment — Specifies the scene, vehicles, and map data used for simulation.
Planner Configuration Parameters — Specifies the configuration parameters required for the planner algorithm.
Highway Lane Change Planner — Implements the lane change planner algorithm for highways.
Lane Change Controller — Algorithm model that specifies the controller.
Vehicle Dynamics — Specifies the dynamic model for the ego vehicle.
Metrics Assessment — Assesses system-level behavior.

Configure this test bench model by using the helperSLHighwayLaneChangeSetup script. This setup script takes scenarioName as input, where scenarioName can be any one of the previously described test scenarios. To run the setup script, use this code:

scenarioName = "scenario_LC_15_StopnGo_Curved";
helperSLHighwayLaneChangeSetup(scenarioFcnName=scenarioName)

You can now simulate the model and visualize the results. For more details on the design of individual components in the test bench model, see the Highway Lane Change example.

Next, automate the simulation runs for this test bench model using Simulink Test for the different test scenarios. The Metrics Assessment subsystem enables integration of system-level metric evaluations with Simulink Test. This subsystem uses Check Static Range (Simulink) and Check Static Lower Bound (Simulink) blocks for this integration. Open the Metrics Assessment subsystem.

open_system("HighwayLaneChangeTestBench/Metrics Assessment")

The Metric Assessment subsystem outputs these metrics:

Verify Time gap — Verifies that the time gap between the ego vehicle and the lead vehicle is greater than 0.8 seconds. The time gap between the two vehicles is the ratio of the calculated headway distance to the ego vehicle velocity.

Verify No Collision — Verifies that the ego vehicle does not collide with any vehicle in the scenario at any point during the simulation.

Verify Longitudinal Jerk — Verifies that the ego vehicle has a longitudinal jerk value between –5 m/s^3 and 5 m/s^3.

Verify Lateral Jerk — Verifies that the ego vehicle has a lateral jerk value between –5 m/s^3 and 5 m/s^3.

Verify Safety — Verifies the safety of the ego vehicle with respect to the vehicles present in the current lane, left lane, and right lane. This ensures that the ego vehicle performs a safe maneuver while following the trajectory.

Disable Runtime Visualizations

The system-level test bench model opens a visualization window showing the ego vehicle, the sampled trajectories, and a capsule list in both the chase view and top view of the scenario. You can disable the visualization window to make the simulation run faster.

Configure the Visualization block to disable the visualization window.

blk = "HighwayLaneChangeTestBench/Visualization";
set_param(blk,EnableChaseView="off")
set_param(blk,EnableTopView="off")

Automate Testing

Configure the Test Manager to automate the testing of the lane change application. Open the HighwayLaneChangeMetricAssessments.mldatx test file in the Test Manager.

sltestmgr;
testFile = sltest.testmanager.load("HighwayLaneChangeMetricAssessments.mldatx");

Observe the populated test cases previously authored in this file. These tests are configured to run the model.

Each test case uses the POST-LOAD callback to run the setup script with appropriate inputs. After simulating the test case, the Test Manager uses the helperLaneChangeReport function from the CLEAN-UP callback to generate the plots for curvature, lateral deviation, heading angle, velocity, steering angle, acceleration, and jerk.

Run and Explore Results for Single Test Scenario

Test the system-level model on the scenario_LC_15_StopnGo_Curved scenario.

testSuite = getTestSuiteByName(testFile,"Test Scenarios");
testCase = getTestCaseByName(testSuite,"scenario_LC_15_StopnGo_Curved");
resultObj = run(testCase);

Generate test report after the simulation.

sltest.testmanager.report(resultObj,"Report.pdf", ...
Title="Highway Lane Change", ...
IncludeMATLABFigures=true, ...
IncludeErrorMessages=true, ...
IncludeTestResults=false, ...
LaunchReport=true)

Examine the Report.pdf file. Observe that the Test environment section shows the platform on which the test is run and the MATLAB version used for testing. The Summary section shows the outcome of the test and duration of the simulation in seconds. The Results section shows pass or fail results based on the assessment criteria. This section also shows the plots logged from the helperLaneChangeReport function.

Run and Explore Results for All Test Scenarios

Simulate the system for all the tests by using the run(testFile) command. Alternatively, you can simulate the system by clicking Play in the Test Manager app.

When the test simulations are complete, view the results for all of the tests in the Results and Artifacts tab of the Test Manager. For each test case, the Test Manager uses the Check Static Range (Simulink) and Check Static Lower Bound (Simulink) blocks in the model to visualize overall pass or fail results.

You can find the generated report in the current working directory. This report contains a detailed summary of pass or fail statuses and plots for each test case.

Verify Test Status in Requirements Editor

Open the Requirements Editor and select Display. Then, select Verification Status to see a verification status summary for each requirement. Green and red bars indicate the pass or failure, respectively, of simulation results for each test.

Automate Testing with Generated Code

The HighwayLaneChangeTestBench model enables integrated testing of the Highway Lane Change Planner and Highway Lane Change Controller components. Generating code for these components requires a Simulink Coder™ license. This workflow enables you to verify that the generated code produces expected results that match the system-level requirements throughout simulation.

Set the Highway Lane Change Planner to run in software-in-the-loop (SIL) mode.

model = "HighwayLaneChangeTestBench/Highway Lane Change Planner";
set_param(model,SimulationMode="Software-in-the-loop")

Set the Lane Change Controller to run in SIL mode.

model = "HighwayLaneChangeTestBench/Lane Change Controller";
set_param(model,SimulationMode="Software-in-the-loop")

Simulate the system for all the test scenarios by using the run(testFile) command. After the tests are complete, review the plots and results in the generated report. If you have a license for Simulink Coverage™, you can also get the code coverage analysis for the generated code in the generated report. This license also enables you to visualize the coverage results for individual test cases and aggregated coverage results.

Automate Testing in Parallel

If you have a Parallel Computing Toolbox™ license, you can configure the Test Manager to execute tests in parallel using a parallel pool. To run tests in parallel, save the models after disabling the runtime visualizations by using the save_system("HighwayLaneChangeTestBench") command. The Test Manager uses the default Parallel Computing Toolbox cluster and executes tests only on the local machine. Running tests in parallel speeds up execution and decreases the amount of time it takes to get test results. For more information on how to configure tests in parallel from the Test Manager, see Run Tests Using Parallel Execution (Simulink Test).