Submitted by leader of SG1a
Document: VMAD-05- 17
Proposal for traffic scenarios
The development of traffic scenarios for validating automated vehicle (AV) safety.
In order for the international community to maximize the potential safety benefits of AVs, a robust safety validation framework, that can be adopted by parties of both the 1958 and the 1998 UN vehicle regulations agreements, must be established. Such a framework must provide clear direction for assessing the safety of AVs in a manner that is repeatable, objective and evidence-based, while remaining technology neutral and flexible enough to foster ongoing innovation by the automotive industry.
At this relatively early stage in the development of AVs, much of the existing literature that assesses the current state of AV development uses metrics such as miles/kilometers travelled in real-world test situations with the absence of a collision, a legal infraction, or a disengagement by the vehicle’s automated driving system (ADS).
Simple metrics such as kilometers travelled without a collision, legal infraction, or disengagement can be helpful for informing public dialogue about the general progress being made to develop AVs. Such measurements on their own however, do not provide sufficient evidence to the international regulatory community that an AV will be able to safely navigate the vast array of different situations a vehicle could reasonably be expected to encounter.
In fact, some observers have suggested that an AV would have to drive billions of miles in the real-world to experience an adequate number of situations without an incident to prove that it has a significantly better safety performance than a human driver (Kalra & Paddock, 2016). Safety validation through such testing would not be cost and time effective, nor would it be feasible to replicate the testing later on.
A scenario-based approach, by contrast, can help to systematically organize safety validation activities in an economical, objective, repeatable, and scalable manner.
Scenario-based testing, as it applies to AVs, involves reproducing specific real-world situations that exercise and challenge the capabilities of an AV to safely operate in a given operational design domain (ODD) /operational domain (OD). Scenarios include a dynamic driving task (DDT) or sequence of DDTs. The DDT can be planned (e.g., make a left turn) or unplanned (e.g., in response to another vehicle cutting in). Scenarios can also involve a wide range of elements, such as different roadway layouts; interactions with a variety of different types of road users and objects exhibiting static or diverse dynamic behaviours; and, diverse environmental conditions (among many others factors).
The use of scenarios can be applied to different testing methodologies, such as virtual simulation, closed track, and real-world testing. Together these methodologies provide a multifaceted testing architecture, with each methodology possessing its own strengths and weaknesses. As a result, some scenarios may be more appropriately tested using certain test methodologies over others.
Validation and Verification of ADS (automated driving systems) safety requires a combination of processes and tools able to assess how the different elements of the complete system contribute to the safety of its operations in real world conditions. Traffic scenarios define one set of scenarios, reflecting ODD/OD boundary conditions which is to comply with all relevant traffic rules in the country of operation and is appropriate in the current situation in executing the dynamic driving task, and not to cause any collision which is reasonably foreseeable and preventable as defined in paragraph 2.3.3 and paragraph 2.3.4., in which the vehicle’s operations are tested before its introduction in the market.
Traffic scenarios are logically derived by combining a number of relevant properties, taken from disjunct layers describing the scenario space systematically.
When validating overall performance of ADS, it is necessary to confirm that ADS shall not cause any traffic accidents resulting in injury or death that are reasonably foreseeable and preventable under traffic conditions in the real world.
Reasonably foreseeable scenarios could be identified using a number of approaches, such as empirical traffic monitoring data, naturalistic driving data, analyzing collision data, such as by law enforcement and insurance copanies crash data bases.
As a first step, boundary conditions to divide reasonably foreseeable scenarios for traffic disturbance testing into preventable and unpreventable scenarios are defined based on a suitable performance model, which by its parameters, can reflect human driver capabilities or state of the art vehicle technology. With preventable scenario's, we mean the range of scenario's where the validation should prove that automation does not result in an accident .
Besides, this common understanding as the first step for boundary condition for ADS is favorable to be harmonized internationally.