Subgroup 2b: Test track- / real world testing
General overview test approach (“One pager”)
Brussels, 19 November 2019
Decision 4 th VMAD meeting Ottawa, Canada
During the last VMAD meeting a decision was taken regarding deliverables for the three subgroups 1a, 2a and 2b. First of all a document should be developed (“one pager”) in which the general approach for testing automated systems is described (Why, What and How) for each of the subgroups. On the basis of these principles a test case should be elaborated for validating ALKS.
Stepwise approach in testing/assessment
Assessment/Testing by authorities of an automated vehicle is expected to start by reviewing the design and verification/validation used by manufacturers (including the use of risk assessment, simulators, track test, field testing- task of subgroup 2a) with a particular focusing on known critical traffic scenarios (to be developed by subgroup 1a). Once the vehicle has passed this stage, it will enter the test track and finally a test will be taken in real traffic.
While repeatability of traffic test scenarios is limited (traffic is constantly changing and happens in an interaction where participants negotiate situations), testing on a test track will be strongly based on selected known critical scenarios (designed by subgroup 1a). On the test track the vehicle will be put “under pressure” (by adding unexpected events to the selected scenarios) to confirm whether the vehicle in its dynamic traffic task (DDT) is as safe as expected from the audit part (subgroup 2a) and is able to deliver some basic capabilities expected from an automated vehicles. It could also be used to confirm the simulation used by manufacturers.
Real world testing
Once passed the test track, the vehicle will be tested in real world traffic. The assessment will concentrate on how well the vehicle is performing in most of the traffic situations in a safe manner and how well other vehicles and operators in traffic can predict its intentions in traffic in different road and weather conditions. The ultimate goal is that the vehicle is able to keep in the traffic flow and to perform the driving task in diverse traffic environments.
Therefore the driving test route (in mixed traffic) should present a diverse selection of traffic situations typically but challenging encountered by drivers, different densities of traffic and speed areas, also transition from a specific speed area to another one. The vehicle should be able to adjust the speed, location and distance to other vehicles and participants in the traffic according to the traffic conditions and environment in order to behave in a safe manner. Special attention will be given to the vehicle behavior around the borders of the ODD (limitation of systems, role of human driver).
Assessment scale in desired behaviour
Test criteria are based on:
- Ability to deliver on the Safety concept and ability to deliver basic capabilities in particular in critical scenarios (e.g. abrupt change).
- Fluent and smooth driving and ability to modify behavior in order to prevent accidents happening in different traffic situations, environment (urban area, outside urban area, mixed traffic, more controlled environment) and weather conditions (demanding conditions; weather, congestion).
- Ability to respect traffic rules.
The test principles as mentioned above will be elaborated for ALKS.