Case Studies
The Use Case actual demonstrations are planned for Q1-Q3 2025, thus the relevant information will be uploaded in due course. Stay tuned !
There are 4 Use Case categories which are further distributed in 8 sub-use cases, as shown below. Please click on a UC/sub-UC title to read more.
UC1 – Urban AD validation
UC1.1 – Perception testing
In Use Case 1.1, the SUNRISE SAF is tested for perception systems in urban environments. The aim here is to test those systems in complex urban intersections/scenarios and adverse weather conditions. There are camera-based, LiDAR-based and radar-based perception systems.
There are six behaviours defined for UC1.1: Car-following, outdoor parking with other vehicles entering or leaving parking spots, obstacle avoidance due to construction work, pedestrian and bicycles, and mainly urban intersections like roundabouts and right/left turn junctions. All of them with day/night conditions and/or adverse weather conditions like fog, rain and others.
UC1.2 – Connected perception testing
In Use Case 1.2, the SUNRISE SAF is tested for cooperative perception and decision-making in urban intersection scenarios. The system under test is a cooperative Adaptive Cruise Control (ACC) that is expected to improve with enhanced perception obtained through Vehicle to Everything (V2X) connectivity.
There are four functional scenarios defined in T7.1. Three of these scenarios involve improving vehicle behaviour when approaching a traffic light and receiving traffic light phases, timings, and map information. The scenarios cover three types of unexpected events: a baseline scenario with no unexpected event, a violation by a distracted pedestrian, and a reset of phases due to a pedestrian call. The fourth scenario deals with a red-light violation by a crossing car, and in this case, V2V information complements the traffic light connection.
UC1.3 – Cooperative perception testing
In Use Case 1.3, the SUNRISE SAF is tested for cooperative perception and decision-making in urban intersection scenarios. The SUT is the perception AD subsystem of an urban chauffer ADS which is also capable of sending/receiving and processing (on-board or off-board e.g. employing a remote smart RSU node) rich V2X information data mainly consisting of object-level data perceived from other road users.
There are three functional scenarios defined from T7.1 for assessing the cooperative perception system performance in Use Case 1.3. The first is the “darting-out pedestrian” scenario (1.3 A), the second one is the “urban junction” scenario (1.3 B), and the third one is the “urban roundabout” scenario (1.3 C).
UC2 – Traffic jam AD validation
UC2.1 – Safety assessment & decision making
Use Case 2.1 involves testing the SUNRISE SAF with an AD function that controls the ego-vehicle in a traffic jam scenario. The system is expected to control both its longitudinal and lateral position by a combination of braking, and steering commands. All of this is expected to happen while respecting a few constraints, such as a maximum longitudinal speed and a distance to the vehicle in front of the subject vehicle.
UC3 – Highway AD validation
UC3.1 – Map based perception & decision making
Use Case 3.1 involves testing the SUNRISE SAF with map-based AD functions for highway scenarios. The systems under test will include an advanced ACC/HWP designed to control longitudinal dynamics based on HD map data and onboard sensors.
There are two defined functional scenarios: adapting speed to varying speed limits and varying road curvature, using information from sensors and maps. [NOTE: A third functional scenario, green driving on slopes, initially proposed in SUNRISE deliverable D7.1, has been dismissed.]
UC3.2 – Cooperative perception & decision making & control
The main aim of sub-UC 3.2 will be to demonstrate how safety could be improved on motorways by including cooperative functions in the HWP system. One example of this HWP functionality is the leveraging and upgrading of the driver assistance functionality developed previously in C-ACC from sub-UC 1.2.
By applying the SUNRISE SAF, the ODD coverage can be demonstrated, not spatially or functionally to specific operating fields and conditions. This is possible by taking advantage of different virtual validations, e.g., usage of extended realistic scenarios where cooperative manoeuvres between agents can be proven by variating all the interesting parameters, even in corner cases or in safety-critical conditions. After extensive virtual tests demonstrating the virtual ODD coverage, the designed cooperative functions can be proven in real-world scenarios regarding the main identified use cases and parameters, in order to provide a sufficient test coverage of the ODD.
UC4 – Freight vehicle automated parking validation
UC4.1 – Truck low-speed perception & decision making
Confined areas, with perimeter protections and a reduced risk of unauthorised entry, offer the advantage of a well-defined ODD. These environments and highway use cases are expected to be among the first to support deploying highly automated (L4) heavy vehicle functions, aspects that make them essential to SAF validation efforts.
The reverse parking of a truck with a semitrailer is a strong use case for validating specific aspects of the SAF due to the controlled environment and limited range of possible scenarios. Validation in such spaces benefits from lower speeds and regulated traffic conditions, although the presence of mixed traffic and humans introduces additional complexity.
UC4.1 focuses on the reverse parking manoeuvre of a truck with a semitrailer. The idea from the application phase is that the sub-UC is a reverse-driving truck with a semitrailer at a logistic hub.
UC4.2 – Truck low-speed connected perception cyber-security
In Use Case 4.2, the SUNRISE SAF is tested for truck low-speed connected perception cyber-testing. The SUT is a connected perception AD subsystem that is compromised by cyber-security threats. The main aim is to combine in a simulation environment several aspects simultaneously (physical environment, perception, V2X connectivity) and study the effects of physical or remotely executed cyber threats on collective environment awareness.
There are two functional scenarios defined from T7.1 for assessing the connected perception cyber-security performance in use case 4.2. The first is the “distorted camera input” scenario, whereas the second one is the “CPM message attacked” scenario.
The following table summarizes the SAF blocks that will be covered by each UC. Please click an ‘X’ to read more details.
The function of the federated layer in this context is to allow for queried scenarios to be selected for testing, which have been requested and verified as provided in the correct format. The federated layer is not developed at this stage. Current scenarios mimic the use of the federated layer in that the requirements for the scenarios were established in the SUNRISE deliverable D7.1 and queried to meet said requirements. The format of the scenarios is consistent as OpenSCENARIO 1.0 and OpenDRIVE files. Due to the state of readiness of the Federated DF, the scenarios were created outside but will be validated by a series of queries to the DF and subsequent analysis to ensure they align. Thus, an online SCDB (Streetwise) and another SCDB (Safety PoolTM) will be queried based on the requirements of the use case set out by T7.1 for UC 2.1 by the federated layer.
Provision of scenarios tailored to the requirements of the UC in the required common format for execution, except for V2X elements which are not supported directly in ASAM OpenSCENARIO. Despite the scenarios being created before the deployment of the Federated DF, the scenarios will be validated as though they were output from the Federated DF. Several queries will be tested based on different ODDs and analysed whether the logical scenarios used in this use case (and their parameter ranges) match the ODDs and the expectations of the experts.
Provision of scenarios tailored to the requirements of the UC in the required common format for execution, with the exception of V2X elements which are not supported directly in ASAM OpenSCENARIO. As for use case 3.1 the scenarios for use in testing were created when the federated DF was not available for use. The Federated DF will be validated by testing several queries based on different ODDs and analysed if the logical scenarios provided for the use case as an outcome (and their parameter ranges) match the ODDs and the expectations of the experts for the scenarios provided.
Creation of the scenarios defined in D7.1 in CARLA with different parametrization. The method described in D3.4 will be used to create and sample the scenarios.
Use of a knowledge-based approach to creating test cases (Euro NCAP scenarios, which are based on expert knowledge and represent an important aspect to consider for supporting ADS validation). This approach enables the demonstration of the capability of handling knowledge-driven scenarios as well, addressing the needs of many vehicle safety bodies, especially for urban-based environments. This approach actively supports the validation of this SAF block.
Usage of a sampling method described in D3.4 to create test cases related to the scenarios defined in D7.1. There will be checked resulting test parameters, behaviours of the other participants and quantities to be measured that are aligned with the provided ODD, external requirements and test objectives. Not any missing critical scenario will be also checked.
Usage of two of the test cases that are previously defined and utilization of same sampling method defined in D3.4 for parametrization.
Usage of a sampling method described in D3.4 to create test cases that can be allocated in the following block.
Provision of concrete scenarios for simulation based on the methodology developed in WP3. For the case generation, the query process is a required input and will be validated against the validation criteria defined in D7.2/chapter 2 for the logical scenarios defined in the use case. The sampling methodology will be validated based on the coverage of the parameter space by the KPI’s defined in the use case.
Definition of the logical scenario and provision of the sampling according to the SAF block.
Creation of an automatic probabilistic process to concretize the scenarios stemming from a logical scenario described by a given parameter space with defined parameter ranges. The concretization will be performed by using pass/fail criteria for the SUT and generating scenarios with uncertain outcomes close to the pass/fail boundary. The sampling of concrete scenarios will be validated by verifying that their parameter values fall within the parameter space ranges of the corresponding logical scenario. Sanity checks with respect to the handling of missing parameter values or out-of-range parameter values will be performed.
Use of hand-crafted concrete scenarios or application of a ready-to-use tool from T3.3 to concretize scenarios using some criteria. Concretization will be validated with simulation execution and verifying that each concrete scenario is successfully executed and leads to slightly different results/KPIs.
Validation of the Query & Concretise block by creating concrete scenarios from logical scenarios by using the methodology described in deliverable D3.4.
Generation of concrete scenarios for simulation based on the methodology developed in WP3. The case generation process requires input from the query process, which will be validated against the criteria outlined for the logical scenarios defined in the use case. Additionally, the sampling methodology will be evaluated by assessing how well the parameter space is covered by the KPIs specified in the use case.
Comparison of logical scenarios from the use case to internally defined scenarios.
Queries will be created to search for scenarios via the federated SCDB. The suggested scenarios will be compared to the ones defined by the experts in WP7.
The bulk of the scenarios come from translations of the abstract scenarios defined as ‘key’ in the D7.1. Whilst the provision of scenarios from an existing database such as Safety PoolTM would provide real measurement data, the novelty of the communications described in the use case prevented this. The scenario subspace methodology is in progress. The UC will test the completeness by checking whether the concrete scenarios missed any critical case (this is done by sampling the parameter space with many new randomly generated samples to verify that nothing was missed). The efficiency of the concrete scenarios will be tested with a ML surrogate model and bootstrapping aggregation method (this is done by comparing the number of concrete scenarios to the minimum number that still captures all the important features). Additionally, the obtained behaviours of participants and the quantities that must be measured can be checked for completeness.
Generation of concrete scenarios for simulation based on the methodology developed in WP3. The case generation process requires input from the query process, which will be validated against the criteria outlined in D7.2 for the logical scenarios defined in the use case. Additionally, the sampling methodology will be evaluated by assessing how well the parameter space is covered by the KPIs specified in the use case.
Comparison of logical scenarios from the use case to internally defined scenarios.
A hypothetical query, built on the SUT, ODD, and test objectives, guides the collection of initial scenario data from these experiments. This data then undergoes scenario concretisation to ensure the generated tests are efficient and complete. Validate sufficient coverage of relevant test space and test coverage to be free from critical gaps within the region of interest or across the parameter space. Since an exhaustive search is foreseen to be achieved, the effectiveness of combinatorial testing can be investigated.
Using these concretised data, test cases are generated to evaluate safety performance indicators (SPIs). Each test case specifies a concrete test scenario, including inputs, preconditions, and expected results.
Validation of the “Concretisation” part in this block by evaluating the criteria assessing scenario data completeness, data accuracy, and absence of data corruption in combination with test execution. The versioning and traceability criterion is planned to be assessed during the evaluation of the block “Test Evaluation” for the planned test data collections.
Logical scenarios were hand-written as instantiations of the functional scenarios defined in D7.1. Parameterization of these scenarios will be applied and a few concrete scenarios will be created to cover a small subset of the scenario space without attempting to fulfil coverage requirements. This UC will not focus on coverage aspects as this is already covered by the work in UC1.3 (Collective Perception testing).
Usage of virtual testing using CARLA. The process described in D3.3 for the initial allocation will be used.
Verification that a test case can be allocated to the available test instance, through knowledge-driven scenarios (Euro NCAP).
Verification that a test case can be allocated to the available test instance, through proving ground testing.
Verification that a test case can be allocated to the available test instance, through virtual testing.
Provision of an allocation strategy based on the methodology developed in WP3. The allocation strategy can be validated by comparing the results of the simulation and physical testing against the output of the allocation strategy.
Conduction of the cases that are assigned to simulations. By evaluating whether the simulations successfully generated accurate evaluation metrics, one can determine whether the simulation test instances were appropriate or if the allocation block should have indicated test instances with higher fidelity.
Definition of a list of cases based on three parameters: vehicle speed, the distances to the traffic light, and the state of the light. An essential variable is added to this: the occupancy of the crossing area.
Use of the initial allocation of the test cases to the test instances with the process described in the deliverable D3.3. All tests that can be performed using simulations will be allocated to virtual testing. However, in a second run, a few cases will be re-allocated to physical or hybrid testing.
Allocation of concrete scenarios to test instances as described in D3.3 [5] to validate defined scenarios in D7.1. Selected scenarios from the simulation, which are validated, then will be executed in the proving ground.
Application of the initial allocation process of Deliverable D3.3 and validation of the test cases according to the upcoming Deliverable D3.5. Virtual simulation and proving ground testing will be used as test instances to validate the allocation. It will be verified that test cases can be allocated to test instances and checked whether the initial allocation indicates test instances of unexpected high fidelity. Potential reallocation will be applied according to Deliverable D3.5.
Support of the allocation validation by providing the connection to the development in WP3.
Conduction of the cases that are assigned to simulations. By evaluating whether the simulations successfully generated accurate evaluation metrics, one can determine whether the simulation test instances were appropriate or if the allocation block should have indicated test instances with higher fidelity.
Allocation validation by providing the connection to the development in WP3.
Conduction of the cases that are assigned to simulations. By evaluating whether the simulations successfully generated accurate evaluation metrics, one can determine whether the simulation test instances were appropriate or if the allocation block should have indicated test instances with higher fidelity.
Application of the initial allocation process in chapters 3 and 4 of D3.3 and validation of the test cases according to the upcoming D3.5. Virtual simulation and proving ground testing will be used as test instances to validate the allocation. It will be verified that test cases can be allocated to test instances and checked whether the initial allocation indicates test instances of unexpected high fidelity. Potential reallocation will be applied according to D3.5.
Use of the initial allocation of the test cases to the test instances with the process from D3.3 “Report on the Initial Allocation of Scenarios to Test Instances”.
With two or more test instances that can execute comparable tests, evaluating the effectiveness of the test instances allocation is possible.
Usage of the framework specified in D4.4 using CARLA for camera sensors.
Description of the virtual testing setup which eventually enables to demonstrate that the execution block of the SAF is able to handle EuroNCAP scenarios as well.
Preparation for the execution of the allocated test cases on the AD prototype. This proves if all tests could be executed and if the needed signals can be obtained for further test evaluation.
Execution of allocated test cases on V&V virtual simulation framework defined in D4.4. The additional changes include a perception pipeline to meet test case requirements.
Preparation of the execution of the allocated test cases. The execution will be done by using a customised version of the harmonised V&V simulation framework described in D4.4.
Provision of physical testing for the use case on a proving ground and validation of the execution block for physical testing.
Running of the simulations and reporting whether each concrete scenario was correctly executed or if it was not completed and why.
Execution of concrete scenarios on the simulation model in the loop and with a car on Proving ground. The used simulator and the vehicle have exactly the same core functions.
Execution of allocated test cases on the V&V virtual simulation framework defined in D4.4 where two pipelines will be implemented, i) virtual testing and ii) hybrid testing with one real agent. Upon completion of sanity checks and results logs completeness checks will be performed to validate execution.
Validation of the “Execute” block in two phases, adhering to deliverables that define specific validation requirements. The primary objective of this validation is to ensure that test case results meet the input requirements of the “Test Evaluate” and “Coverage” blocks, as outlined in the SAF framework.
Demonstration of automated scenario executions in the simulation toolchain. Additionally, results will be used for the KPI calculations and test reports.
Demonstration of the execution of physical test runs in a black-box approach (based on the outcomes of T4.6), like it will be done by consumer testing or market surveillance of a regulator using the SUNRISE SAF.
Development of the SUT and support for the integration of the System into the virtual testing environment.
Run the simulations using the framework described in D7.2 section 3.5.1.1 and conduction of proving ground testing as described in D7.2 section 3.5.1.2. It will be reported whether each concrete scenario was executed correctly and the information needed in the following blocks, Test Evaluate and Decide, is available. In case of missing data (e.g., a failed simulation), the block which causes the problem will be identified.
Support in the validation of the sensor models. For the validation, the sensor information from real-world tests is compared to the simulated sensor output.
Running the simulations and reporting whether each concrete scenario was correctly executed or if it was not completed and why.
Run the simulations using the framework described in D7.2 section 3.6.1.1 and conduction of proving ground testing as described in D7.2 section 3.6.1.2. It will be reported whether each concrete scenario was executed correctly and the information needed in the following blocks, Test Evaluate and Decide, is available. In case of missing data (e.g., a failed simulation), the block which causes the problem will be identified.
Support in the validation of the sensor models. For the validation, the sensor information from real-world tests is compared to the simulated sensor output.
Run the simulations and reporting whether each concrete scenario was correctly executed or if it was not completed and why.
Tests will be performed as virtual tests (D4.4) and physical tests. Efficient orchestration requires test scenarios to be machine-readable and suitable for batch testing. A goal for efficiency is that the recorded data needed to evaluate the validity and results of tests will be collected automatically. Verifying that all results comply with the input required by the “Test Evaluate” and “Coverage” blocks, also confirming that data is free from errors related to test preparation or execution.
Support of the validation of the execution aspect in this block by planning, preparing, and conducting several test data collections with a real-scale truck according to the validation criteria as listed in D7.2 Section 2.2. In particular focus are concrete aspects such as data presence, data synchronicity, absence of data corruption, and data completeness to meet the expectations for the validation of the block “Test Evaluate”.
Tests will be performed as virtual tests using a simulation toolchain developed around CARLA. All data that is needed to evaluate the validity and results of tests will be collected automatically. Additionally, all the tests can be run, obtaining the corresponding metrics related to the test objectives.
The simulation framework presented in D7.2 Figure 42 is used to execute the scenarios and the validation criteria from D7.2 section 2.2 will be applied to verify the quality of the simulation execution.
Optionally, integration of a communication network simulator into Carla to be able to then run wireless communication-based cybersecurity tests on the truck backing function. identification and definition of relevant communication-based attacks such as jamming attacks for cybersecurity testing of the communication between the truck and the camera mounted in the logistic hub.
Developing of a backing function in Carla for the truck (connected to a trailer). Validation of the execute block by assuring that the backing function has been prepared and executed. Tests will be performed as detailed in D4.4. As part of this, we develop an orchestrator and set points to be used by the truck to perform the backing manoeuvre. The result of the test run, with respect to the backing function, will be logged so that the correctness of the backing function is evaluated in the Test Evaluate block. Validation of this block by providing a ground truth execution and logs to be used to check whether log data reveal any potential errors that occurred during test preparation or execution.
Usage of a collision and route completion metric to decide the pass/fail criteria of the scenarios. Other metrics provided by the SAF can be computed to compare.
Review of the evaluation process based on the obtained parameters on the vehicle and reception of input from the external requirements. In this use case, a good metric for analysing a perception system could be if a specific obstacle has been properly classified/detected and if the distance/ speed to the obstacle is also correct
Conduction of test evaluation necessary for the methodology and usage of the validation criteria to validate the physical tests.
Performance indicators per concrete scenario, evaluation of the KPIs according to the SAF and comparison of these evaluations with internal evaluations.
Computation of KPIs for every single test case and evaluation and comparison for both simulation and real car.
The pass/fail criteria will be defined based on popular KPIs employed in the perception testing literature and based on experts’ feedback. XiL testing will be verified w.r.t offering a different perspective than the virtual one, via recording of new KPIs for safety argumentation. Tests allocated to a hybrid testing environment (XiL) will be validated against their counterparts in simulation.
The proximity to the pass/fail boundary will be assessed which will inform on the confidence of the test evaluate output.
Use of KPIs, defined in D7.1 and D3.2, KPI calculation scripts and test report templates for both simulation and proving ground test results evaluations.
Evaluation of the test data after test execution by taking into account the metrics/KPIs defined in D7.1 and T3.5.
For each concrete scenario executed in virtual simulation or on proving ground, the metrics from the SAF for evaluating the safety performance of the HWP and the validity of the test runs will be returned. In case a metric cannot be calculated, the block which causes the problem will be identified. Based on experience, it will be checked whether the metrics from the SAF are appropriate for measuring the safety performance of the HWP and the validity of the test run.
Return of the performance indicators per concrete scenario and comparison of the SAF evaluations with internal evaluations.
Returns the performance indicators per concrete scenario and compares evaluations based on SAF with internal evaluations.
For each concrete scenario executed in virtual simulation or on proving ground, the metrics from the SAF for evaluating the safety performance of the HWP and the validity of the test runs will be returned. In case a metric cannot be calculated, the block which causes the problem will be identified. Based on experience, it will be checked whether the metrics from the SAF are appropriate for measuring the safety performance of the HWP and the validity of the test run.
UC4.1 aims firstly to evaluate the validity of the test and then the relevance of the result contribution to the safety performance indicators previously described and criteria from the upcoming deliverable D3.5. The UC will validate that it is possible to use distribution-based and rule-based indicators. The results must lend themselves to be returned to the SAF to be aggregated towards the Required test coverage.
Support the validation for this block by extracting, analysing, and providing meta-information about the executed tests relevant to the validity of the data.
Relevant pass/fail criteria will be defined for the truck parking system following guidelines from WP2.Those KPIs are automatically logged in each run along with some metadata which will allow to verify the results’ validity through KPIs analysis and visual inspection of the scenario in simulation.
Coverage of the cases in the parameter space, necessary for the methodology. By comparing against results from simulation and real-world testing we validate that the surrogate model from which the generated test cases are derived provides an approximation of the underlying distributions of the KPI’s over the parameter space. This only validates part of the “Coverage” block since scenario coverage cannot be determined.
Cross-checks with one of the following methods: a) new independent test samples or (either in the parameter space or with new parameters that were not considered in the logical scenario, b) with a surrogate ML model of the test results across the logical scenario.
Coverage of the test cases
Assession of coverage on the logical scenario level by using the probabilistic process developed in T3.3. The proposed method allows for i) estimation of the pass/ fail probability of unseen concrete scenarios without the need for execution/simulation, and ii) generation of scenarios close to the pass/fail boundary, which are the hardest to predict their outcomes (high uncertainty). The coverage of the logical scenario’s parameter space will be assessed by estimating the method’s output exhaustively for each scenario parameter instance of the parameter space (which would not be possible if simulation was required). The coverage validation will be performed by drawing scenarios close to the estimated boundary (uncertain), as well as random scenarios in the estimated pass and fail regions and evaluating them in simulation to verify pass or fail decision.
Coverage analysis by using a range of parameter values through virtual testing, following the process developed in T3.3.
Provision of results from virtual simulations and proving ground testing for validation of the coverage.
Evaluation of the coverage of cases within the parameter space required for the methodology. By comparing the results against simulations and real-world testing, we validate that the surrogate model, which generates the test cases, effectively approximates the underlying distributions of the KPIs across the parameter space. However, this validation only addresses part of the coverage block, as scenario coverage cannot be assessed.
Cross checks SAF coverage with the bootstrapping aggregation methods mentioned above, in query and concretize.
Provision of results from virtual simulations and proving ground testing for validation of the coverage.
Evaluation of the coverage of cases within the parameter space required for the methodology. By comparing the results against simulations and real-world testing, we validate that the surrogate model, which generates the test cases, effectively approximates the underlying distributions of the KPIs across the parameter space. However, this validation only addresses part of the coverage block, as scenario coverage cannot be assessed.
Comparison of SAF coverage with the ML bootstrapping aggregation method mentioned in UC3.1 (D7.2 section 3.5.2).
Ensuring that the pass/fail criteria of this block includes the criteria from EuroNCAP, demonstrating that this block of capable of handling these types of scenarios as well.
Evaluation if at that stage there is all the needed information from the ‘Coverage’ and ‘Test evaluate’ blocks to determine if the perception system is safe or not. Those metrics could be accuracy/ precision/ recall and average tracking errors.
Following the execution of virtual test cases, some metrics such as collision or time-to-collision will be used to evaluate the severity of scenarios and group them as safe or unsafe. Additionally, unsafe scenarios may be categorized under failed SAF subsystem, including perception or control blocks
Cross-check of the SAF decision with one of the methods mentioned above.
Evaluation if at that stage there is all the needed information from ‘Coverage’ and ‘Test evaluate’ to determine if the SUT is safe or if more testing is required.
To evaluate the performance of the system in the TJA Use Case, Task 3.5 will provide pass/fail criteria. These assessment criteria are KPIs, metrics and other requirements. In the EU regulation R157, on which UC 2.1 is based, the requirements are openly formulated (e.g. system must stick to the traffic rules and avoid collisions up to a specific TTC). The SAF will specify concrete evaluation metrics related to the defined test scenarios and based on the general requirements from the regulations.
Validation of the “Decide”-block after the evaluation of the test results by taking into account external requirements from the EU R157, on which UC2.1 is based.
Falsification of the SAF decision with one of the following methods: a) new independent test samples (either in the parameter space or with new parameters that were not considered in the logical scenario), b) with a surrogate ML model (bootstrapping aggregation method) of the test results across the logical scenario.
Derive a safety evaluation of the HWP from the SAF. Based on experience, these guidelines and the result will be checked for plausibility and feasibility.
Falsification of the SAF decision with one of the following methods: a) new independent test samples (either in the parameter space or with new parameters that were not considered in the logical scenario), b) with a surrogate ML model (bootstrapping aggregation method) of the test results across the logical scenario.
Derive of a safety evaluation of the HWP from the SAF. Based on experience, these guidelines and the result will be checked for plausibility and feasibility.
Validate the applicability of the guidelines from the upcoming Deliverable D2.3 on the results of the UC.
Support the validation of the applicability of the guidelines focusing in particular on the evaluation of the several real test scenarios using the real-scale truck to unveil potential improvement aspects with the goal to incorporate the experiences from the tests.
Please click an ‘X’ to read more details.
