M
"

Directory

High-Level Overview

Glossary

Input

Data Framework

Performance Assurance

Scenario

Create

Format

Store

Environment

Query & Concretise

Allocate

Execute

Safety Argument

Coverage

Test Evaluate

Safety Case

Decide

Audit

Use Cases

SAF Solutions

Submit Feedback

SUNRISE Safety Assurance Framework

Coverage

Input Scenario Create Format Store Environment Query & Concretise Allocate Execute Safety Argument Coverage Test Evaluate Safety Case Decide Audit

When referring to Coverage, it contains two aspects, for example ‘X is covering Y’, with this in mind bullets below provide a summary of the types of coverage considerations discussed in the SUNRISE project, followed by more detailed explanation.

  • At the scenario set level: 1) Operational Design Domain (ODD) coverage by scenario set, 2) Requirements coverage by scenario set, 3) Behaviour coverage by scenario sets
  • At individual scenario level: logical parameter covered by concrete parameter values

After execution, the data is analysed as part of the safety assessment of the system. There are two main blocks in this analysis: Coverage and Test Evaluate. The Coverage block examines the test from multiple perspectives and derives a combined coverage outcome. These perspectives might include analysing coverage of the parameter value ranges in the logical scenario using individual concrete parameter values. Based on the analysis outcome, an iterative process derives new concrete scenarios within given logical scenario’s parameter ranges to identify failure conditions (e.g., parameter combinations that lead to system failure). This iterative feedback loop is indicated by the red feedback arrow.

Another aspect of Coverage Analysis is ODD analysis. Here, the accumulated ODD features covered by the set of test scenarios are examined to determine if enough of the system’s ODD is covered. ODD is the focus because it defines the operational boundary within which the system is expected to operate safely. Thus, it is essential to thoroughly explore this claimed boundary using a diverse set of scenarios. These are just two examples that form the coverage concept, and the SUNRISE project will continue to explore this further in later stages.
 

SAF Application Guidelines for 'Coverage'

By following the steps outlined below, users of the SUNRISE SAF can apply the Coverage block to ensure that the combined set of test case results provides sufficient coverage of the ODD and the relevant parameter value ranges necessary for a robust safety argumentation.  

In the list below, “D” stands for Deliverable. All deliverables of the SUNRISE project can be found here.

 

  1. Review ODD coverage (D5.3 section 5.5.1):
    1. Assess Tag Coverage (D5.3 section 5.5.1.1):
      • Review how scenarios are categorised and tagged
      • Check that all important driving situations are represented
      • Verify each tag appears across different scenario types
      • Look for missing or underrepresented driving conditions
    2. Assess Time Coverage (D5.3 section 5.5.1.2):
      • Check that scenarios cover different time periods adequately
      • Identify any time gaps where no scenarios exist
      • Review scenarios that happen at the same time
      • Document any uncovered time periods
    3. Assess Actor Coverage (D5.3 section 5.5.1.3):
      • Review which vehicles and road users are included in scenarios
      • Check coverage at different distances from the test vehicle
      • Verify that vehicles in critical positions are well represented
      • Assess if important actors are missing
    4. Assess Actor-Time Coverage (D5.3 section 5.5.1.4):
      • Check that important actors appear throughout relevant time periods
      • Verify actors stay covered during their entire critical presence

 

  1. Review parameter space coverage (D5.3 section 5.5.2):
    1. Review how selected parameter values are analysed for completeness
    2. Check that statistical assumptions are properly verified
    3. Identify areas with insufficient parameter coverage
    4. Verify that synthetic scenarios fill identified gaps

 

  1. Check SAF integration (D5.3 section 3 introduction):
    1. Verify that coverage analysis supports scenario selection
    2. Check that coverage results inform the safety argumentation
    3. Ensure that coverage metrics support completeness arguments

 

  1. Final verification (D5.3 sections 5.5 and 6.5):
    1. Confirm all coverage metrics meet acceptable levels
    2. Validate that coverage gaps are properly addressed
    3. Ensure synthetic scenarios appropriately fill coverage gaps
    4. Cross-check results with implementation examples