M
"

Directory

Introduction

Glossary

Input

Data Framework

Performance Assurance

Scenario

Create

Format

Store

Environment

Query & Concretise

Allocate

Execute

Safety Argument

Coverage

Test Evaluate

Safety Case

Decide

Audit

Use Cases

SAF Solutions

Submit Feedback

SUNRISE Safety Assurance Framework

Coverage

Input Scenario Create Format Store Environment Query & Concretise Allocate Execute Safety Argument Coverage Test Evaluate Safety Case Decide Audit

The Coverage block evaluates whether testing has been comprehensive enough to provide reliable safety evidence on a CCAM system under test. It examines test results from multiple perspectives and derives a combined coverage outcome to determine if the testing adequately covers the system’s operational boundaries and requirements. The main coverage levels examined, are as follows:

  1. Coverage of the combined scenario set:
    • Operational Design Domain (ODD) Coverage: Checks if the combined set of scenarios sufficiently explores the operational boundary within which the system is expected to operate safely, including the identification of possible gaps.
    • Requirements Coverage: Verifies that the combined set of scenarios addresses all specified system requirements.
    • Behavior Coverage: Confirms that relevant system behaviors are tested across the combined set of scenarios.
  2. Coverage of the individual scenarios:
    • Parameter Coverage: Analyzes how concrete parameter values cover the logical parameter ranges within individual scenarios, including the identification of possible gaps.

Depending on the analysis outcomes, the Coverage block might trigger an iterative process that derives new concrete scenarios within logical scenario parameter ranges to identify failure conditions (for example parameter combinations leading to system failure). This feedback loop for continuous testing improvement, is indicated by the red arrow in the image above.

The Coverage block ultimately provides essential input to the Safety Argument block, which combines information from the Coverage block with that of the Test Evaluate block, to generate safety evidence for the CCAM system under test.

SAF Application Guidelines for 'Coverage'

By following the steps outlined below, users of the SUNRISE SAF can apply the Coverage block to ensure that the combined set of test case results provides sufficient coverage of the ODD and the relevant parameter value ranges necessary for a robust safety argumentation.

In the list below, “D” stands for Deliverable. All deliverables of the SUNRISE project can be found here.

 

  1. Review ODD coverage (D5.3 section 5.5.1):
    1. Assess Tag Coverage (D5.3 section 5.5.1.1):
      • Review how scenarios are categorised and tagged
      • Check that all important driving situations are represented
      • Verify each tag appears across different scenario types
      • Look for missing or underrepresented driving conditions
    2. Assess Time Coverage (D5.3 section 5.5.1.2):
      • Check that scenarios cover different time periods adequately
      • Identify any time gaps where no scenarios exist
      • Review scenarios that happen at the same time
      • Document any uncovered time periods
    3. Assess Actor Coverage (D5.3 section 5.5.1.3):
      • Review which vehicles and road users are included in scenarios
      • Check coverage at different distances from the test vehicle
      • Verify that vehicles in critical positions are well represented
      • Assess if important actors are missing
    4. Assess Actor-Time Coverage (D5.3 section 5.5.1.4):
      • Check that important actors appear throughout relevant time periods
      • Verify actors stay covered during their entire critical presence

 

  1. Review parameter space coverage (D5.3 section 5.5.2):
    1. Review how selected parameter values are analysed for completeness
    2. Check that statistical assumptions are properly verified
    3. Identify areas with insufficient parameter coverage
    4. Verify that synthetic scenarios fill identified gaps

 

  1. Check SAF integration (D5.3 section 3 introduction):
    1. Verify that coverage analysis supports scenario selection
    2. Check that coverage results inform the safety argumentation
    3. Ensure that coverage metrics support completeness arguments

 

  1. Final verification (D5.3 sections 5.5 and 6.5):
    1. Confirm all coverage metrics meet acceptable levels
    2. Validate that coverage gaps are properly addressed
    3. Ensure synthetic scenarios appropriately fill coverage gaps
    4. Cross-check results with implementation examples