SUNRISE Safety Assurance Framework

Query and Concretise

So far, the Create, Format, and Store steps have covered the complete Scenario block. Within the Execute block, Query & Concretise, Allocate, and Execute stages are included. Query & Concretise takes input from the Input Layer, which contains Operational Design Domain (ODD) behaviour, external requirements, and test objectives. It then passes these requirements to the SUNRISE Federated Layer as a query and retrieves scenarios from the individual scenario databases. The scenarios returned from the Federated Layer to the Concretise stage could be either of logical or concrete scenario levels, meaning that all parameters are defined using value ranges, allowing for an infinite number of concrete scenarios to be derived from a single logical scenario. The next step is to concretise these parameter ranges into specific values and to combine these concrete scenarios with the test objectives. Once combined, the scenarios are allocated within an execution environment.

Text below provides a workflow of the Query & Concretize block.

1. Since the scenario creation and formatting occurs in individual Scenario Databases (SCDB), the initial input carrying the test objectives, ODD & behaviour requirements and external requirements will be fed directly into the ‘process for creating concrete test scenario with their test objectives’.
2. These external requirements will then be used to query into individual SCDBs via the SUNRISE federated layer, and logical or concrete scenarios will be then returned. It is at this step that scenarios hosted within SCDBs become test scenarios, because it is associated with the intended testing purposes upon the retrieval from the scenario databases.
3. If the returned scenarios are at the logical level (i.e., parameters are described in ranges)
   1. the first function of the block will create concrete scenario with concrete parameter values.
   2. the second function of the block will then combine the concrete test scenario with further test objectives.
4. If the returned scenarios are already concrete scenarios, then this block will skip the creation of the concrete scenario step, but combining it with test objectives.
5. The block will send the concrete test scenarios with their test objectives to the environment allocation block for test execution.
6. During or post execution, the Analyse block will then feed back the test outcome back to this block, the next set of concrete parameter combinations can be created. An example of such process could be incorporating an optimisation algorithm, with the intention of explore a testing objective, e.g., to explore the failure points within a logical scenarios.

Logical or concrete scenarios can be retrieved from databases connected to the SUNRISE Federated Layer through queries. These queries are constructed using tags recorded in the OpenLABEL format, which adheres to a harmonized ontology developed within the SUNRISE project. This approach ensures a unified understanding of all elements and their interrelationships across the connected databases.

To derive concrete scenarios from the logical ones for testing purposes, several sampling methodologies have been developed. These methodologies facilitate the discretization of the continuous parameter space and enable the selection of specific samples (concrete scenarios) within this space. These samples are chosen to estimate the distribution of a safety measure across the parameter space. Alternatively, these methodologies can be applied to optimize for other testing objectives, such as identifying the pass/fail boundary within the parameter space or identifying parameter subspaces.