Query & Concretise
6S: Scenarios Database, Statistic Structured Space for Smart Sampling
Provider: VEDECOM, France
Contact: laurent.durville@vedecom.fr
Description
VEDECOM has developed and validated across multiple use cases a scenario sampling tool for ADAS testing, designed to reduce the testing budget while providing statistically rigorous performance guarantees. The tool enables the inference of the true accident rate of the ADAS from a limited number of test scenarios, ensuring a controlled uncertainty level and relying on real-world driving data observations.
Key Features
- Structured critical scenario selection ensuring full behavioral coverage with controlled sample size.
- Reduces test volume, accelerates execution, and quantifies residual risk based on test outcomes.
- Statistical guarantee through absence of accidents on the critical sample, with improved precision over random sampling.
- Guides validation planning for ADAS.
How does this solution cover this SAF block?
- Efficiently covers the full behavioral space by selecting one critical scenario per zone, ensuring complete and representative coverage with minimal test effort.
- Provides statistical guarantees of safety through accident-free validation on critical samples, with precise quantification of residual risk.
- Accelerates validation by optimizing sample size, integrating concrete scenario data and statistically inferenced ones.
- Offers a rigorous operational framework for planning, executing, and justifying ADAS and autonomous system validation campaigns.
AVL CAMEO
Provider: AVL List GmbH, Austria
Contact: marijn.hollander@avl.com
Description
AVL CAMEO is a validation and verification software that supports test sequencing, AI-driven active Design of Experiments (DoE), and optimization. It provides a comprehensive frontloading solution that significantly enhances testing efficiency, reducing time, effort, and resource consumption. In ADAS testing, CAMEO enables systematic, scenario-based validation and optimization of systems through real-time test automation and intelligent data analysis.
Key Features
- Facilitates scenario-based testing for CCAM system development through automated execution of test scenarios across simulation environments, hardware-in-the-Loop (HiL) systems, and proving grounds.
- Leverages Active Design of Experiments (ADoE) and Machine Learning (ML) techniques to identify and prioritize critical scenario parameters for efficient validation.
- Guarantees reproducibility of test results, supporting regulatory compliance and consistent system performance evaluation.
How does this solution cover this SAF block?
- Automatically retrieves and executes relevant test scenarios across simulation, HiL, and real-world environments, based on predefined ODD, behavioral, and test objective inputs.
- Integration of ADoE enables intelligent sampling and concretization of logical scenarios, efficiently deriving concrete scenarios that target high-risk or under-tested areas.
- Ensures traceability and reproducibility of concrete test scenarios and their outcomes, supporting iterative scenario refinement and regulatory compliance.
- Enables closed-loop testing by using test feedback to guide future scenario selection and concretization for targeted safety validation.
AVL SCENIUS™
Provider: AVL List GmbH, Austria
Contact: heiko.scharke@avl.com
Description
The AVL SCENIUS™ is a comprehensive, end-to-end solution designed to meet the complex demands of ADAS/AD test management and safety validation. It supports the entire validation process — from managing requirements, defining and organizing tests to allocating them across diverse environments, coordinating execution, and enabling thorough result evaluation through a seamless, automated workflow.
Key Features
- Automated Testing: Significantly reduce manual effort, allowing for more efficient testing processes and quicker turnaround times through a smart variety of reliable testing approaches.
- Regulatory Compliance: Stay ahead of the curve with built-in compliance checks that ensure your system meets the latest safety standards and regulations.
- Enhanced Traceability: Track requirements, tests, and results across the entire development lifecycle, ensuring nothing falls through the cracks.
- Virtual Assessment: Leverage advanced features to conduct virtual assessments, proving safety and credibility in a controlled environment.
- Real-time Analytics: Gain insights that help you make informed decisions and quickly identify and address issues.
- Comprehensive Reporting: Generate detailed dashboards and reports on test results, helping you demonstrate system reliability and safety to stakeholders.
- Seamless Integration: Painless integration with various CI/CD – CI/CT pipelines and interfaces, reducing complexity and ensuring smooth workflows.
How does this solution cover this SAF block?
- Scenario Designer with full ASAM OpenX compatibility
- Scenario Data Manager
- Validation Project Manager
- Test Definition Manager
- Test Execution Manager
- Test Analyzer
- Reporting and Dashboarding
- Safety Assessment
- Requirement Manager
- ODD Manager
- Rich REST API to integrate in workflow and tools
Scenario Center
Provider: IKA – RWTH Aachen University, Germany
Contact: info@scenario.center
Description
Scenario.center is a scenario database that automatically builds a model of the Operational Design Domain (ODD) from traffic observations, makes the ODD systematically testable in simulation, and tracks the coverage.
Key Features
- Automatically create a scenario-based model of the ODD based on traffic observations.
- Analyse and compare traffic observations based on scenarios and their parameters.
- Estimates the scenario and scenario parameter distributions based on the observed scenarios.
- Search for specific traffic interactions and sequences of those.
- Get scenarios as ASAM OpenSCENARIO and ASAM OpenDRIVE, ready for simulation in esmini and CARLA.
How does this solution cover this SAF block?
- Create/Format/Store: Automatically analyses traffic observations and identifies situations that match scenarios. These identified scenarios are stored in a scenario database.
- Query and Concretise: The scenario database allows to query observed scenarios and to sample from conditional distributions of scenarios and their parameters (conditioned by the query).
- Coverage: With respect to the scenario concept of scenario.center, it can compute a coverage estimation.
StreetWise
Provider: TNO, The Netherlands
Contact: olaf.opdencamp@tno.nl, henk.goossens@tno.nl
Description
StreetWise is a methodology developed by TNO to use driving data to extract real-world scenarios, determine the statistics of these scenarios, and use the scenarios for assessing automated driving systems. To support the research and development of StreetWise, it comes with a concept database that contains real-world scenarios.
Key Features
- Automatically mining scenarios from real-world driving data for creation of scenario database content.
- Multivariate distributions of scenario parameter values are derived using statistical method.
- Test scenario generation from stored scenario database information, via TNO GUI or API in OpenDRIVE and OpenSCENARIO format.
- The estimated multivariate distributions can be used to determine the exposure of concrete scenarios and generate test scenarios that are not necessarily observed in real-world data.
How does this solution cover this SAF block?
- TNO StreetWise provides a methodology to create a scenario database required for the scenario-based approach, covering the Scenario block as a whole, including the Create, Format and Store sub-blocks.
- TNO StreetWise GUI and API facilitate the access between Environment – Query & Concretise and Scenario – Store.
- TNO StreetWise enables the concretisation of test scenarios covering Environment – Query & Concretise.
- Statistics derived from the created scenario database allow for coverage estimation on concrete scenario level.