Verification and Validation: Difference between revisions
No edit summary |
|||
Line 43: | Line 43: | ||
[[File:2.6-QAforAIoT.png|800px|frameless|center|link=|QA for AIoT]] | [[File:2.6-QAforAIoT.png|800px|frameless|center|link=|QA for AIoT]] | ||
= Homologation = | |||
[[File:Homologation.png|800px|frameless|center|AIoT and Homologation]] | |||
== Authors and Contributors == | == Authors and Contributors == |
Revision as of 09:34, 14 September 2021
Quality Management (QM) is responsible for overseeing all activities and tasks needed to maintain a desired level of quality. QM in Software Development traditionally has four main components: quality planning, quality assurance, and quality control. In many agile organizations, QM is becoming closely integrated with the DevOps organization. Quality Assurance (QA) is responsible for setting up the organization and its processes to ensure the desired level of quality. In an agile organization, this means that QA needs to be closely aligned with DevOps. Quality Control (QC) is responsible for the output, usually by implementing a test strategy along the various stages of the DevOps cycle. Quality Planning is responsible for setting up the quality and test plans. In a DevOps organization, this will be a continuous process. For safety and security relevant systems, Verification & Validation (V&V) usually plays an important role as well. V&V is closely related to Quality Control; it aims to link back test results to requirements.
QM for AIoT-enabled systems must take into consideration all the specific challenges of AIoT-development, including QM for combined hardware / software development, QM for highly distributed systems (including edge components in the field), as well as any homologation requirements of the specific industry. In case of critical systems, Independent Verification & Validation (IV&V) via an independent third party can be required.
Quality Assurance and AIoT DevOps
Verification & Validation
Verification and Validation for AIoT
V&V for AI
From the perspective of the integrated system, verification and validation of the AI-related services usually focuses on functional testing, considering the AI-based services a black box ("Black Box Testing"), which is tested in the context of the other services which make up the complete AIoT system. However, it will usually be very difficult to ensure a high level of quality if this the only test approach. Consequently, V&V for the AI services in an AIoT system also requires a "white box" approach, which specifically focuses on the AI-based functionality.
In AI, important techniques for Training Data Validation include the validation of data quality, completeness and representativeness. Techniques for Model Testing & Optimization include K-fold Cross-Validation, Holdout Method, Hyper Parameter Tuning, Statistical Validation Methods and AI Model Simulations. Field tests and production tests can also provide critical feedback to the V&V team of the AI services.
Integrated V&V for AIoT
On the service-level, AI services can usually be tested using the methods outlined in the previous section. After the initial tests performed by the AI service team, it is important that the AI services will be integrated into the overall AIoT product for real-world integration tests. This means that the AI services are integrated with the remaining IoT services, to build the full AIoT system. This is shown in the figure following. The fully integrated system can then be used for User Acceptance Tests, load and scalability tests, and so on.
Homologation
Authors and Contributors
|