The V-Model is a software development method often found in areas with high requirements on safety and security, which are common in highly regulated areas. Combining the traditional V-Model with a disciplined agile approach promises to allow as much agility as possible, while addressing the issues often found in AIoT initiatives: complex dependencies, different speeds of development, and the "first time right" requirements of those parts of the system which can`t be updated anymore after the Start of Production (SOP).
Foundation: V-Model
The V-model is a systems development lifecycle which has verification and validation "built in". It is often used for development of mission critical systems, e.g. in automotive, aviation, energy and military applications. It also tends to be used in hardware-centric domains. The V-model - not surprisingly - is using a v-shaped visual representation, where the left side of the "V" represents the decomposition of requirements, as well as the creation of system specifications ("definition and decomposition"). The right side of the "V" represents the integration and testing of components. Moving up on the right side, testing usually starts with the basic verification (e.g. unit tests, then integration tests), followed by validation (e.g. user acceptance tests).
When applying the V-model to AIoT, it needs to take different dimensions into consideration, usually including hardware, software, AI and networking. In addition to the standard verification tests (unit tests, integration tests) and validation tests (user acceptance and usability tests), the V-model for AIoT also needs to address interoperability testing, performance testing, scalability testing and reliability testing. The highly distributed nature of AIoT systems of will pose specific challenges here.
Test automation is key to ensure a high level of test efficiency and test coverage. On the software-side, there are many tools and techniques available to support this. In the AI-world, these kinds of tools and techniques are only just beginning to emerge, which makes it likely that a more custom approach will be required. In the embedded and hardware world, simulation techniques like Hardware-in-the-Loop (HIL), Software-in-the-Loop (SIL) and Model-in-the-Loop (MIL) are well established. However, most AIoT products will also require testing of the actual physical product, and how it performs in the field in different types of environments. This again will be a challenge, and some ingenuity will be required to automate testing of physical products wherever possible.
Agile V-Model
The AIoT framework is trying to strike a good balance between the agile software world and the less agile world of often safety-critical, complex and large(r)-scale AIoT product development with hardware and potentially even manufacturing elements to it. Therefore it is important to understand how an agile method works well together with a V+V-centric approach like the V-model. The logical consequence is the Agile V-model. Combining agile development with the V-model is no contradiction. They both can work very well together, as shown in the figure below:
- Agile methods use story maps including epics, themes, features and user stories for logical decomposition. This maps well to the left side of the V
- Continuous Integration / Continous Test / Continuous Delivery are inherently agile methods, which map well to the right side of the V
- The key assumption is that the V-model is not used like one large waterfall approach. Instead, the Agile V-model must ensure that the sprints themselves will become Vs according to the V-model
There are two options to implement the latter:
- Each sprint becomes a complete V, including development and integration / test
- The agile schedule introduces the concept of dedicated integration sprints.
- One V becomes 2 sprints: One development sprint, one integration sprint
- There are pros and cons to both approaches
- Complexity and scale of the project will surely play a role in determining the best setup
For most projects / product teams, it is recommended that development and integration are combined in a single sprint. Only for projects with a very high level of complexity and dependencies - e.g. between components developed by different organizations - it is recommended to alternate between development and integration sprints. The latter approach is likely to add inefficiencies to the development process, but might be the only way to effectively deal with alignment across organizational boundaries.
Story Map + DoD Component Architecture User Stories + AC Mapping Coding / Doing Component Integration Verification System Integration Validation
Production
Example: The ACME Vacuum Robot
To illustrate the use of the Agile V-Model, a realistic example is discussed in the following. This example is a robot vacuum cleaning system, which combines a smart, connected vaccum robot with a cloud-based backend, as well as a smart app for control. The example will provide a general introduction first, before discussing an example sprint in the system development in more detail.
Vacuum Robot Systems
As introduced earlier, modern robot vacuum cleaner are very intelligent, connected products. Even the most basic versions provide collision, wheel, brush and cliff sensors. More advanced versions use visual sensor combined with a VSLAM algorithm (Visual Simultaneous Location and Mapping). The optical system can identify landmarks on the ceiling, as well as judge the distance between walls. The most advanced systems utilize LIDAR technology (Light Detection and Ranging) to map out their environment, identify room layouts and obstacles, and as input for computing efficient routes and cleaning methods. For example, the robot can decide to make a detour vs. switching into the build-in „climb over obstacle“-mode. Another example is the automatic activation of a „carpet boost“ mode. IoT-connectivity to the cloud enables integration with user interface technology such as smart mobile devices or smart home appliances for voice control („clean under the dining room table“). Edge AI algorithms deployed on the robot are used to control these processes in advanced models.
Story Map
According to the story map structure proposed by the AIoT Framework, the story map for the vacuum robot system includes epics and features on the top level. The epics include Human / Machine Interfaces, the actual cleaning functions, management of the maps for the areas to be cleaned, navigation / sensing, system configuration, and status / history.
Component Architecture
User Story & Acceptance Criteria
Mapping User Story to Components and Feature Team
Sprint Planning
Verification & Validation
Agile V-Model and AIoT
Taking everything together that was discussed so far for IoT and AI, as well as Agile V-Model, and combining it with the product organization discussion from the chapter on Product Organization, the complete picture looks like in the diagram below: The AIoT workstreams are synchronized by applying the Agile V-model accross all AIoT workstreams. Development and integration sprints alternate (or are integrated into single sprints), in order to ensure that at the end of each agile V-sprint, a potentially shippable increment is achieved (although in reality this will not be achievable for the hardware components - some ingenious approach still has to be applied here).
Agile V-Model, AIoT, and SOP
Finally, this discussion should conclude by mentioning the probably biggest differentiator between a "normal" cloud project and an AIoT project: The Start of Production, or SOP. This is the point in time when the mass production of the smart, connected products is starting, and they are leaving the factory to be deployed in the field. Any required changes to the hardware configuration of the product will now be extremely difficult to be achieved (usually involving costly product recalls, which nobody wants). In a world where manufacturers want to utilize the AIoT to constantly stay in contact with their products after they have reached the customer, and provide the customer with new digital services and applications - rolled out via OTA - it becomes extremely important to understand that the V&V process does not stop with the SOP - at least not for anything that is software. And with software defined vehicles, software defined robots and software defined everything, this is fast becoming the new normal.
