Line 79: Line 79:


[[File:Pipelines Feature Teams.png|800px|frameless|center|link=|AIoT Features]]
[[File:Pipelines Feature Teams.png|800px|frameless|center|link=|AIoT Features]]
= Agile V-Model =
Another concept that is introduced by the AIoT Playbook to better address the specific requirements of an AIoT organization is the Agile V-Model. This model combines elements of the traditional V-Model with agile methods. The V-Model assumes a V-like development approach, where the V has two legs: the left leg starts with high-level requirements, planning, architectural de-composition, and development, followed by component and system integration on the up-moving leg on the right side. The problem with many V-Model projects is that they are assuming a waterfall-like approach and only go through the V from start to system release. The Agile V-Model assumes an agile, sprint-based cadence, where each sprint goes through a lightweight version of the V-Model. This makes especially sense for AIoT, where multiple technical components must be developed independently and then be integrated. This will usually require more planning and Verification and Validation than a normal agile software project, which is why the Agile V-Model approach seems to be ideally suited. The section on [[Agile_AIoT|Agile AIoT]] provides a more detailed discussion of this concept and the motivation for it.
[[File:Agile V-Model.png|800px|frameless|center|link=|Agile V-Model]]


= AIoT DevOps =
= AIoT DevOps =

Revision as of 17:15, 14 September 2021

More...More...More...More...More...Hardware.exeMore...More...More...More...More...More...Overview of Ignite AIoT Framework


Technical Execution

The technical execution must ensure delivery of the AIoT-enabled product or solution in close alignment with the business execution. In the software world, this would usually be managed with an agile approach to ensure continuous value creation and improvement. However, in the AIoT world, we usually face a number of impediments which will prevent a pure agile setup. These impediments exist because of the typical complexity and heterogeneity of an AIoT system, including hardware, software, and AI development. In addition, an AIoT system usually includes components which have to be "first time right" because they can`t be changed after the Start of Production (especially hardware-based components or functionally relevant system components). Designing the system and the delivery organization in a way which maximizes those areas where continuous improvement can be applied while also efficiently supporting those areas where this is not possible is one of the key challenges of the technical execution.

Consequently, the technical execution part of the AIoT Playbook looks at ways of supporting this. This starts with looking again at the data, AI, IoT, Digital Twin and hardware perspective from the AIoT 101 section, but this time with the technical execution perspective ("*.exe").

In addition, this section provides a set of good practices and templates for the design of AIoT-enabled product and solutions, the implementation of an agile approach for AIoT (including the so-called "Agile V-Model"), AIoT DevOps (including cloud DevOps, MLops and DevOps for IoT), Trust & Security, Reliability & Resilience, Functional Safety, and Quality Management. Before going into the details, the following provides an overview of how all of this fits together, starting with the development life-cycle perspective.

Development life-cycle perspective

The development life-cycle of an AIoT-enabled product or solution usually includes a number of different sub-elements, which need to be brought together in a meaningful way. The following will discuss this for both products and solutions.

Smart, connected products

Smart, connected products usually combine two types of features: physical and digital. The physical features are enabled by physical elements and mechanical mechanisms. The digital features are supported by sensors and actuators as the interface to the physical product, as well as edge and cloud-based components. The digital features can be realized as hardware, software or AI.

This means that the development life-cycle of a smart, connected product must include physical product development as well as manufacturing engineering. The development life-cycle of the digital features focuses on DevOps for the edge components (including MLops for the AI deployed to the edge, DevOps for embedded and edge software, and embedded/edge hardware), as well as the cloud (including MLops for the cloud-based AI, and standard DevOps for cloud-based software).

All of this must be managed with a holistic Product Lifecycle Management approach. In most cases, this will require integration of a number of different processes and platforms. For example, the development life-cycle of the physical features is traditionally supported by an engineering PLM platform, while software development is supported through a CI/CT/CD pipeline (Continuous Integration, Continuous Testing, Continuous Deployment). For AI, these kinds of pipelines are different and not yet as sophisticated and mature as in the software world. The following will describe how such a holistic lifecycle can be supported.

Lifecycle - Product Perspective

Topics closely related to this include Cyber Physical Systems (CPS), as well as mechatronics. Mechatronics is an interdisciplinary engineering approach which focuses on the integration of mechanical, electronic and electrical engineering systems. The term CPS is sometimes used in the embedded world, sometimes with a similar meaning like IoT: integrate sensing and control as well as computation and networking into physical assets and infrastructure. Both concepts and the related development life-cycles can support smart, connected products.

Smart, connected solutions

For smart, connected solutions supporting the Digital Equipment Operator, the picture looks slightly different, since the physical product development is usually not within our scope. Sensors, actuators and edge nodes are usually deployed to existing assets in the field by using a retrofit approach. This means that the holistic life-cycle in this case does not include physical product design and manufacturing engineering. Other than this, it looks similar to the product perspective, expect that usually the required development pipelines will not be as sophisticated and highly automated as in the case of standardized product development (which typically invests more in these areas).

Lifecycle - Solution Perspective

AIoT design

An important element in the development life-cycle is the end-to-end design of the product or solution. The design section will provide a set of detailed templates which can be used here. These template are supporting the key viewpoints developed by the AIoT Playbook: Business Viewpoint, UX Viewpoint, Data / Functional Viewpoint, and Implementation Viewpoint. These design viewpoints must be aligned with the agile product development perspective, in particular the story map as the top level work breakdown. They will have to be updated frequently to reflect any learning from the implementation sprints and the continuous improvement. TBD: Which means that they can only have a level of which permits to do this.

AIoT Design Viewpoints

AIoT pipelines

Pipelines have become an important concept in many development organizations, especially from a DevOps perspective. This section introduces the concept of AIoT pipelines and also discusses pipeline aggregations.

Definition

There are a number of different definitions for the pipeline concept. On the technical level, a good example is the popular development support tool git, which provides a set of tools to allow flexible creation of pipelines to automate the continuous integration process. On the methodological level, for example, the Scaled Agile Framework (SAFe) introduces the concept pf Continuous Delivery Pipelines (CDP) as the automation workflows and activities required to move a new piece of functionality from ideation to release. A SAFe pipeline includes Continuous Exploration (CE), Continuous Integration (CI), Continuous Deployment (CD), and Release on Demand. This makes sense in principle.

The AIoT Playbook is also based on the concept of pipelines. An AIoT pipeline helps moving a new functionality through the cycle from ideation and design to release, usually in a cyclic approach - meaning that the released functionality can enter the same pipeline at the beginning to get updated in a subsequent release. The assumption is that AIOT pipelines are usually bound to a particular AIoT technical platform, e.g. edge AI, edge SW, cloud AI, cloud SW, smart phone apps, etc. Each AIoT pipeline usually has an associated pipeline team with skills specific to the pipeline and the target platform.

AIoT Pipeline - Definition

Pipeline aggregations

Due to the complexity of many AIoT initiatives, it can make sense to logically aggregate pipelines. This is something that many technical tools with built-in pipeline support such as git are providing out of the box. From the point of view of the target platform, the aggregation concept also makes sense. Take, for example, an edge pipeline, which aggregates edge AI components, edge software components, and potentially even custom edge hardware into a higher-level edge component. On the organizational level, this can mean that a higher-level pipeline organization aggregates a number of pipeline teams. For example, the edge pipeline team consists of an edge AI and an edge software team.

This way of looking at an organization can be very helpful to manage complexity. It is important to notice that it requires careful alignment of the technical and organizational perspectives. Usually it is best to create a 1:1 mapping between technical pipelines, target platforms and pipeline teams.

The diagram below shows an edge pipeline which aggregates three pipelines, namely edge AI, edge HW and edge SW. The combined output of the three lower-level pipelines is combined into integrated edge components.

AIoT Pipelines Aggregates

AIoT pipelines & feature-driven development

Technical pipelines are useful for managing and -- at least partially -- automating the creation of new functionalities within a single technology platform. However, many functional features in an AIoT system will require support from components on a number of different platforms. Take, for example, the function to activate a vacuum robot via the smart phone. This feature will require components on the smart phone, the cloud and the robot itself. Each of these platforms is managed by an individual pipeline. It is now important to orchestrate the development of the new feature across the different pipelines involved. This is best done by assigning new features to feature teams, which work across pipelines and pipeline teams. There are a number of different ways this can be done, e.g. by making the pipeline teams the permanent home of technology experts in a particular domain, and then creating virtual team structures for the feature teams which get the required experts from the technical pipelines teams assigned for the duration of the development of the particular feature. Another approach can be to permanently establish the feature teams, and look at the technical pipeline teams more as a loose grouping. Unfortunately, different technology stacks and cross-technology features tend to require dealing with some kind of organizational matrix structure, which must be addressed one way or another. There are some examples of how other organizations are looking at this, e.g. the famous Spotify model. The AIoT Playbook does not make any assumptions about how this is addressed in detail, but recommends the combination of pipelines / pipelines teams on the one hand, and features / features teams on the other.

AIoT Features

AIoT DevOps

Finally, the pipeline concept must be closely aligned with DevOps. DevOps is a well established set of practices which combine software development and IT operations. In more traditional organizations, these two functions used to be in different silos, which often was causing severe problems and inefficiencies. DevOps is focusing on removing these frictions between development and operations teams by ensuring that developer and operations experts are working in close alignment across the entire software development life-cycle, from coding to testing to deployment.

An AIoT initiative will have to look at DevOps beyond the more or less well established DevOps for software. One reason is that AI development usually requires a different DevOps approach and organization. This is usually referred to as MLops. Another reasons is that the highly distributed nature of an AIoT system usually requires that concepts such as Over the Air Updates are included, which is another complexity usually not found in cloud-centric DevOps organizations. All of these aspects will be addressed in the AIoT DevOps section in more detail.

In addition to the DevOps focused on continuous delivery of new features and functionalities, an AIoT organization will usually also need to look explicitly at security and potentially also functional safety, as well as reliability and resilience. These different aspects will have to be looked at through the cloud and edge software perspective, as well as the AI perspective. The AIoT Playbook is building on existing concepts like DevSecOps (an extension of DevOps to also cover security) to address these issues specifically from an AIoT point of view.

AIoT Pipelines + DevOps