AIoT Framework

Technical execution must ensure delivery of the AIoT-enabled product or solution in close alignment with the business execution. In the software world, this would usually be managed with an agile approach to ensure continuous value creation and improvement. However, in the AIoT world, we usually face a number of impediments that will prevent a pure agile setup. These impediments exist because of the typical complexity and heterogeneity of an AIoT system, including hardware, software, and AI development. In addition, an AIoT system usually includes components that have to be "first time right" because they cannot be changed after the Start of Production (especially hardware-based components or functionally relevant system components). Designing the system and the delivery organization in a way that maximizes those areas where continuous improvement can be applied while also efficiently supporting those areas where this is not possible is one of the key challenges of the technical execution.

The technical execution part of the Digital Playbook defines an AIoT Framework which looks at ways of supporting this. This starts with looking again at the data, AI, IoT, Digital Twin and hardware perspective from the AIoT 101 chapter, but this time with the technical execution perspective ("*.exe").

In addition, this section provides a set of good practices and templates for the design of AIoT-enabled products and solutions, the implementation of an agile approach for AIoT (including the so-called "Agile V-Model"), AIoT DevOps (including cloud DevOps, MLops and DevOps for IoT), Trust & Security, Reliability & Resilience, Functional Safety, and Quality Management. Before going into detail, the following provides an overview of how all of these fit together, starting with the development life-cycle perspective.

Development Life-cycle Perspective

The development lifecycle of an AIoT-enabled product or solution usually includes a number of different sub-elements, which need to be brought together in a meaningful way. The following will discuss this for both products and solutions.

Smart, Connected Products

Smart, connected products usually combine two types of features: physical and digital. The physical features are enabled by physical elements and mechanical mechanisms. The digital features are supported by sensors and actuators as the interface to the physical product, as well as edge and cloud-based components. Digital features can be realized as hardware, software or AI.

This means that the development life-cycle of a smart, connected product must include physical product development as well as manufacturing engineering. The development lifecycle of digital features focuses on DevOps for the edge components (including MLops for the AI deployed to the edge, DevOps for embedded and edge software, and embedded/edge hardware), as well as the cloud (including MLops for cloud-based AI and standard DevOps for cloud-based software).

All of this must be managed with a holistic Product Lifecycle Management approach. In most cases, this will require the integration of a number of different processes and platforms. For example, the development life cycle of the physical features is traditionally supported by an engineering PLM platform, while software development is supported through a CI/CT/CD pipeline (Continuous Integration, Continuous Testing, Continuous Deployment). For AI, these kinds of pipelines are different and not yet as sophisticated and mature as in the software world. The following will describe how such a holistic lifecycle can be supported.

Topics closely related to this include Cyber Physical Systems (CPS), as well as mechatronics. Mechatronics is an interdisciplinary engineering approach that focuses on the integration of mechanical, electronic and electrical engineering systems. The term CPS is sometimes used in the embedded world, sometimes with a similar meaning as IoT: integrate sensing and control as well as computation and networking into physical assets and infrastructure. Both concepts and the related development life-cycles can support smart, connected products.

Smart, Connected Solutions

For smart, connected solutions supporting the Digital Equipment Operator, the picture looks slightly different since physical product development is usually not within our scope. Sensors, actuators and edge nodes are usually deployed to existing assets in the field by using a retrofit approach. This means that the holistic lifecycle in this case does not include physical product design and manufacturing engineering. Other than this, it looks similar to the product perspective, expect that usually the required development pipelines will not be as sophisticated and highly automated as in the case of standardized product development (which typically invests more in these areas).

Designing Smart Connected Products and Solutions with AIoT

An important element in the development lifecycle is the end-to-end design of the product or solution. The design section will provide a set of detailed templates that can be used here. These templates support the key viewpoints developed by the Digital Playbook: Business Viewpoint, UX Viewpoint, Data/Functional Viewpoint, and Implementation Viewpoint. These design viewpoints must be aligned with the agile product development perspective, in particular the story map as the top-level work breakdown. They will have to be updated frequently to reflect any learning from the implementation sprints. This means that they can only have a level of detail that permits them to do this.

AIoT Pipelines

Pipelines have become an important concept in many development organizations, especially from a DevOps perspective. This section introduces the concept of AIoT pipelines and discusses pipeline aggregations.

Definition

There are a number of different definitions for the pipeline concept. On the technical level, a good example is the popular development support tool git, which provides a set of tools to allow flexible creation of pipelines to automate the continuous integration process. On the methodological level, for example, the Scaled Agile Framework (SAFe) introduces the concept of Continuous Delivery Pipelines (CDP) as the automation workflows and activities required to move a new piece of functionality from ideation to release. A SAFe pipeline includes Continuous Exploration (CE), Continuous Integration (CI), Continuous Deployment (CD), and Release on Demand. This makes sense in principle.

The Digital Playbook is also based on the concept of pipelines. An AIoT pipeline helps move a new functionality through the cycle from ideation and design to release, usually in a cyclic approach, meaning that the released functionality can enter the same pipeline at the beginning to be updated in a subsequent release. The assumption is that AIOT pipelines are usually bound to a particular AIoT technical platform, e.g., edge AI, edge SW, cloud AI, cloud SW, smartphone apps, etc. Each AIoT pipeline usually has an associated pipeline team with skills specific to the pipeline and the target platform.

Pipeline Aggregations

Due to the complexity of many AIoT initiatives, it can make sense to logically aggregate pipelines. This is something that many technical tools with built-in pipeline support such as git are providing out of the box. From the point of view of the target platform, the aggregation concept also makes sense. Take, for example, an edge pipeline that aggregates edge AI components, edge software components, and potentially even custom edge hardware into a higher-level edge component. On the organizational level, this can mean that a higher-level pipeline organization aggregates a number of pipeline teams. For example, the edge pipeline team consists of an edge AI and an edge software team.

This way of looking at an organization can be very helpful to manage complexity. It is important to note that careful alignment of the technical and organizational perspectives is required. Usually, it is best to create a 1:1 mapping between technical pipelines, target platforms and pipeline teams.

The diagram below shows an edge pipeline that aggregates three pipelines, namely edge AI, edge HW and edge SW. The combined output of the three lower-level pipelines is combined into integrated edge components.

AIoT Pipelines & Feature-driven Development

Technical pipelines are useful for managing and -- at least partially -- automating the creation of new functionalities within a single technology platform. However, many functional features in an AIoT system will require support from components on a number of different platforms. Take, for example, the function to activate a vacuum robot via the smartphone. This feature will require components on the smartphone, the cloud and the robot itself. Each of these platforms is managed by an individual pipeline. It is now important to orchestrate the development of the new feature across the different pipelines involved. This is best done by assigning new features to feature teams, which work across pipelines and pipeline teams. There are a number of different ways this can be done, e.g., by making the pipeline teams the permanent home of technology experts in a particular domain and then creating virtual team structures for the feature teams that get the required experts from the technical pipelines teams assigned for the duration of the development of the particular feature. Another approach can be to permanently establish the feature teams and look at the technical pipeline teams more as a loose grouping. Unfortunately, different technology stacks and cross-technology features tend to require dealing with some kind of organizational matrix structure, which must be addressed one way or another. There are some examples of how other organizations are looking at this, e.g., the famous Spotify model. The Digital Playbook does not make any assumptions about how this is addressed in detail but recommends the combination of pipelines/pipelines teams on the one hand, and features/features teams on the other.

Jan Bosch is Professor at Chalmers University and Director of the Software Center: There are two different ways in which you're going to organize. In the component-based organizational model, you have the overall system architecture and assign teams to the different components and subsystems. The alternative model is a feature teams model; you have teams pick work items from the backlog. That team can then touch any component in the system and make all the changes they need to make to deliver their features. That is, in general, my preferred approach, but it is an important caveat. The companies that do this in an embedded systems context are associating the required skills typically with work items in the backlog. They say whatever team picks this up has to have at least these skills to deliver on this feature successfully. So it is not that any team can pick any work item.

Holistic AIoT DevOps

The pipeline concept must be closely aligned with DevOps. DevOps is a well-established set of practices that combine software development and IT operations. In more traditional organizations, these two functions used to be in different silos, which often caused severe problems and inefficiencies. DevOps focuses on removing these frictions between development and operations teams by ensuring that developer and operations experts are working in close alignment across the entire software development lifecycle, from coding to testing to deployment.

An AIoT initiative will have to look at DevOps beyond the more or less well-established DevOps for software. One reason is that AI development usually requires a different DevOps approach and organization. This is usually referred to as MLops. Another reason is that the highly distributed nature of an AIoT system usually requires that concepts such as Over the Air Updates be included, which is another complexity usually not found in cloud-centric DevOps organizations. All of these aspects will be addressed in the AIoT DevOps section in more detail.

In addition to the DevOps focused on continuous delivery of new features and functionalities, an AIoT organization will usually also need to look explicitly at security and potentially functional safety, as well as reliability and resilience. These different aspects will have to be examined through the cloud and edge software perspective, as well as the AI perspective. The Digital Playbook builds on existing concepts such as DevSecOps (an extension of DevOps to also cover security) to address these issues specifically from an AIoT point of view.

Managing Different Speeds of Development

One of the biggest challenges in most AIoT projects is managing the different speeds of development that can usually be found. For example, hardware and manufacturing-related topics usually move much slower (i.e., months) than software or AI development (weeks). In some cases, one might even have to deal with elements that change on a daily basis, e.g., automatically retrained AI models. To address this, one must carefully consider the organizational setup. Often, it can make sense to allow these different topics to evolve at their own speed, e.g., by allowing a different sprint regime for different pipelines that produce AIoT artifacts and components at different speeds. An overview is given in the figure following. Please note that there is often no straightforward answer for dealing with AIoT elements that require either very long or very short iterations. For example, for very slow moving elements, one can choose very long sprints. Alternatively, one can have all teams work with a similar spring cadence but allow the slower moving topics to deliver non-deployable artifacts, e.g., updated planning and design documents, etc. Similarly, for very fast moving elements the strict sprint cadence might be too rigid, so it could be better to allow them to be worked on and released ad hoc. For example, like automatically retrained AI models, this makes perfect sense since for an automated process no sprint planning seems required.

However, there is a key prerequisite for this to work: dependencies between artefacts and components from the different AIoT pipelines have to be carefully managed from a dependency point of view. In general, it is OK for fast moving artefacts to depend on slower moving artefacts, but not the other way around - otherwise the evolution of the fast moving artefacts will have a negative impact on the slower moving artefacts. These dependencies can be of a technical nature (e.g., call dependencies between software components, or deployment dependencies between hardware and software) or of a more organizational nature (e.g., procurement decisions). The technical dependencies and how to deal with them will be discussed in more detail in the Data/Functional Viewpoint of the Product / Solution Design. Finally, the Agile V-Model is introduced later as an option to manage product development teams in these types of situations.

Jan Bosch from Chalmers University and the Software Center: This is a key question: How do you do a release? There are companies in the earliest development stage that do heartbeat-based releases; every component releases every third or every fourth week at the end of the agile sprints. You release all the new versions of the components simultaneously, so that is one way. However, this requires a high level of coordination between the different groups who are building different subsystems in different parts of the system. This is why many companies aim to reach a state where continuous integration and testing of the overall system is so advanced that any of the components in the system can release at any point in time, as long as they have passed the test cases. Then, the teams can start to operate on different heartbeats. Some of the leading cloud companies are now releasing multiple times a day. This should also be the goal for an AIoT system: frequent releases, early validation, less focus on dependency management between different teams.