AIoT DevOps and Infrastructure: Difference between revisions

No edit summary
(Editorial Changes)
 
(22 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<imagemap>
<imagemap>
Image:2.3-DevOps.png|frameless|1000px|AIoT - DevOps and Infrastructure
Image:2.3-DevOps.png|frameless|1000px|AIoT - DevOps and Infrastructure


rect 4 0 651 128 [[AIoT_Execution_and_Delivery|More...]]
rect 4 0 651 133 [[AIoT_Framework|More...]]
rect 651 0 1023 128 [[Artificial_Intelligence|More...]]
rect 970 0 1298 133 [[AIoT_Data_Strategy|More...]]
rect 1023 0 1439 128 [[Internet_of_Things|More...]]
rect 651 0 970 133 [[Artificial_Intelligence|More...]]
rect 1439 0 1989 128 [[Digital Twin Execution|More...]]
rect 1298 0 1767 133 [[Digital_Twin_Execution|More...]]
rect 1989 0 2467 128 [[AIoT_Data_Strategy|More...]]
rect 1767 0 2095 133 [[Internet_of_Things|More...]]
rect 2764 128 3539 257 [[Agile AIoT|More...]]
rect 2095 0 2542 133 [[Hardware.exe|Hardware.exe]]
rect 2764 257 3539 390 [[Product_Architecture|More...]]
 
rect 2764 128 3539 257 [[Product_Architecture|More...]]
rect 2764 257 3539 390 [[Agile AIoT|More...]]
rect 2764 385 3539 518 [[AIoT_DevOps_and_Infrastructure|More...]]
rect 2764 385 3539 518 [[AIoT_DevOps_and_Infrastructure|More...]]
rect 2764 518 3539 651 [[Trust_and_Security|More...]]
rect 2764 518 3539 651 [[Trust_and_Security|More...]]
rect 2764 651 3539 784 [[Reliability_and_Resilience|More...]]
rect 2764 651 3539 784 [[Reliability_and_Resilience|More...]]
rect 2764 779 3539 917 [[Verification_and_Validation|More...]]
rect 2764 779 3539 917 [[Verification_and_Validation|More...]]
rect 2764 912 3539 1045 [[Homologation|More...]]


desc none
desc none
</imagemap>
</imagemap>


__NOTOC__
<s data-category="AIoTFramework"></s>


== AIoT Framework: DevOps and Infrastructure ==
The introduction of DevOps -- together with Continuous Integration/Continuous Delivery (CI/CD) -- has fundamentally changed the way software is developed, integrated, tested, and deployed. DevOps and CI/CD are key enablers of agile development. However, today's DevOps practices predominantly focus on cloud and enterprise application development. For successful AIoT products, DevOps will need to be extended to include AI and IoT.  
The introduction of DevOps - together with Continuous Integration / Continuous Delivery (CI/CD) - has fundamentally changed the way how software is developed, integrated, tested, and deployed. DevOps and CI/CD are key enablers of agile development. However, today`s DevOps practices are predominantly focusing on cloud and enterprise application development. For successfull AIoT products, DevOps will have to be extended to include AI and IoT.  
__TOC__


=== Agile DevOps for Cloud and Enterprise Applications ===
= Agile DevOps for Cloud and Enterprise Applications =
DevOps organizations break down the traditional barriers between development and operations, focusing on cross-functional teams which support all aspects of development, testing, integration and deployment. Successful DevOps organizations avoid overspecialization and instead focus on cross-training and open communication between all DevOps stakeholders.
DevOps organizations breakdown the traditional barriers between development and operations, focusing on cross-functional teams that support all aspects of development, testing, integration and deployment. Successful DevOps organizations avoid overspecialization and instead focus on cross-training and open communication between all DevOps stakeholders.


The DevOps culture is usually closely aligned with the agile culture, both are required for incremental and explorative development.  
DevOps culture is usually closely aligned with agile culture; both are required for incremental and explorative development.  


Continuous Integration / Continuous Delivery (CI/CD) emphasize automation tools that drive build and test, ultimately enabling a highly efficient and agile software life cycle. The Continuous Integration (CI) process typically requires commitment of all code changes to a central code repository. Each new check-in triggers an automated process which re-builds the system, automatically performs unit tests, and executes automated code-quality checks. The resulting software packages are deployed to a CI server, with optional notification of a repository manager.
Continuous Integration / Continuous Delivery (CI/CD) emphasize automation tools that drive building and testing, ultimately enabling a highly efficient and agile software life cycle. The Continuous Integration (CI) process typically requires commitment of all code changes to a central code repository. Each new check-in triggers an automated process that rebuilds the system, automatically performs unit tests, and executes automated code-quality checks. The resulting software packages are deployed to a CI server, with optional notification of a repository manager.


Continuous Testing (CT) goes beyond simple unit tests, and utilizes complex test suites which combine different test scripts to simulate and test complex interactions and processes.
Continuous Testing (CT) goes beyond simple unit tests, and utilizes complex test suites that combine different test scripts to simulate and test complex interactions and processes.


Finally, Continuous Delivery (CD) uses [https://en.wikipedia.org/wiki/Infrastructure_as_code Infrastructure-as-Code (IaC)] concepts to deploy updated software packages to the different test and production environments.
Finally, Continuous Delivery (CD) uses [https://en.wikipedia.org/wiki/Infrastructure_as_code Infrastructure-as-Code (IaC)] concepts to deploy updated software packages to the different test and production environments.


[[File:2.3-DevOps-Enterprise.png|800px|frameless|center|Agile DevOps for Cloud and Enterprise Applications]]
[[File:2.3-DevOps-Enterprise.png|800px|frameless|center|link=|Agile DevOps for Cloud and Enterprise Applications]]


=== Agile DevOps for AI ===
= Agile DevOps for AI: MLOps =
The introduction of AI to the traditional development process is adding many new concepts, which are create challenges for DevOps:
The introduction of AI to the traditional development process is adding many new concepts, which create challenges for DevOps:
* New roles: data scientist, AI engineer
* New roles: data scientist, AI engineer
* New artefacts (in addition to code): Data, Models
* New artefacts (in addition to code): Data, Models
* New methods / processes: AI/data-centric, e.g. „Agile CRISP-DM“, Cognitive Project Management for AI (CPMAI)  
* New methods/processes: AI/data-centric, e.g., "Agile CRISP-DM", Cognitive Project Management for AI (CPMAI)  
* New AI tools + infrastructure
* New AI tools + infrastructure


The development of AI-based systems is also introducing a number of new requirements from a DevOps perspective:
The development of AI-based systems also introduces a number of new requirements from a DevOps perspective:
* Reproduceability of models: Creating reproduceable models is a key prerequisite for a stable DevOps process
* Reproducibility of models: Creating reproducible models is a key prerequisite for a stable DevOps process
* Model validation: Validating models from a functional and business perspective is key
* Model validation: Validating models from a functional and business perspective is key
* Explainability (XAI, or 'explainable AI'): How to ensure that the results of the AI are comprehensible for humans?  
* Explainability (XAI, or 'explainable AI'): How to ensure that the results of the AI are comprehensible for humans?  
Line 50: Line 52:
* Versioning: Models, code, data
* Versioning: Models, code, data
* Lineage: Track evolution of models over time
* Lineage: Track evolution of models over time
* Security: Deliberately skewed models as new attack vector / adversarial attacks
* Security: Deliberately skewed models as new attack vector/adversarial attacks
* Monitoring and re-training: Model decay requires constant monitoring and re-training
* Monitoring and retraining: Model decay requires constant monitoring and retraining


The figure below provides an overview of how an AI-specific DevOps process can help addressing many of the issues outlined above.
The figure below provides an overview of how an AI-specific DevOps process can help in addressing many of the issues outlined above.


[[File:2.3-DevOps-AI.png|800px|frameless|center|DevOps for AI]]
[[File:2.3-DevOps-AI.png|800px|frameless|center|link=|AI DevOps]]


=== Agile DevOps for IoT ===
= Agile DevOps for IoT =
Finally, we need to look at the DevOps challenges from an IoT point of view. The main factors are:
Finally, we need to look at the DevOps challenges from an IoT point of view. The main factors are:
* OTA: [[OTA_Updates|Over-the-Air updates (OTA)]] require a completely different infrastructure and process than traditional, cloud-based DevOps approaches
* OTA: [[OTA_Updates|Over-the-Air updates (OTA)]] require a completely different infrastructure and process than traditional, cloud-based DevOps approaches
* Embedded Software & Hardware: The lifecylcle of embedded hardware and software is very different from cloud-based software. Testing and test automation are possible, but require special efforts and techniques
* Embedded Software & Hardware: The lifecycle of embedded hardware and software is very different from cloud-based software. Testing and test automation are possible, but require special efforts and techniques.


The OTA update process is described in more detail [[OTA Updates|here]]. The figure below provides a high-level overview. The OTA Update process usually comprises of three phases. During the authoring phase, new versions of the software (or AI models or other content) are created. The distribution phase is responsible for physical distribution (e.g. between different global regions) and the management of update campaigns. Finally, once arrived on the asset, the local distribution process ensures that the updates are securely deployed, and the updated system is validated.
The OTA update process is described in more detail [[OTA Updates|here]]. The figure following provides a high-level overview. The OTA Update process usually comprises three phases. During the authoring phase, new versions of the software (or AI models or other content) are created. The distribution phase is responsible for physical distribution (e.g., between different global regions) and the management of update campaigns. Finally, once arrived on the asset, the local distribution process ensures that the updates are securely deployed, and the updated system is validated.


[[File:OTA Overview.png|600px|frameless|center|OTA Overview]]
[[File:OTA Overview.png|600px|frameless|center|link=|OTA Overview]]


Looking again at the 4 quadrant DevOps overview, this time from the IoT perspective, a number of differences compared to the standard DevOps approach can be noticed:
Looking again at the four quadrant DevOps overview, this time from the IoT perspective, a number of differences compared to the standard DevOps approach can be seen:
* The agile development is structured to match the needs of an IoT organization, as discussed [[Product_Organization#OrgAndArch|here]]
* Agile development is structured to match the needs of an IoT organization, as discussed [[Product_Organization#OrgAndArch|here]]
* Continuous Integration (CI) will usually have to cover a much more diverse set of development environments, since it needs to cover cloud and embedded development
* Continuous Integration (CI) will usually have to cover a much more diverse set of development environments since it needs to cover cloud and embedded development
* Continuous Testing (CT) will have to address test automation of embedded components, e.g. by utilizing different abstraction and simulation techniques like HIL (hardware in the loop), SIL (software in the loop) and MIL (model in the loop)
* Continuous Testing (CT) will have to address the test automation of embedded components, e.g. by utilizing different abstraction and simulation techniques such as HIL (hardware in the loop), SIL (software in the loop) and MIL (model in the loop)
* Continuous Delivery (CD) will have to utilize OTA not only for the production system, but already for Quality Assurance and User Acceptance Tests
* Continuous Delivery (CD) will have to utilize OTA not only for the production system but also for Quality Assurance and User Acceptance Tests


Finally, all of the above will also have to be looked at from the perspective of [[Verification_and_Validation|Verification and Validation]].
Finally, all of the above will also have to be examined from the perspective of [[Verification_and_Validation|Verification and Validation]].


[[File:2.3-DevOps-IoT.png|800px|frameless|center|DevOps for IoT]]
[[File:2.3-DevOps-IoT.png|800px|frameless|center|link=|AIoT DevOps]]


=== Agile DevOps for AIoT ===
= Agile DevOps for AIoT =
The AIoT DevOps approach will have to combine all 3 perspectives outlined in the previous sections: Cloud DevOps, AI DevOps and IoT DevOps.
The AIoT DevOps approach will need to combine all three perspectives outlined in the previous sections: Cloud DevOps, AI DevOps and IoT DevOps.
Each of these three topics in itself is complex, and integrating the three into a single, homogeneous and highly automated DevOps approach will be one of the main challenges of each AIoT product. However, without succeeding in this effort, it will be near impossible to deliver an attractive and feature rich product which can also evolve over time - as far as the limitations of hardware deployed in the field will allow for. Utilizing OTA to evolve the software and AI deployed on the assets in the field will be a key success factor for smart, connected products in the future.
Each of these three topics in itself is complex, and integrating the three into a single, homogeneous and highly automated DevOps approach will be one of the main challenges of each AIoT product. However, without succeeding in this effort, it will be nearly impossible to deliver an attractive and feature-rich product that can also evolve over time, as far as the limitations of hardware deployed in the field will allow. Utilizing OTA to evolve the software and AI deployed on the assets in the field will be a key success factor for smart, connected products in the future.


== Authors and Contributors ==
= Authors and Contributors =
{|{{Borderstyle-author}}
{|{{Borderstyle-author}}
|{{Designstyle-author|Image=[[File:Dirk Slama.jpeg|left|100px]]|author={{Dirk Slama|Title=AUTHOR}}}}
|{{Designstyle-author|Image=[[File:Dirk Slama.jpeg|left|100px]]|author={{Dirk Slama|Title=AUTHOR}}}}
|}
|}

Latest revision as of 17:43, 27 June 2022

More...More...More...More...More...Hardware.exeMore...More...More...More...More...More...AIoT - DevOps and Infrastructure

The introduction of DevOps -- together with Continuous Integration/Continuous Delivery (CI/CD) -- has fundamentally changed the way software is developed, integrated, tested, and deployed. DevOps and CI/CD are key enablers of agile development. However, today's DevOps practices predominantly focus on cloud and enterprise application development. For successful AIoT products, DevOps will need to be extended to include AI and IoT.

Agile DevOps for Cloud and Enterprise Applications

DevOps organizations breakdown the traditional barriers between development and operations, focusing on cross-functional teams that support all aspects of development, testing, integration and deployment. Successful DevOps organizations avoid overspecialization and instead focus on cross-training and open communication between all DevOps stakeholders.

DevOps culture is usually closely aligned with agile culture; both are required for incremental and explorative development.

Continuous Integration / Continuous Delivery (CI/CD) emphasize automation tools that drive building and testing, ultimately enabling a highly efficient and agile software life cycle. The Continuous Integration (CI) process typically requires commitment of all code changes to a central code repository. Each new check-in triggers an automated process that rebuilds the system, automatically performs unit tests, and executes automated code-quality checks. The resulting software packages are deployed to a CI server, with optional notification of a repository manager.

Continuous Testing (CT) goes beyond simple unit tests, and utilizes complex test suites that combine different test scripts to simulate and test complex interactions and processes.

Finally, Continuous Delivery (CD) uses Infrastructure-as-Code (IaC) concepts to deploy updated software packages to the different test and production environments.

Agile DevOps for Cloud and Enterprise Applications

Agile DevOps for AI: MLOps

The introduction of AI to the traditional development process is adding many new concepts, which create challenges for DevOps:

  • New roles: data scientist, AI engineer
  • New artefacts (in addition to code): Data, Models
  • New methods/processes: AI/data-centric, e.g., "Agile CRISP-DM", Cognitive Project Management for AI (CPMAI)
  • New AI tools + infrastructure

The development of AI-based systems also introduces a number of new requirements from a DevOps perspective:

  • Reproducibility of models: Creating reproducible models is a key prerequisite for a stable DevOps process
  • Model validation: Validating models from a functional and business perspective is key
  • Explainability (XAI, or 'explainable AI'): How to ensure that the results of the AI are comprehensible for humans?
  • Testing and test automation: AI requires new methods and infrastructure
  • Versioning: Models, code, data
  • Lineage: Track evolution of models over time
  • Security: Deliberately skewed models as new attack vector/adversarial attacks
  • Monitoring and retraining: Model decay requires constant monitoring and retraining

The figure below provides an overview of how an AI-specific DevOps process can help in addressing many of the issues outlined above.

AI DevOps

Agile DevOps for IoT

Finally, we need to look at the DevOps challenges from an IoT point of view. The main factors are:

  • OTA: Over-the-Air updates (OTA) require a completely different infrastructure and process than traditional, cloud-based DevOps approaches
  • Embedded Software & Hardware: The lifecycle of embedded hardware and software is very different from cloud-based software. Testing and test automation are possible, but require special efforts and techniques.

The OTA update process is described in more detail here. The figure following provides a high-level overview. The OTA Update process usually comprises three phases. During the authoring phase, new versions of the software (or AI models or other content) are created. The distribution phase is responsible for physical distribution (e.g., between different global regions) and the management of update campaigns. Finally, once arrived on the asset, the local distribution process ensures that the updates are securely deployed, and the updated system is validated.

OTA Overview

Looking again at the four quadrant DevOps overview, this time from the IoT perspective, a number of differences compared to the standard DevOps approach can be seen:

  • Agile development is structured to match the needs of an IoT organization, as discussed here
  • Continuous Integration (CI) will usually have to cover a much more diverse set of development environments since it needs to cover cloud and embedded development
  • Continuous Testing (CT) will have to address the test automation of embedded components, e.g. by utilizing different abstraction and simulation techniques such as HIL (hardware in the loop), SIL (software in the loop) and MIL (model in the loop)
  • Continuous Delivery (CD) will have to utilize OTA not only for the production system but also for Quality Assurance and User Acceptance Tests

Finally, all of the above will also have to be examined from the perspective of Verification and Validation.

AIoT DevOps

Agile DevOps for AIoT

The AIoT DevOps approach will need to combine all three perspectives outlined in the previous sections: Cloud DevOps, AI DevOps and IoT DevOps.

Each of these three topics in itself is complex, and integrating the three into a single, homogeneous and highly automated DevOps approach will be one of the main challenges of each AIoT product. However, without succeeding in this effort, it will be nearly impossible to deliver an attractive and feature-rich product that can also evolve over time, as far as the limitations of hardware deployed in the field will allow. Utilizing OTA to evolve the software and AI deployed on the assets in the field will be a key success factor for smart, connected products in the future.

Authors and Contributors

Dirk Slama.jpeg
DIRK SLAMA
(Editor-in-Chief)

AUTHOR
Dirk Slama is VP and Chief Alliance Officer at Bosch Software Innovations (SI). Bosch SI is spearheading the Internet of Things (IoT) activities of Bosch, the global manufacturing and services group. Dirk has over 20 years experience in very large-scale distributed application projects and system integration, including SOA, BPM, M2M and most recently IoT. He is representing Bosch at the Industrial Internet Consortium and is active in the Industry 4.0 community. He holds an MBA from IMD Lausanne as well as a Diploma Degree in Computer Science from TU Berlin.