More...More...More...More...More...More...More...More...More...More...More...More...More...More...AIoT and Agility

Agility vs. AIoT: Impediments

With the emergence of Internet- and Cloud-based development, Agile software development has risen to become the de-factor standard for many organizations - even though there are still many (sometimes religious) debates about how to do Agile right. Software projects are usually plagued by very high levels of complexity and highly volatile requirements. Agile development is addressing this by combining collaborative efforts of self-organizing and cross-functional teams with a strong sense of customer-centricity and focus on value creation. Agile promotes continuous exploration, continuous integration and continuous deployment. Scrum - as probably the most prominent Agile method - is based on an Adaptive planning approach, which combines higher-level epics with a more detailed requirements backlog. Detailed requirements ("user stories") are typically only created for the next 1-2 sprints ahead. Successful Scrum organizations are very thorough and consequential in their Scrum rigor, while still supporting adaptive and flexible planning.

Unfortunately, in most AIoT projects, there are some impediments to a fully Agile approach, including:

  • Cultural differences between Cloud-native developers and product/manufacturing type of engineers
  • Scalability across multiple, often distributed teams (e.g. AI development, cloud and edge/embedded software development, hardware and network engineering)
  • Sourcing & external dependencies which require more documentation and long term planning
  • The integration of teams working on Artificial Intelligence, which does not yet have a proven agile method to support it
  • Hardware and embedded software engineering, which play by different rules than cloud-based software development
  • Components/features which have to be "First Time Right", e.g. because they will be deployed on assets in the field and can`t be easily changed afterwards anymore
  • Functional Safety requirements, which often do not play well with an agile approach

The following will look at each of these impediments in detail, followed by a recommendation for how to address them in an AIoT product organization.

Agile Inhibitors

Impediment 1: Cultural Incompatibility

The first impediment for the adoption of a pure agile approach in an AIoT-driven product organization are the cultural differences typically found in the heterogeneous teams which need to work together. Developers who are used to working frequent, Cloud-enabled updates have a very different to managing projects compared to manufacturing-centric engineers who know that after the SOP (Start-of-Production) any change to a physical asset after it has left the factory involves a costly and painful recall procedure. As shown in the figure below, different types of organizations typically have different types of cultures - even within the same company. Applying a one-size-fits all methods to such a cultural mix will be difficult.

Corporate Cultures and Agile Setup

Depending on the culture found in the organizational units involved in the development of the AIoT product, a matching agile approach must be chosen. Approaches like Scrum of Scrums (SoS) or Large Scale Scrum (LeSS) could be a good fit for an entrepreneurial culture which permits a "pure" agile approach. The Scaled Agile Framework (SAFe) could be a good fit for organizations which are more coming from a "Command & Control" background, often combined with elements of a more matrix-like organization.

Impediment 2: Organizational Scalability

The second impediment is organizational scalability: Most "pure" Agile methods like Scrum are working best for single, co-located teams. Chances are that an AIoT project will be larger, involving multiple teams which are dealing with the AI development, on-asset components (Edge software, embedded, hardware), the IoT network, the backend business logic (cloud or on-premise), as well as the integration of existing enterprise systems. Given the diversity of these tasks, it also seems likely that the teams and team members will be distributed across multiple locations.

Consequently, most AIoT projects will require an Agile method that can be scaled beyond a single team and potentially a single location. In the discussion above, different approaches for scaled Agile development were introduced, including Scrum of Scrums (SoS), Large Scale Scrum (LeSS) and the Scaled Agile Framework (SAFe). We will discuss later how they can be adapted to also overcome some of the other limitations of AIoT development.

Impediment 3: Sourcing & External Dependencies

Many AIoT-enabled products have a complex supply chain, often using different suppliers for cloud software, edge software, embedded hardware and software, AI, and telecommunications infrastructure. The combination of technical complexity and supply chain complexity can be difficult to manage in a purely agile setting. Sourcing can already be a complex topic in normal software projects. With the added complexities of an AIoT project, any form of Agile organization will have to be closely aligned with the sourcing process.

In some organizations, the Agile teams will have a high level of autonomy over the sourcing process. In a matrix-type of organization as described above, a central sourcing team will have a high level of influence and potentially little experience with this type of project. If the AIoT product is a large-scale, strategic product, it is very likely that procurement will dominate the first one or even two years of the development. In addition, lead times for hardware acquisition can be much longer than those of software components.

Probably the biggest challenge from an Agile perspective is that most sourcing projects require extensive documentation of requirements or even solution designs. Many procurement organizations still see fixed-price offers as the preferred way of sourcing complex solutions. The level of documentation detail needed for most fixed-price contracts runs directly diametral to the low-documentation mantra of the Agile world.

Consequently, the product team and the procurement team will have to find ways to satisfy both their needs, e.g. through creative solutions like "fixed-price agile contracts" (e.g. a time-and-material contract with a performance-based compensation component based on individual sprint performance).

Finally, another big issue are SLAs (Service Level Agreements) and warranties. The buyer typically wants this fixed as early in the development cycle as possible (ideally as part of the contract), while the supplier will have a hard time agreeing to any SLA or warranty if the requirements are not fixed.

Impediment 4: Artificial Intelligence

While AI is quickly becoming a popular element of modern product development, there are currently no well established methodologies for AI development. Some teams are applying methods like CRISP-DM, KDD, or CPMAI to AI projects. However, there is not yet a well documented and established method out there which explains how to combine agile and AI. Which is not to say that it can`t be done, but the lack of proven agile methods for AI development is certainly an inhibitor for AIoT & Agile.

Impediment 5: Hardware / Embedded

Most AIoT projects require a combination of traditional (backend) software development with some form of edge or embedded software development. In many cases, custom hardware development and manufacturing will also be a requirement.

If the edge platform is a powerful, server-like system, e.g. running Linux and C++ or even high level languages, the difference to an Internet or enterprise software project might not be as high. However, in many cases AIoT projects also require custom embedded software and hardware, which is a different animal all together, and many Agile principles can not be directly applied. Embedded software is typically very technical, e.g. focusing on hardware-specific drivers or controllers, running on specialized real-time operating systems. And hardware development is following its own rules. Some differences include:

  • Software in general is easier to change than hardware designs
  • It is not possible to change hardware after it has been manufactured
  • Higher-level hardware designs must often incorporate existing, standardized lower-level parts
  • The lead times, acquisition times and feedback loops for hardware components are much longer
  • Time to first full integration test with custom hardware usually significantly longer than with software

All of the above does not mean than an Agile mindset can not be applied to embedded hardware/software development. However, an Agile embedded approach will have to take these specifics into consideration and adapt the methodology accordingly.

Impediment 6: First Time Right

Most AIoT-enabled products have some components and/or features which have to be "First Time Right", e.g. because they will be deployed on assets in the field and can`t be easily changed afterwards anymore. This can make it very difficult to apply agile principles: In the magic project triangle, the agile approach is usually aiming to fix the time and budget side of things, while the features / scope are seen as variable. For First Time Right features, this approach does not work.

Impediment 7: Functional Safety

Finally, if the AIoT-enabled product has Functional Safety requirements, this can impose another significant impediment on agility. Many Functional Safety standards are based on a “requirements first, code later” philosophy, while Agile is focused on continuous delivery of working software, but not on extensive documentation and detailed requirements management. The Agile V-Model introduced by the AIoT Framework is addressing this by providing a bridge between both worlds.

Conclusions

Stacey Matrix

[1]

References

  1. Strategic management and organisational dynamics: the challenge of complexity. 3rd ed., Stacey RD., Prentice Hall, 2002