Continuously improve your agility

Introduction

Organizations increasingly struggle with delivering and innovating their products and services at the pace their customers expect from them. Customer experiences are no longer determined within a single industry, but standards are set across formerly unrelated businesses ([Term18]). Technological developments drive the need for flexibility, customer focus and a ‘technology savvy’ workforce.

To meet these challenges, organizations focus on agility as evidenced by the Global KPMG CEO Outlook survey in 2019 titled “Agile or Irrelevant” ([KPMG19]). The shift towards agility through an Agile transformation strategy results in new operating models. These models aim to apply ‘Agility at scale’ across business units. Inspired by technology firms such as Spotify ([Knib14]) and Zappos ([Zapp19]), as well as banks such as ING ([McKi17]), organizations embrace concepts such as squads and tribes. Building on the Agile Manifesto from 2001 ([Fowl00]), an increasing number of frameworks are available, such as: SAFe, LeSS, Nexus, Scrum at scale, etc. to show organizations how to do things differently.

After a great start and lots of positive energy, we frequently see Agile transformations reach a point where the organization is no longer improving the agility, despite a strong desire to still do so.

The organization has adopted a new structure based on one or multiple frameworks, has held many awareness sessions across the organization, has visited various inspiring examples and invested heavily in strategic consulting and Agile coaching. All the while leaving the following question open: how far did we actually move towards our ambition, and how do we proceed? Measuring one’s Agile maturity is a logical step at this point to refuel the Agile roadmap. Subsequently, these insights can be easily repeated to provide the teams and stakeholders with insights on how to increase agility going forward. The periodic repetition of these measurements and sharing of these insights transparently and broadly motivate employees to take the next steps. Motivation and empowerment of the employees stimulate continuous improvement ([Cull17]), which can be regarded as the key driver and heart of an Agile transformation.

Agile maturity insights are key for progress

In the past two years, the authors of this article have helped over a dozen organizations build their Agile ambition and drive their Agile transformation. The Agile ambition is typically determined through awareness sessions detailing possibilities and implications, which usually develop over time. The ambition can range from pilots to IT development; from scaling practices through programs up to enterprise agility ([Cool18]).

After setting their ambition and sometimes after already executing the first steps into the Agile transformation, many organizations experience challenges in measuring progress and determining improvements with respect to their ambition. They basically want to fill or refuel, and prioritize the backlog of the team responsible for driving the Agile transformation. The following examples show that the challenge of progress analyses are always similar in nature regardless of the maturity of the organization:

• An organization that was recently supported by the authors is a utility company primarily active within Europe. The authors helped build the Agile ambition to initiate an Agile transformation in an entire business unit across five countries. The organization wanted to know their Agile maturity across the various countries and levels of the organization to kick-start their Agile transformation towards their ambition.
• Another organization that is active in the financial sector wanted to implement Agile in one of their newly started smaller business units, and applied insights into their Agile maturity to challenge their structure and collaboration model.
• A large government organization had an ambition to drastically increase agility and completely get rid of their demand-supply model by implementing agility at scale through new business and IT value streams. Once the transformation to agility at scale was underway, they asked the authors to determine their focal points for the coming three years, and reflect on the progress they had already made.
• A large financial organization wanted to use insights into Agile maturity to re-focus their approach of the internal audit department in performing IT audits and align them with the state of the Agile transformation within the organization. Finally, the authors worked with a large logistical company to embrace Scaled Agile.

Although all these applications of Agile maturity may differ in nature, we have learned that the challenges of the actual analysis are always similar in nature. From our experience, we have distilled five key perspectives on successfully measuring Agile maturity.

The five key perspectives on the maturity of Agile at scale

There is a large variety of methodologies available to determine the Agile maturity of an organization. However, these maturity methods mainly focus on shaping the events and artefacts of Scrum, combined with the associated technology and culture. Although these elements are crucial in an Agile transformation, the existing models frequently cover a limited number of challenges to the complete transformation and focus on team-level Scrum or a specific framework. We see the need for five perspectives on Agile maturity.

1. Don’t focus solely on team level; measure maturity at all levels in the organization

Agile maturity is not about team performance only. When the ambition is to scale agility beyond team level, it is important to recognize the various levels in the organization and the role of senior management and leadership. In our opinion, organizations should not only incorporate organizational governance across the various levels in the organization, but they should also incorporate vision, ambition and Agile strategy in any attempt to measure Agile maturity at scale.

In popular frameworks such as SAFe ([Leff19]) and LeSS ([Larm19]), as well as in the Spotify model, different layers can be identified ranging from team level to portfolio or enterprise level. Most of the popular scaling frameworks focus on the value stream from epic owner to realization in the teams through three levels. SAFe has added the “Large Solution” layer to facilitate the creation of very large and complex solutions (e.g. airplanes), which require many release trains at program level. In line with scaling frameworks, epics are typically broken down across levels, for example from epic to capability to feature and user stories. The role of business unit leadership (if not assigned a role of an epic owner) or even management board is typically not shown as a layer in the framework. However, evidently these stakeholders have an important role when building an ambition towards agility at scale or enterprise agility, requiring the transformation into an Agile organization. From a senior leadership perspective, aspects such as Agile transformation strategy and approach, sponsorship, awareness and Agile leadership become important.

Figure 1. The four levels of SAFe 4.6. [Click on the image for a larger image]

In most cases, we see organizations recognizing the difference between the portfolio level, program (scaling of teams) level and team level itself. For example, a Dutch logistics company and a large governmental organization – where the authors were recently involved in conducting Agile maturity scans – both chose portfolio, cluster and team levels.

In line with the SAFe framework – which also has specific SAFe maturity measurements across three levels – we emphasize the need to measure maturity not only at team level, but also across all levels. The measurement should at least cover the three aforementioned levels, while explicitly including the pivotal role of senior leadership in the transformation.

2. Consider the value stream as a whole (customer to operations and suppliers)

Agility is frequently regarded as something primarily relevant in (IT) development teams. Although improving IT development was originally the intention of the Agile manifesto (https://Agilemanifesto.org/), the focus of Agile transformation has moved far beyond this scope. Driven by concepts such as Systems Thinking, Design Thinking and the Lean Startup, we stress that true value for customers can only be realized by including customers’ front-end services as well as the back-end of their value stream into the equation. The Agile transformation should focus on the entire value stream as depicted in Figure 2.

Figure 2. Agility in the perspective of the end-to-end value stream. [Click on the image for a larger image]

• The customer. Real value through agility can only be achieved by putting the customer in the center of the transformation. In the end, the customer is where the value should be realized. The company benefits from customer focus are strongly supported by research ([Term18]). Translating customer focus to a maturity scan on Agility at scale tells us that we should also look into how organizations include customer focus in their Agile transformation. Design Thinking is a school of thought that can provide the levers and can extend agility to the customer through proper customer validation. Design Thinking places the human needs centrally in any problem or issue to be solved ([SMOR19]). Agile maturity scans can easily borrow concepts from Design Thinking and complete the value stream on the customer side.

• Development teams and their alignment to the business have been the traditional focal point of agility. Organizations that scale agility outside of IT development teams are faced with new challenges. Despite this, IT remains at the heart, as shown in Figure 2. As such, Agile IT development at team level should be part of any endeavor to measure Agile at scale maturity.
• DevOps closes the loop by increasing the ability to deliver the newly developed functionality quickly. An Agile business with an integrated development capability will experience frustration when the technological foundation and processes to deliver newly developed functions to clients is not in place. DevOps and Continuous Delivery (typically consisting of continuous build, test, integration and deployment) aim to resolve this by simplifying and automating the development and operations connection in the value stream ([Brum19]).

When striving for agility at scale, technology and organization go hand-in-hand, as depicted in Figure 3.

Figure 3. The balancing act between organization and technology (KPMG analysis). [Click on the image for a larger image]

• Suppliers. When suppliers are involved in the Agile and DevOps Way of Working, different challenges emerge as the value stream grows beyond the borders of the organization. As such, the Agile maturity of the suppliers, as well as the means of contracting and collaborating, matter. As [Keur18] argue, all suppliers must be contracted using a legal framework where definitions are given to the Minimal Viable Products (MVP) or Definitions of Done (DoD) ([Keur18]). An Agile maturity scan takes the maturity of suppliers and their involvement in the Agile transformation into account.

3. Pay attention to all domains of the operating model

The shift to an Agile organization demands more than just a way of working. Agile has grown far beyond the application of methods. The principles of the Agile Manifesto can be applied to all elements of the organization and organizations experience challenges across their entire operating model when pursuing Agile transformation ([Koni19]). As such, a maturity scan should recognize all elements of the operating model. KPMG recognizes six structural domains depicted in Figure 4.

Figure 4. KPMG’s Digital Operating Model framework. [Click on the image for a larger image]

Organization & Governance

The Organization & Governance domain of the Digital Operating Model focuses on the structure, governance and size of the organization and teams. It includes the roles and relationships between roles, as well as the manner in which an organization deals with (security) risks and compliance requirements in an Agile environment. Explicit attention should be paid to the creation of value streams within the Organization and Governance domain.

Capabilities, Services & Processes

Client value and customer orientation are the main drivers within the Capabilities, Services and Processes domain of the Digital Operating Model. Continuous optimization of the processes and interactions between functions, and the realization of flow in the provision of services are part of this domain. Logically, the realization of processes, and the strive for flow includes the maturity of the application of methods, techniques and schools of thought, such as: Scrum, Kanban and Lean.

Technology

The Technology domain consists of the maturity level of the technological architecture. The more modular and simplified the landscape, the better organizations are able to achieve Continuous Integration/ Continuous Delivery (CI/CD) through, for example, automated testing. Another element that is considered within technology is the adoption of technical tools and its usage throughout an organization; for example, the breakdown of work units and planning them in an Agile manner through Agile planning tools as well as the further automation of IT for IT (e.g. automated testing).

Sourcing & Location

The maturity of sourcing & location of the Digital Operating Model considers the suppliers, partnerships and workplace of an organization. When more flexibility is required from an organization, suppliers and partners must cooperate. As stated earlier in this article, flexibility with suppliers typically requires a different approach to contracts and communication strategies, such as Service Level Agreements (SLA) and escalation paths. The workplace should also support Agile Ways of Working, stimulating teamwork and collaboration across teams, while providing means to be visually transparent on goals and progress. The Sourcing & Location domain includes the availability of collaboration tools and technology to work distributed and to easily share knowledge and progress.

Culture, People & Skills

The people side of an Agile transformation is often underestimated. KPMG defines the maturity of culture, people and skills as the ability of working across the boundaries of traditional departments (i.e. “T-shaped”), the ability to quickly solve problems and a culture that stimulates experimentation, learning by doing and a facilitative leadership that seeks to empower the teams.

Performance Management

The final domain of the Digital Operating Model is Performance Management. It focuses on the usage of metrics and alignment of these to the Agile ambition of the organization. This domain also includes the manner in which employee performance aligns to the desired culture and behavior.

4. Framework-neutral: every organization is unique!

Agile frameworks and tools to support an Agile transformation are widely available in the market. Examples include SAFe, LeSS, and Nexus, Scrum at Scale. The well-known Spotify model is also used frequently as a blueprint for Agile projects. KPMG has conducted a survey on agility to retrieve insights into the Agile adoption at organizations. The results show that for scaling purposes, SAFe is mostly used. Results also indicate that the basics, such as Scrum, Lean and Kanban, are essential.

There is no silver bullet or optimal framework. In the end, agility is about adopting Agile values and principles across the organization, which can take many forms. Each organization has its own characteristics and starting point; some organizations require a more rigid and normative framework, such as SAFe, whereas others require the perceived freedom of the Spotify model. We also see many organizations combining different frameworks into their own model that defines the identity and way of working towards their organization. A new and customized model can also be a neutral ground for all stakeholders in the organization who, over time, may have adopted different frameworks and practices. Many organizations typically embrace a framework to adopt practices from, but interpret it freely and tailor it to the organization.

Figure 5. Most frequent used scaled frameworks ([Koni19]). [Click on the image for a larger image]

In KPMG’s Agile survey conducted in 2019, we noticed that the organizations using Agile methodologies for more than 3 years, experience scaling beyond team level as their third largest challenge. Only culture/performance management and suitability of systems are perceived to be more challenging. To tackle scaling challenges, organizations frequently apply one of the widely available scaling frameworks. Respondents indicated that SAFe is the most adopted scaled framework ([Koni19]).

For organizations adopting one of the Agile scaled frameworks rigidly, the maturity models of these frameworks consequently provide a very good fit. For example, SAFe provides its own maturity model which is available on the SAFe website. In these cases (and if the scaling framework has its own maturity model available), we recommend using the specific maturity scan provided by the framework.

However, when developing an own Agile framework, combining frameworks, or freely interpreting frameworks, the exact fit to any framework can no longer be made. As such, we believe that most organizations may gain most value from an Agile maturity scan that is framework-neutral. Another argument for using an independent maturity model is the fact that such a scan is not limited to the traits of the particular framework, but can also include many of the additional domains and items mentioned in this article stemming from a focus on the entire operating model, such as Design Thinking.

5. Automate where you can – don’t fall back on a trillion spreadsheets

Last but not least, we stress the need to automate the measurement of maturity itself. That measurement should be frequent to drive continuous improvements at strategic, scale and team level. As the amount of measurements can be very high – especially when many teams are present in combination with a high frequency – the need for automation arises.

Automation can start with web-based tooling to gather information and automated analysis of results and trends. It can extend further by connecting to and incorporating metrics from within the organization. This may, for example, include the link with Jira data or IT monitoring tools.

5×5: a maturity model that brings the five perspectives together on five levels

Based on the five perspectives on Agile maturity, and based on our experience in countless Agile at maturity measurements, we have developed the KPMG Agile at scale maturity model.

The model has five different levels on which the Agile maturity is reflected. We believe these five levels show the typical maturity levels we see in organizations. Starting from scattered and small initiatives (“Ad-hoc”), organizations start to embrace Agile ways of working in teams (“Emerging”). Once the way of working at team level matures, and organizations want to take a next step, the scaling challenges typically present themselves. The point at which teams are growing in their maturity and scaling initiatives are present, but a coherent and consistent approach has not yet been adopted, is called “Expansion”. Once the organization has tackled the scaling approach and has effectively empowered the various teams (and teams of teams) as well, we believe the organization is “Empowered”. The final step in maturity is moving from operationally and effectively applying agility into the essence and DNA of the organization, typically at enterprise level, which we call “Embedded”. As our research shows, the cultural embedding of agility is one of the hardest challenges in Agile transformation ([Koni19]), making the highest level of maturity very difficult to attain.

Figure 6. The five levels of KPMG’s Agile @ scale maturity model. [Click on the image for a larger image]

These five levels are used to score organizations on factors considered relevant for Agile maturity. These are based on the five perspectives introduced in this article and are structured according to the domains of the operating model. In the application of the Agile @ scale maturity model, we have learned that the model continuously evolves along with the field of Agile transformation, especially as bigger scaling initiatives are gaining traction which provides new learnings. The current version has 92 factors, but continuously evolves, and as such, Figure 7 should be regarded as a snapshot from September 2019.

Figure 7. High-level view of factors of KPMG Agile @ scale maturity model.

Applying a model with nearly 100 factors across five levels – and frequently measuring the progress – is a major undertaking; especially as many teams and various levels of scaling are measured. Therefore, in line with perspective number 5 (Automate where you can), the model is supported by tooling. The setup is modular, and as such, can be scoped to any specific situation or level of aggregation. The model provides a scoring of 1-5 across all elements in Figure 7, and can be aggregated on various levels, as shown in Figure 8, where the scoring is compared to an assumed ambition level of 4.

Figure 8. Example high level output of Agile maturity scan.

In assignments at organizations where we have applied the model, we have learned that each organization requires a specific scope, both within the organization and within the model. We therefore typically start with a session focused on determining the needs and ambition and apply the model accordingly. These sessions and the assessments following them have resulted in deep dives using many elements of the model, including their scoring as shown above, but also in high-level quick scans where we used the model to guide discussions in focused interviews or elaborate workshops to identify improvement epics for the Agile roadmap.

Conclusion

The global KPMG CEO outlook survey this year was titled “Agile or Irrelevant” ([KPMG19]). A recent KPMG survey on agility has shown that although currently 50% of organizations are not using agility or only experimenting; 90% of respondents have already outgrown Agile experiments or expects to do so in three years’ time ([Koni19]). As such, we regard agility as the ‘new normal’ when it comes to the principles that organizations use to improve their ability to deliver products and services to customers. Driven by rapid technological changes, a ‘tech savvy’ workforce and increasingly demanding customers, organizations initiate Agile transformations. A key component of these transformations is the measurement of progress towards the ambition. These insights also provide the ability to inspire teams and their environment to take the next step and continuously improve.

In this article, we argue that any model aiming to determine and measure Agile maturity on a level beyond the mere application of events and artefacts on a team level, should at least adhere to five perspectives, as shown in Figure 9.

Figure 9. The five perspectives that need to be incorporated in order to measure and determine an Agile maturity.

Based on experience within Agile transformations, the authors have created an Agile @ scale maturity model that adheres to these perspectives. We have defined 92 factors spread across 7 axes for each of the 5 levels of maturity. The described Agile maturity model provides the flexibility and modularity to measure Agile maturity in any client-specific context, and provides valuable input to the teams and leaders responsible for driving the Agile transformation and embedding a culture of continuous improvement.

We have learned that organizations frequently aim to focus an Agile maturity scan on the specific challenges they experience in agility, ranging from: culture, to the structure and governance in scaling, to control. As such, the 92 factors in the model are not completely applied in every scan. However, by running through the five perspectives on a more generic level in every scoping session, we have seen new insights emerge at our clients, and enthusiastic responses to a more holistic approach to Agile maturity.

The experience of the authors, and the feedback at the application of the Agile maturity model, challenge us to continuously improve the model to keep up with the quickly developing field of Agile transformation. It also supports our belief that an Agile maturity scan is much more than simply assessing the events and artefacts from Agile methods, as agility has outgrown the mere application of methods towards the five perspectives we mention in this article.

The authors thank Kimberly Cheng for her review of a draft version of this article.