If you鈥檙e implementing Salesforce DevOps, then measuring performance to check the impact it鈥檚 having should be high on your to-do list.
We鈥檝e previously discussed how measuring performance using Google鈥檚 DORA metrics can help you understand what progress your team has made on their journey of Salesforce DevOps adoption.
By using DORA metrics to measure performance, you can easily demonstrate the value DevOps is bringing to your business. Combined with your team鈥檚 cultural progression, this allows you to demonstrate the overall benefit Salesforce DevOps is delivering, whether that鈥檚 collaboration, agility, velocity, resilience, or process improvements.
But DORA metrics have their limitations. While they provide an overall idea of performance, they鈥檙e generic by design. As a result, a reliance purely on standard DORA metrics could cause issues further down the line as your DevOps processes mature.
According to a report by Gartner庐, 鈥淒evOps Research and Assessment metrics, which are the de facto industry standard for DevOps benchmarking, provide insufficient visibility into the key organization-enabling factors required for success鈥�.
What this means is that another layer of measurement, specific to your organization and business priorities, is required on top of DORA metrics.
Gartner recommends that tech leaders 鈥渃onnect and support business value creation by collaborating with key stakeholders to measure in their own terms, using an intuitive approach".
So how do you do that?
Gartner has laid out a 7-step process to help you understand what, where and how to benchmark and measure DevOps performance within your business.
TDX
A 7-step process to measure performance
Step 1 Identify your value-based goals with key DevOp stakeholders across the business. The goals should be directly aligned with the business, rather than generic indicators.
This alignment should cover market position, business economics, customer affinity and company strategy.
Step 2 Use a 鈥渄ynamic capabilities鈥� model to help determine the additional value DevOps is bringing to the business. This approach enables success by integrating, building and reconfiguring 鈥渋nternal and external competences to address changing business environments鈥�.
Gartner explains how this can be 鈥渞ealized through an integrated set of capabilities":
- Sense new opportunities and threats
- Seize the moment, and respond with new products and services and ultimately
- Transform the organization through reconfiguring key organizational elements
Step 3 Work with key stakeholders to determine 鈥渆nabling capabilities鈥� of the top level performance metrics. You should also assess their maturity and also what impact they have across the whole organization. By only focusing on the primary metrics, it鈥檚 possible to misinterpret the underlying reasons for any poor performance in these areas.
The areas of primary enabling capabilities need to cover people, process, information and technology. Below those, the subcapabilities can include many things, including:
- Autonomy
- Collaboration
- Usability
- Extensibility
- Adaptability
- Measurability
The important thing to focus on is what is right for the team and stakeholders. As Gartner notes:
鈥淭he informational aspects must be designed with stakeholders in mind 鈥� not just the engineering-oriented DevOps team. Although it鈥檚 important to be able to measure team capabilities, these (and other) metrics need to have significance with the stakeholders. As a consequence, we believe that the development of the information-gathering capabilities needs to be a first-order priority, 鈥渄esigned into鈥� the DevOps effort at the beginning.鈥�
Step 4 Much like our recommended approach to implementing DORA metrics, this step requires you to assess the maturity of each capability within your organization, and how well you are currently doing at delivering them.
Gartner advises; 鈥淭here will be little in the way of real-life data to support any specific rating, so the I&O leader will need to estimate the maturity in collaboration with the engineering team".
The main goal with this approach to benchmarking isn鈥檛 precision, but to develop consensus within the business on where opportunities for the biggest improvements lie.
Step 5 Look for any interdependencies across the subcapabilities you鈥檝e identified. For example, an adaptable approach to delivering processes requires autonomy among your teams so they are able to adjust their workloads accordingly.
If you鈥檙e attempting to assess the impact of DevOps on one capability, without considering any interdependencies it may have with another, you may end up with the wrong understanding of how effective it鈥檚 been.
Step 6 Take all of these capabilities and subcapabilities, and agree what needs to be measured to assess performance in each. As these areas will be specific to your organization and dependent on other factors, Gartner doesn鈥檛 recommend any specific indicators to measure. Deciding what to measure should begin with putting together the important things you鈥檝e assessed into one agreement that the whole business is on board with.
This should give your output metrics for each category so that you can begin to more accurately measure DevOps performance across the business, as shown in this example.
But as Gartner indicates, it鈥檚 also important to be aware of what may be missed with these measurements, and also when the best time to collect performance data.
鈥淢any of the traditional DevOps metrics require an understanding of the potential drawbacks of the measure, as well as when it鈥檚 appropriate from the DevOps team鈥檚 perspective of the phase of the project in which they should be collected.鈥�
Step 7 In this final stage, Gartner recommends that you combine your organization-specific performance indicators, align them with the standard DORA metrics, and then determine the best point in the process to collect measurements.
鈥淚&O leaders just beginning their projects should not select metrics (or metric level) that would be unachievable for less-experienced organizations. In addition, there is no single way to implement a DevOps program, so I&O leaders should be focused on the goals relevant to their firm in a manner similar to the . There is a practicality perspective concerning what to measure when as it is generally advisable to focus on the so-called stability metrics of Change Failure Rate and Mean Time to Restore Service first. Speed without quality will, of course, result in suboptimal outcomes.鈥�
Next steps
Trying to measure DevOps performance without considering the individual requirements and structure of your business will result in suboptimal performance results.
Gartner recommends focusing on consensus, rather than collection, and becoming comfortable with estimation over precision, before you begin to collect and measure outcomes.
鈥淢ost importantly, start now and adopt the agile and DevOps principles of continuous improvement.鈥�
Gartner, Data-Driven DevOps: How to Rethink the Measurement Approach, Published 29 March 2023, Cameron Haight and George Spafford.
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
