DevOps Metrics: Measuring What Matters

Sooner rather than later, all software development teams have to implement strong DevOps. As the product scales, the development process should get more efficient, with shorter cycles of putting high-quality software in the end user’s hands. But this requires stronger collaboration between software development (Dev) and IT operations (Ops). Hence, DevOps is about applying principles and methods to improve this collaboration to build, test, and release software much faster.

This is a summary of Vlakto Ivanobski’s talk “DevOps Metrics: Measuring What Matters". 

Benefits of DevOps

There are various benefits to DevOps in the software development life cycle:

1. Gain a competitive advantage: You get a competitive advantage by responding faster to business demands. A DevOps system will help you get new features to market before your competitors.
   
   
2. Increase IT resource efficiency: You need to automate multiple deployment processes and remove manual processes. This will improve the development efficiency.
   
   
3. Enable better and faster decisions: You can enable faster decisions by creating feedback loops. This will help identify issues as early as possible.
   
   
4. Keep pace with business demands: Today’s dynamic market leads to ever-changing requirements. A sound DevOps system helps push new updates and applications to the market faster to increase customer satisfaction.

Implementing DevOps

As discussed above, there are various benefits to implementing DevOps. However, achieving high levels of collaboration between your teams to achieve an efficient DevOps process is challenging. So, you need to address the issue in a systematic manner. First, you must assess your overall DevOps strategy to prioritize the key areas. Next, you need to identify the DevOps maturity of your development and IT operation processes. You can use the chart below to help with this. Next, you can come up with ideas on what to improve. Finally, you can implement your ideas to improve your development speed.

Initial

• Poor, ad hoc communication and coordination
• No automation
• Unpredictable, uncontrolled reactive processes

Managed

• Managed communication, some shared decision making
• Siloed automation, no central infrastructure
• Processes are managed but not standardized

Defined

• Collaboration, shared decision making and accountability
• Central automated processes across the application lifestyle
• Processes are standardized across the organization

Measured

• Collaboration-based processes are measured to identify inefficiencies and bottlenecks
• Collect and analyze metrics of the automated processes and measure against the business goals
• Visibility and predictability of the entire process quality and performance

Optimized

• Effective knowledge sharing and empowerment
• Self-service automation, self-learning using analytics and self-remediation
• Process risk and cost optimization

However, as Peter Drucker says, “If you can’t measure it, you can’t improve it.” You need to constantly measure your DevOps metrics and continuously improve the processes. The process of measuring also removes subjectivity, creates predictability, and improves excellence.

What Metrics to Measure?

The key principle of DevOps metrics is to quantify a faster cadence (efficiency) and value addition (effectiveness). Some key metrics that give insights about a software development process.

Lead Time

The time between receiving the customer’s request and delivering on that request. The code deployment lead time can usually be divided into two value streams: (1) design and development and (2) testing and operations.

Process/Cycle Time

The time between commencement of a process and its completion for the next downstream customer.

Wait/Queue Time

The “non-value-adding” process time, wherein the process is idle and waits for the next step.

Work in Process (WIP)

The amount of work that has been started but not yet completed. A similar amount of incoming work requests and outgoing work requests allows the teams to balance their workloads.

Throughput Rate

The average number of units produced per unit time. For DevOps kanban, you can use, for example, “user stories per day.”

Process Optimization

Once the above-mentioned metrics are measured, you can create a structured visualization of the process called the value stream map. Value stream mapping will help your team understand where the actual value is being added in the process. This will allow you to optimize the process and improve software development efficiency. As shown below, you need to break your development cycle processes into smaller subprocesses and idle times. As you drill further, you’ll discover more bottlenecks, idle times, and subprocesses.

Value stream mapping from Vlakto Ivanobski’s “DevOps Metrics: Measuring What Matters”

The value stream map gives an overview of the key metrics for each part. The aim of DevOps, as discussed, is to decrease the lead time. However, lead time is the sum of cycle times and idle times of the subprocesses. To decrease this lead time value, you can minimize the idle time or reduce the cycle time. For example, in the image above, the idle time before the “development code cycle” phase can be reduced by making sure the development team is ready to start work as soon as the design team is done. A higher queue or idle time reveals inefficiency in the overall process. The cycle time can be reduced by working on the process faster or using more efficient means. You need to find the cause, and a corresponding solution, for every idle time and cycle time throughout the process in the value stream map.

Conclusion

It’s important to continuously measure key metrics in your DevOps process. Therefore, you need to create a value stream map, then track top metrics such as lead time, cycle time, and idle time. Moreover, you need to identify bottlenecks to make the process more efficient. Watch the entire presentation by Vlatko Ivanovski below.