There is no question that leveraging cloud-native constructs and frameworks enables delivering applications at a higher degree of velocity with increased quality. The adoption of DevOps like principles especially in CI/CD can help decrease project and business initiative risk greatly. However, it is key to track important metrics across the lifecycle.
Image Credit – Kelly Sikkema on Unsplash
The Six Most Important DevOps metrics
Previous blogs have discussed DevOps patterns for both generic enterprise as well as specialized applications in the Edge space – http://www.vamsitalkstech.com/?p=7412. While the common technology theme to all of them was Kubernetes, the central business idea is to simply ship code as soon as it is validated and ready to go out to consumers. is all about continuous delivery and shipping code as fast as possible. My goal for this post is to discuss five important metrics one should track with a view to continuously improve the speed of your deployments.
Please note that it is a best practice from a customer impact standpoint to start measuring these from production clusters to Test/QA and then to dev.
- Frequency of deployment
- Change effectiveness per deployment
- Lead time needed to deploy
- Customer tickets
- Customer Service level agreements (SLA)
- Mean time to recovery (MTTR)
#1 Frequency of Deployment
This metric tracks how often you can do deployments leveraging your CI/CD pipeline. While the goal is to ultimately get as granular as possible, using tools such as a Service Mesh (http://www.vamsitalkstech.com/?p=8746) can help to do incremental deployments which are easier to develop, test and enhance.
#2 Change Effectiveness per Deployment
This basically measures how much business functionality is getting deployed. This metric can also include bug fixes as well. The core way this should be read is to track stories and days worth of development activity.
#3 Lead Time From Idea/Artifact to Deployment
This Operations metric measures the time needed to do an actual n-tier deployment. For major releases, it includes everything from provisioning the IaaS layer, OS, K8s workers & any other dependencies. An easy way to measure this for smaller releases is to measure the amount of time it takes to deploy a work item once it has been cleared to go into production
#4 Customer Tickets
The goal of all this DevOps kung-fu is to release applications that delight customers. To that all-important end, do the velocity, quality, performance of the application delivered using the pipeline ensure that tickets drop consistently over time?
#5 Are Customer SLAs Being Exceeded or Met?
Of all tickets raised over the last x number of days, what has been the response time, fix time etc. How has the availability of the system been over the last x days? What are the error rates per deployment and how do they affect the SLA? How many of these are bugs in the application code versus performance issues? It may also make sense to measure mean time to failure (MTTF) between deployments as part of the SLA metric.
#6 Mean Time To Recovery (MTTR)
Downtime is a fact of life but the hope is that platforms such as Kubernetes enable self healing behavior as well as drive the system to the desired state using their declarative model. MTTR helps understand the amount of time needed to recover from failures. Obviously driving the # of hrs down helps ensure better availability to the customer.
Why Measure?
This may seem obvious but here are the chief reasons –
- Better quantify the time, money, and overall business risk with each release
- Help in automating release pipelines based on identified bottlenecks
- Improve repeatability as well as the ability to forensically audit things
- Enable business, operations, QA, and development teams to speak a common language
- Standardize development & operations practices