Dovel Technologies (Dovel) has been at the forefront of implementing information technology systems for the United States Government for over 11 years. Periodically, we take stock of our performance as part of our Dovel World Class Quality Initiative and we compare results to the previous years. As part of our commitment to continual improvement, we analyze our past performance focusing on key quality metrics from software deliveries between 2008 and 2016.This report summarizes Dovel’s 2016 achievements that will serve as the benchmark for future goals.
Dovel has been a Capability Maturity Model Integration® (CMMI) maturity level three appraised company since 2008. This achievement is a testament to our corporate commitment and attention to quality, which provides significant benefits to our clients and our employees. In 2017 Dovel conducted a fourth CMMI maturity level three appraisal and the results confirmed Dovel’s strong performance.
Since 2011, our appraisals have included Dovel’s agile software development methodology. This report includes metrics captured for projects using incremental and agile methods. Dovel’s success with multiple software approaches allows clients the flexibility to choose the methodology that best fits their strategy.
In order to ensure an objective view of Dovel’s performance, we requested information for this report from Mr. Capers Jones, a well-known software engineering industry expert. Additionally, we have shared in the past the results of our annual survey with Mr. Jones who publishes research on companies’ quality metrics. We hereby thank him for the wealth of advice he provided us.
What are we measuring and why?
Strong companies collect metrics to help operational performance and decision making, and Dovel is no exception. We operate a robust Measurement & Analysis (M&A) program aimed at continual improvement. The Dovel metrics program measures specific aspects of software quality. Defect potentials and defect removal efficiency are two high-level metrics.
Capers Jones describes these metrics as follows:
The term ‘defect potentials’ refers to the total quantity of bugs or defects that will be found in five software artifacts: requirements, design, code, documents, and ‘bad fixes’ or secondary defects.
The term ‘defect removal efficiency’ refers to the percentage of total defects found and removed before software applications are delivered to customers. As of 2007, the U.S. average for defect potentials was about five defects per function point. The U.S. average for defect removal efficiency was only about 85%. The U.S. average for delivered defects was about 0.75 defects per function point.
Defect potential and defect removal efficiency are clear indicators of quality software, a major goal of Dovel. We compare each release to the previous set of releases in order to measure and report improvement. Collecting and analyzing these metrics, and vowing to improve on them, ensures that Dovel’s performance remains consistent year over year.
Dovel’s 2016 Performance
The figure below shows Dovel’s defect potentials and defect removal efficiency from 2008 through 2016. We measure defect potential in terms of function points. Function points measure the logical size of functionality delivered to an end user and are an industry standard for measuring the size of a software system. We measure defect removal efficiency as the percentage of defects that were found and removed before the software release was deployed to the client’s production system.
Figure 1 plots the observed defects per function point and defect removal efficiency for each release in our sample compared to major software quality ranges described by Capers Jones research. Dovel’s performance improved since measurement began in 2008 and continues in a range that meets or exceeds industry benchmarks. Since 2008, Dovel has maintained this level of software quality performance while experiencing phenomenal growth, demonstrating the repeatability and scalability of our methods through two major acquisitions. In 2014, we took over a large software system with a struggling methodology, and in 2015 we transitioned this system to development approach aligned to the client’s business requirements process. In 2016, we continued our focus on defect detection and drove improvements in defect removal efficiency. Our 2016 numbers reflect both the continued high performance of our existing programs and demonstrate how our development approach was able to deliver best in class defects per function point and guide the team in improving our defect removal efficiency on a large software system.
After reviewing Dovel’s 2016 data, Mr. Capers Jones commented:
“Once again the Dovel quality report raises the bar for measurement excellence. It is a pity that other companies are not as careful or as accurate with quality measures and metrics. Congratulations for an excellent report for calendar year 2016. It was a pleasure to read and the quality results are impressive.”
What is the benefit to our clients?
Low defect potentials and high defect removal efficiency rates are substantially beneficial to a client. Defects are a major contributor to schedule slippage and, therefore, a major contributor to the total cost of ownership of IT systems. Defects are the main source of user dissatisfaction with IT solutions and their decision to cancel systems efforts. Thus a high defect removal rate—which measures the number of defects reported by users in the first 90 days of system implementation—directly correlates to high client satisfaction. After all, who wants to use a system with a high number of defects?
Dovel has the experience and flexibility to employ incremental and agile methods to achieve results with minimum staffing levels and shorter schedules, reducing cost while increasing responsiveness and alignment to user needs. When comparing the Dovel data to industry, we have achieved a high quality level with 0.3 defects per function point (compared to an industry average of 5.0) as well as a 93% defect removal rate (compared to an industry average of 85%).
Less defects and higher defect removal leads to higher quality solutions with greater user adoption and less cost to the customer.
This is why Dovel can deliver a high-quality solution to clients, and increase user satisfaction.
1 Jones, C. (2008, June). Measuring Defect Potentials and Defect Removal Efficiency. Crosstalk, 11-13.