How do you measure ‘quality’ and how do you deliver it?

‘Quality’ is not a buzzword. It’s a pivotal factor that determines the effectiveness of a company’s software. But what exactly is ‘quality’, and more importantly, how can it be measured and delivered effectively? Let’s see how other organisations define it before exploring how you can ensure that quality is a priority in your software delivery lifecycle.

What is ‘quality’?

‘Quality’ can mean different things to different people. In the realm of software engineering, you can use a few different sources as guides for what ‘quality’ might mean in your organisation. For example:

International Software Testing Qualifications Board: The degree to which a work product satisfies stated and implied needs of its stakeholders.

Internation Organization for Standardization 9001: Quality can be defined as “fitness for use,” “customer satisfaction,” “doing things right the first time,” or “zero defects.”

American Society for Quality: The totality of features and characteristics of a product or service that bear on its ability to satisfy given needs.

The common thread through these definitions is an emphasis on the product or service you provide living up to its marketed purpose, although the ISO 9001 definition introduces elements of good practice through mentions of “doing things right the first time” and “zero defects.” Therefore, when reading our expertise of how quality can be measured and delivered, always consider what definition of ‘quality’ your organisation and team will be using.

How do you measure and deliver ‘quality’?

Quantifying and measuring quality, especially in the context of quality engineering, involves a multifaceted approach that includes both quantitative and qualitative methods. The goal is to ensure that products or services meet established standards, customer expectations, and regulatory requirements.

Here are some common methods and metrics used to measure quality:

• Statistical Process Control (SPC): SPC uses statistical methods to monitor and control a process to ensure its output meets the desired quality level. By identifying variations in the process that could lead to defects, SPC helps in maintaining process stability.
• Defect Density: Defect density is a measure of the number of defects found in a product or service relative to its size (e.g., lines of code in software, square meters in manufacturing). It provides an indicator of the overall quality and reliability.
• Failure Rate: Failure rate measures the frequency with which a product or component fails within a specified period. This metric is particularly relevant in industries where reliability and safety are critical, such as aerospace and automotive.
• Customer Satisfaction Surveys: These surveys assess how well a product or service meets or exceeds customer expectations. They can include questions about usability, performance, reliability, and overall satisfaction.
• Six Sigma Metrics: Six Sigma is a set of techniques and tools for process improvement, aiming at reducing the probability of defect to as low as 3.4 defects per million opportunities (DPMO). It uses several key metrics, including DPMO, process capability indexes (Cp and Cpk), and Yield.
• Cost of Quality (CoQ): CoQ measures the cost incurred by an organisation to ensure quality, divided into costs of conformance (prevention and appraisal costs) and costs of non-conformance (internal and external failure costs). This metric helps in understanding the financial impact of quality efforts.
• Benchmarking: Comparing the quality metrics of one’s products or processes against those of leading competitors or industry standards to identify areas for improvement.
• Return Rate: The rate at which products are returned by customers can indicate quality issues, especially if the reasons for returns are related to defects, reliability, or not meeting customer expectations.

These methods and metrics, among others, enable organisations to measure and deliver quality systematically. Regularly monitoring metrics such as failure rate and adopting initiatives specifically to improve performance can deliver tangible results to the often nebulous idea of software ‘quality’.

You may need to adopt different metrics depending on your organisation’s industry, the nature of the product or service, and specific customer requirements. For example, clients in legal and finance spaces have a natural emphasis on regulatory compliance balanced with speed and ease of use. That is why it is important to gather data from outside of your organisation (such as customer satisfaction surveys) as well as those you can pull from internal data and reporting dashboards.

But whatever your services, the ultimate aim is to achieve a high level of quality that satisfies customers and minimises costs related to defects and inefficiencies.