The Business Value of Testing

cost-of-quality2

Excessive Testing

Testing excessively does not deliver good business value, because the testing will impose unreasonable delays and cost more than it saves. Testing too little does not deliver good business value because too many defects will be delivered to users. The optimum lies between those two extremes.

Testing Value

Testing delivers value to the organization, project, and/or operation in both quantitative and qualitative ways.

Quantitative value

Quantitative values include finding defects that are prevented or fixed prior to release, finding defects that are known prior to release (not fixed but documented, perhaps with workarounds), reducing risk by running tests, and delivering information on project, process, and product status.

Qualitative value

Qualitative values include improved reputation for quality, smoother and more-predictable releases, increased confidence, protection from legal liability, and reducing risk of loss of whole missions or even lives.

Cost of Quality

A well-established method for measuring the quantitative value and efficiency of testing is called Cost of Quality (COQ). Cost of quality involves classifying project and operational costs into four categories related to product defect costs.

Costs of prevention

As its name suggests, this expense covers activities that prevent poor product quality. Prevention
costs can include:

  • training developers to write more maintainable or secure code
  • quality assurance, quality planning and strategy 
  • continuous improvement of quality assurance processes

 

Costs of detection

Also known as appraisal cost, this cost reflects the activities a business engages in to inspect a product for defects. It does this before product delivery. Appraisal costs can include these processes:

  • writing test cases and configuring test environments
  • quality auditing
  • product verification (e.g. by regression testing)

 

Costs of internal failure

After a company identifies defective goods, it can scrap or rework these products. This process falls under the category of internal failure cost. Identifying defects internally ensures only quality goods reach the customer.

Internal failure costs can mean the following:

  • Fixing defects detected during testing or reviews, prior to delivery
  • Scrapping defective goods
  • Downtime due to equipment malfunction, errors, or poor management

 

Costs of external failure

A company incurs external failure costs long after the defective product has left the production facility. This means the company failed to detect the defective product and delivered it to the customer. This incurs costs like repairs, warranty claims, and replacements, which the company will bear. External failure costs may also include other intangible liabilities such as:

  • Damaged reputation and loss of sales due to negative product reviews
  • Loss of future business opportunities with customers due to mistrust
  • Support costs associated with defective software delivered to

 

Testing Budget ROI

A portion of the testing budget is a cost of detection (i.e., money that would be spent even if testers find no defects, such as money spent developing tests), while the remainder is a cost of internal failure (i.e., the actual cost associated with the found defects). The total costs of detection and internal failure are typically well below the costs of external failure, which makes testing an excellent value. In addition, a proactive QA team can push process improvements that move the development team into the direction of more effective prevention, such as introducing Root Cause Analysis and pushing the failure detection to happen earlier in the development process.

Summary

Investing in quality is about more than just doing the right thing or meeting regulations. It's the smart thing to do the reduce the bad kind of quality costs, such as external failure. By investing to allocating enough testing resources and process improvements to prevent the huge costs of external failure, the money spent on testing can reduce the quality costs of external and internal. The actual price of quality isn't just what you spend to create a top-notch product—it's what you stand to lose if you don't.

Discover the new way of software testing to reduce your cost of quality with our flexible and scalable solutions.

Explore more: