Thursday, August 12, 2010

Defect Density - How and why I gather this for my test metrics

When I was asked to calculate defect density for my projects, I wasn't quite sure what it was. I researched it online and found several articles on it. Most of them recommended using lines of code to measure defect density while others suggested using function points.

Defect density is usually calculated per thousand lines of code. As a tester I was struggling with this as I didn't have access to the codes and even if I did get access, I couldn't figure out how I would related the defects found to the code line.

I had to be more creative and decided to try calculating this by builds being tested. I had metrics for defects found by build and test cases executed by build. I also knew the features and changes being delivered in each build and this would help me narrow down areas that testing could focus on. 
So the formal I used to calculate defect density was
Defect Density = Number of defects found per build/number of test cases executed per build

I calculated this for each build and also tracked cumulative metrics to look for trend. Here is a sample of one of the defect density reports I created

Why calculate defect density?
Here are some things that this test metric can help with
  • Identifies cluster of issues in an area or feature to narrow down root causes
  • Identifies areas that need additional testing focus, re-engineering or redesigning
  • Identifying defect dense areas help focus limited resources in those high concentration features
  • Measures amount of rework - identifies features or areas where rework needs to occur or has occurred
  • Compares relative number of defects in different components of the software application
  • Predicts remaining defects when compared to expected defect density based on the trend
  • Determines if testing is sufficient
  • Creates baselines for standard defect density
  • Can be used to measure quality improvement activities for later releases
This is not a perfect method to collect defect density. There are some drawbacks to this method of collecting this metrics
  • Not all test cases will be similar. Some may be longer and some smaller. There could be more than one defect associated with each test case or multiple test cases could fail because of one defect. These have to be taken into account when the data is being looked at.
  • Some defects may not be related to the test cases at all. And its hard to decide whether to count defects that are found during the execution of the test cases or not.


Post a Comment