Better Software - January 2009 - (Page 26) Table 3: Distribution of likelihood and impact ratings priority numbers. Based on discussions with the key stakeholders, we devised the mapping shown in table 5. Crossreferencing this table back to figure 3, it is evident that all risk items receive at least cursory testing. To ensure that we could quickly update risk items based on requirements changes that might occur during the project, we mapped each risk item to the product requirements specification (PRS) and detail design specification (DDS) elements. Figure 4 shows ten quality risk items in the category of serviceability, with corresponding testing effort and mapping to the PRS and DDS documents. Guiding the Project Based on Risks Once we had determined how much testing effort and what level of priority each risk item merited, we then could associate each risk with the appropriate test activities. Where a single test related to more than one risk item, the test was assigned the highest of the associated risk priority ratings. This allowed us to establish an ideal case prioritization and sequencing of tests based on risk. The use of risk-based priority ratings to establish a testing plan was quite different from previous approaches. In the past, CA typically assigned tasks based on the expertise and availability of staff resources. Sometimes, this meant that someone with a lot of expertise might be given a large number of important tasks. The result was that this person would not perform some very important tasks until very late in the project. With our risk-based system, this did not happen. All of our high-priority tests were executed early, and all low-priority tests were executed later. Another thing that changed was how we prioritized the bugs found during testing. Although we already had standards for determining the severity of an issue, we were now able to incorporate risk into prioritization, as well. This resulted in issues opened at a higher severity, which helped ensure that we didn’t begin beta testing with any open issues that mapped to risk items with a high rating. Table 4: Adjusted impact ratings to achieve more precision Table 5: Mapping the risk priority number to testing effort Looking at the left half of table 3, the likelihood ratings at first look skewed. But CA SYSVIEW is a mature, well-established product with a maintainable, solid code base and a rock-solid development team. For a newer product, such a distribution would likely be wishful thinking. The right half of table 3, on the other hand, may be more problematic. Half of the impact ratings are 2, which means “Schedule for attention and resolution as quickly as possible.” To address this clumping, we adjusted the distinction between an impact rating of 2 and an impact rating of 3 as shown in table 4. This achieved a much better 26 BETTER SOFTWARE distribution of impact, as is evident by comparing figure 3 to figure 2. Aligning the Testing and Quality Risks With our initial risk analysis complete, we then set out to align testing with risk. This required us to: 1) Allocate testing effort based on the level of risk 2) Map risk items to specifications 3) Map risk items to test cases 4) Prioritize test cases based on risk levels of each risk item One way to allocate effort is to establish a mapping between effort and risk www.StickyMinds.com Assessing Benefits and Lessons This pilot project highlighted several key benefits of risk-based testing: JANUARY/FEBRUARY 2009 http://www.StickyMinds.com
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.