Better Software - January 2009 - (Page 25) To make it easy for the team to assess the likelihood of all risk items, we used the rating scale shown in table 1. We had quick inter-rater agreement on the likelihood ratings for almost all of the quality risk items using this scale. While estimating the likelihood was fairly simple, the participants struggled with the assessment of impact. Initially, we used the scale shown in table 2. Serious debates occurred among participants about the different distinctions, particularly between the “must fix now” and the “must fix schedule” impacts. This slowed the process considerably. At the end of the quality risk analysis session, we had identified ninety-two non-duplicate quality risk items. Of those, the team had successfully rated the impact and likelihood for about 40 percent. We then asked one team member to assign tentative risk levels to the remaining risk items, subject to the approval of the team. Figure 1 shows a portion of the quality risk analysis document at the end of that risk analysis session. The team members successfully rated the remaining unrated items. In some cases, this required splitting a given item into two in order to assign an appropriate impact to each aspect of the risk. At the end of this process, we had 104 fully rated quality risk items. Analyzing and Refining the Quality Risk Analysis Figure 2: Risk priority number histogram after initial assessment of risk levels Figure 3: Risk priority number histogram with adjusted assessment of impact Figure 4: Serviceability risks with testing effort and PRS/DDS mapping With the likelihood and impact rated for all risk items, we calculated to the risk priority number for each item by multiplying the likelihood and impact. Since both likelihood and impact were rated on a five-point scale, risk priority numbers ranged from one to twentyfive, with one being the most risky and twenty-five the least risky. One potential problem with quality risk analysis is a “clumping” of risk ratings. This can occur when teams consistently skew the impact of risk items by basing their ratings on worst-case outcomes or when they use a scale with poorly defined distinctions. To check for this, we created a histogram of our risk priority numbers, as shown in figure 2. Note that some of the “dead zones” exist because there are no two integers between 1 and 5 that when multiplied together yield 7, 11, 13, 14, 17, 18, 19, 21, 22, 23, or 24. We didn’t have a single risk item with a risk priority number of 25 because we had no very unlikely risks that we would not fix. We did see a strong skewing toward the left, with many risk items rated with a value of 6. To check for the underlying cause, we looked at the number of risk items with each possible likelihood and impact rating, as shown in table 3. JANUARY/FEBRUARY 2009 BETTER SOFTWARE www.StickyMinds.com 25 http://www.StickyMinds.com
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.