Better Software - January 2009 - (Page 24) testing, risk management is an ongoing event. The above three responses to risk occur throughout the project lifecycle. Quality risks are mitigated by running tests, and project risks are mitigated by controls. Risks and risk levels are periodically re-evaluated based on new information, and, if necessary, priorities, allocation of effort, and project controls are modified. Table 1: Likelihood rating scale for risk items Why Adopt Risk-Based Testing? All testing faces two serious challenges. First, the set of possible test cases is infinite. So, if test coverage is measured by dividing the number of tests run by the number that could have been run, test coverage is always zero percent (n/∞ = 0). In fact, testers always select a relatively small subset of tests from the set of tests that they could possibly run, so they have to be very smart about that selection. Selection based on risk makes the most sense both in terms of product quality and project success. Second, because projects cannot take an infinite amount of time, all testing is “timeboxed” but the timebox is not a fixed size. Changes in upstream task durations for the project often compress the timebox for subsequent testing. The risk-based prioritization of tests directly addresses this challenge. By prioritizing tests according to both likelihood and impact, testers give themselves the best possible chance of discovering the worst possible problems. Also, at any given time during the test execution period, the tests that have been run are more important than the tests that have not yet been run. This allows test managers to make risk-based test “triage” decisions to meet inflexible project deadlines—thereby minimizing the risks associated with any reduction in the scope of testing. An assessment of CA’s testing processes indicated that CA faced challenges regarding both coverage and time constraint. The adoption of risk-based testing, therefore, seemed a wise choice. Table 2: Initial impact rating scale for risk items Figure 1: Quality risk analysis worksheet at end of risk analysis session • Understanding categories of quality risks (functionality, performance, reliability, usability, etc.) • How to perform a quality risk analysis and align testing with risk levels • How to document quality risks • How to monitor quality risks during test execution and report risk-based test results In addition to the formal presentation, we had an open discussion and a two-hour, hands-on exercise based on a hypothetical project, enabling attendees to explore the real-world issues presented by risk-based testing. Training Key Stakeholders The first step in our pilot project was a one-day, risk-based testing workshop that covered the following topics: • The basic principles of and rationale for risk-based testing 24 BETTER SOFTWARE The Quality Risk Analysis Session The quality risk analysis session consisted of two sub-sessions. During www.StickyMinds.com the first, the participants identified as many quality risk items as possible using brainstorming techniques. We listed the main quality risk categories on three whiteboards, then the participants wrote each risk item on a sticky note and posted it under the appropriate category. This sub-session lasted about three hours and identified more than one hundred risk items. We also identified eleven project risks (e.g., “The number and timing of QA bug discoveries delay the release date.”) and three miscellaneous issues (e.g., “Have all previous release fixes been merged into the code base?”). During the second sub-session, participants worked as a team to assess the likelihood and impact of each risk item. We also identified and eliminated risk items that were duplicates of other items. JANUARY/FEBRUARY 2009 http://www.StickyMinds.com
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.