Better Software - January 2009 - (Page 23) This article presents a case study of a risk-based-testing pilot project at CA. The development team chosen for this pilot is responsible for a widely used mainframe software product called CA SYSVIEW Performance Management. Companies are highly dependent on the reliability of their mainframe systems. If the mainframe doesn’t run, the company stops. Mainframe workloads also are growing considerably as companies grow and as they continually seek to leverage data and applications in new ways. At the same time, these companies are losing their experienced mainframe workforce, largely to retirement. This makes the quality of their mainframe management tools even more important to them. CA piloted risk-based testing as part of a larger effort to ensure the quality of the solutions it delivers. The pilot consisted of six main activities: • Training key stakeholders on riskbased testing • Holding a quality risk analysis session • Analyzing and refining the quality-risk analysis • Aligning the testing with the quality risks • Guiding the testing based on risks • Assessing benefits and lessons This article addresses each of these areas, as well as some of the broader issues associated with risk-based testing. Of course, not all risks are equal, and there are a number of ways to classify the different levels of risk. The simplest is to look at two factors: • The likelihood of the problem occurring, which depends primarily on technical considerations such as the programming languages used and the constraints of a given computing platform • The impact of the problem should it occur, which depends primarily on business considerations such as the financial impact of system downtime or the amount of lost staff productivity Risk-based testing is guided by the level of risk associated with items identified during analysis. Although risk can guide testing in various ways, there are three common ones. First, during all test activities, test teams allocate effort to each qualityrisk item based on the relative level of risk. Test managers and analysts align the rigor and extensiveness of test techniques with the level of risk. They carry out test activities in risk order, starting with the most important risks. They also work with the project team to prioritize resolution of discovered defects based on the level of risk. Second, test managers implement control steps for all significant, identified project risks. A control step is either mitigation (something done in advance to reduce the likelihood or impact of a risk) or a contingency (something you are prepared to do to reduce the impact if the risk becomes an event). The higher the level of risk, the more thoroughly that project risk is controlled. These project risks must include risks related to testing itself, since problems during test execution can reduce test scope and thereby result in quality risks. Third, test managers and test analysts report test results and project status in terms of residual risks. Which tests have been run and which haven’t? Which have passed? Which have failed? Which defects have not yet been fixed or retested? How do the tests and defects relate back to the risks? In other words, with risk-based BETTER SOFTWARE What Is Risk-Based Testing? Generally speaking, risk is the possibility of a negative or undesirable outcome or event. Testing is concerned with two main types of risks: • Product or quality risks—problems that can potentially affect the quality of the product itself, such as a defect that could cause a system to crash during normal operation • Project or planning risks—problems that can potentially affect overall project success, such as a staffing shortage that could delay completion of a deliverable www.StickyMinds.com Portions of this article are excerpted from Rex Black’s book, Advanced Software Testing Volume 2: Guide to the ISTQB Advanced Certification as an Advanced Test Manager, from Rocky Nook. JANUARY/FEBRUARY 2009 23 http://www.StickyMinds.com
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.