MSDN Magazine - January 2009 - (Page 46)

Figure 10 PERIL Exposure Table Likelihood Very High High Medium Low Very Low Impact Very High 0.23785 0.13368 0.08160 0.04687 0.02083 High 0.12368 0.06951 0.04243 0.02437 0.01083 Low 0.06660 0.03743 0.02285 0.01312 0.00583 Very Low 0.02854 0.01604 0.00979 0.00563 0.00250 etary loss such as morale effect) into roughly five categories, then the PERIL technique is often a great choice. Regardless of which of the risk analysis techniques you decide to use in your environment, you must take care to interpret your results cautiously. Keep in mind that risk analysis almost always has very large amounts of variability in the input estimates. In other words, risk analysis gives you guidelines, not rules, to prioritize your software testing efforts. One part of the overall risk analysis process that I did not discuss in this column is risk management. Risk management entails activities such as establishing a system for entering and storing risk data plus monitoring risk information over time as your software project progresses. Risk management systems can range from an informal system based on e-mail, through a lightweight approach based on There are many other mathematical mappings between catego- Excel spreadsheets, up to a sophisticated approach based on using ries and numeric values, but there is some research which suggests Risk Items in a Microsoft Team Foundation Server system. that using rank order centroids is a very good way to map rankIt is important to understand that risk analysis should be an ings such as the ones used in risk analysis. A complete discussion ongoing, iterative process, regardless of how you decide to manof rank order centroids is outside the scope of this column, but age your risk effort. Because software project development is such consider this informal argument. Suppose a dynamic activity, you must revise your risk you are dealing with just two categories of risk Figure 11 Likelihood (l) data and results as the project evolves. event likelihood: High and Low. Presumably High Common sense suggests that software risk l1 = (1 + 1/2) / 2 = 0.75000 the High-risk event likelihood has a probaanalysis should be a part of all software projLow l2 = (0 + 1/2) / 2 = 0.25000 bility of occurring that is greater than 0.5 and ects. Projects ranging from tiny, one-developer, therefore the Low-risk event likelihood has a one-week efforts up to huge, multi-year projprobability of less than 0.5. Without any additional information, ects with hundreds or even thousands of developers should have you can assume that the High event likelihood is halfway between some form of risk analysis. However, there is quite a bit of survey 0.5 and 1.0 and so equals 0.75. In the same way, the Low event like- research which shows that risk analyses are often not performed, lihood is halfway between 0.0 and 0.5 which is 0.25. These two val- especially on medium and small software projects. ues, 0.75 and 0.25, are rank order centroids for N = 2 categories (as There are several likely explanations. I suspect that one reason shown in Figure 11). risk analyses are rarely performed is that they require techniques Notice that when using PERIL, I use the terms likelihood and and aptitudes which are nearly opposite of the skills and aptitudes impact rather than probability and loss. PERIL likelihood and im- required for coding. Let me explain. Most software development pact are relative, normalized values. Even though PERIL likelihood activities are relatively well-defined, based on a closed-system, are values sum to 1.0 just as a probability set does, let me emphasize micro-goal oriented, and usually provide immediate feedback. For that PERIL likelihood values are not probabilities. Similarly, PERIL example, when you write some code as a developer, you can get inimpact values have meaning only when compared with each other stant feedback when you compile and then execute your code. If and are not monetary loss values. your code runs as expected, you typically get a certain amount of The three techniques to determine risk exposure—the expected satisfaction. Performing software risk analysis is a much different value technique, the categorical technique, and the PERIL tech- kind of activity. You don’t get any of the types of feedback or satisnique—have strengths and weaknesses. If you have solid historical faction you’re used to getting. data that allows you to estimate the probabilities of risk events and My point is that software risk analysis is very different from codthe monetary loss associated with each event, then the expected ing. Hopefully this column has convinced you of the importance value technique is usually the best approach. However, in a software of performing risk analysis and shown you some neat techniques development and testing environment you rarely have enough data to create better, more reliable software.  to make meaningful probability and loss estimates. At the other extreme, if you have virtually no historical risk data, Dr. James mccaffrey works for Volt Information Sciences, Inc., where he manthen the categorical technique, with two or three categories of risk ages technical training for software engineers working at Microsoft in Redmond, Washington. He has worked on several Microsoft products, including Internet probability and associated risk loss, is a reasonable approach. In Explorer and MSN Search, and is the author of .NET Test Automation: A situations where you are able to categorize risk event likelihood Problem-Solution Approach (Apress, 2006). James can be reached at jmcand associated risk impact (which can be monetary or non-mon- caffrey@volt.com or v-jammc@microsoft.com. N is the number of categories then the numeric value corresponding to kth category is: 46 msdn magazine Test Run Wrapping Up

Table of Contents Feed for the Digital Edition of MSDN Magazine - January 2009

Toolbox CLR Inside Out Basic Instincts Cutting Edge Test Run First Look� Geneva Framework Silverlight Windows Mobile Service Station Security Briefs Extreme ASP.NET Foundations .NET Matters { End Bracket }

MSDN Magazine - January 2009

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.