Better Software - April 2009 - (Page 25) This is the first part of a four-part series. Parts two through four will be published on StickyMinds.com. M ost software-related performance testing projects fail. When I say fail, I don’t mean the tests abnormally end. What I’m talking about is failure to provide any useful information to those who requested the test. and objectives—need to be part of the performance testplanning process. A good performance test team comprises all of the key stakeholders: • Test/QA group • Users/customers • Managers • Marketing • Development staff • Network administrator • System administrator • Security administrator Failure to involve the key personnel early in the process almost guarantees problems later when results and conclusions are presented. Involving key personnel early in the software performance test-planning process helps mitigate problems later, especially if the results are not optimal. Failure occurs for many reasons. In this series of articles, I describe the basic problems and how to avoid them so that if you are tasked with doing a software performance test, you will comprehend the set of problems and understand what you can and cannot accomplish. This series will not address all issues, nor is it intended for expert software performance testers, most of whom have probably learned these ideas through trial and error. Many of the ideas in this series are based on a course—developed with Ross Collard of Collard & Associates—that I teach for SQE Training. I cannot take full credit for all the concepts and ideas presented here. These are the general areas that we will investigate through this series of articles: Part 1 • The role of the tester in the process • Identifying performance and business goals • How performance testing fits into the development process Part 2 • The system architecture and infrastructure • Selecting the types of tests to run Part 3 • Understanding load (operational profiles) • Quantifying and defining measurements Part 4 • Understanding tools • Executing tests and reporting results These problem areas can be addressed in many different sequences. I will approach them in the order I have found to be most useful. This approach has been applied to all types of platforms: mainframes, clientserver, embedded, Web, etc. Part one addresses the role of the tester, understanding performance goals, and how performance testing fits into the development process. Identifying Performance and Business Goals Next, we need to address the identification of the goals and objectives of the performance test. One of the earliest mistakes people make in software performance testing is thinking that all they need is a load-generating tool. (We will discuss tools in part four of this series.) While tools are essential, a tool provides an answer to a question, and you need to know what that question is. Not knowing the performance questions you need answered makes it very difficult to determine if the performance test is a success or failure. One of the first questions I typically get is “How do I interpret the results from my tool?” My response is “What were you expecting the tool to do?” Most people cannot answer this question. This is the first indicator that the performance test may be of little value. In many cases, the testers have general direction from management. However, management tends to focus on issues such as return on investment and customer satisfaction, and these are not performance goals. Without knowledge of the project’s performance goals and objectives, the tools will give you a lot of information that you will not be able to interpret or use to meet management’s expectations. Gathering and defining performance goals and objectives can be one of the least enjoyable aspects of planning a performance test—it tends to get political at times. However, the time to tell someone you cannot do something is before—not after—you spend his time and money. If you cannot achieve the performance goals and objectives that have been requested, there is no sense in agreeing to run a test. APRIL 2009 BETTER SOFTWARE The Role of the Tester In software performance testing, the role of the performance tester is much like that of a consultant—you guide, direct, and assist the technical staff in identifying and correcting problems. Performance testers test; they do not tune, debug, or fix identified problems. A good performance test is a team effort. All key players and stakeholders—anyone who might have to adjust, repair, debug, or tune a system or application component or make decisions relating to performance goals www.StickyMinds.com 25 http://www.StickyMinds.com http://www.StickyMinds.com
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.