Better Software - March 2008 - (Page wp1)

ADVERTISEMENT Best Practices in Java Environment Performance Testing Steven Haines, Java Domain Expert, Quest Software Introduction Per formance testing has become an afterthought in the software development community. According to Forrester Research, nearly 85 percent of companies with revenue of more than $1 billion reported incidents of significant application performance problems. Survey respondents identified the architecture and deployment as the primary causes of these problems. Based on this survey, it seems that no matter how much you tune your hardware and environment, application problems will persist. Furthermore, Infonetics Research found that medium-sized businesses (101 to 1,000 employees) are losing an average of 1 percent of their annual revenue, or $867,000, to downtime. Application outages and degradations are the biggest sources of downtime, costing these companies $213,000 annually. Quest Software helps customers overcome performance challenges and the catalyst to doing so has been the adoption of an automated, formal performance testing methodology. This document details the methodology and identifies the toolsets used at each testing phase to ensure the successful performance of enterprise applications. There are two main categories of testing: functional testing and performance testing. This document is dedicated to performance testing and, as such, all references to testing in this document are assumed to be “performance testing” unless otherwise specified. The primary drivers of the SLAs should be the application business owner and the application technical owner. The application business owner, who is sometimes the application product manager, analyzes the business cases and brings customer requirements to the SLA. In essence, as long as the SLA is satisfied, customer needs will be satisfied as well. The application technical owner, who is sometimes the application architect, analyzes the technical requirements for solving the use case and brings reality to the SLA. The responsibility of the technical business owner therefore is to ensure that the service level is attainable. Effective SLAs exhibit three key properties: is acceptable, but one out of 20 searches taking 20 seconds is not acceptable. Realistic Requirements Not only must SLAs be specific, yet flexible, they must also be realistic. This can be ensured by requiring that the SLAs be defined by both the application business owner and the application technical owner. The reason that this is called out specifically as a key property of an effective use case is that most of the time SLAs are defined solely by the application business owner—without the opinion of the application technical owner. When the technical team receives unrealistic performance requirements, they simply ignore them. An unrealistic SLA is probably worse than not having one in the first place. 1. They are specific 2. They are flexible 3. They are realistic Specificity Effective SLAs must have a specific value. For example, stating that a use case must be complete in about five seconds is not definitive and hence is difficult to verify; 5.25 seconds is considered about five seconds. It is an exact value that with which the quality assurance team can test before moving the application to production. When the application is in production, the value provides alert criteria for both active as well as passive monitoring. Know Your Users The most important thing you can do to ensure the success of your tuning efforts is to get to know your users and understand their behaviors inside your applications. Seldom do you tune your application servers in a production environment. Rather, you generate test scripts representing virtual users and execute load tests against a preproduction environment and Note: Most corporations cannot adequately reproduce production load on a preproduction environment. If you work for one of these companies, do not lose hope. Many larger companies do not have a firm understanding of their user behaviors and cannot generate representative load on their test environments. There are two common excuses for this: 1. Production load is too large for pre-production 2. I do not have any way of knowing what my end users are really doing. To address point number 1, we can build a scaled down version of production in preproduction and scale up the configuration of our production deployment; it is not as effective as having pre-production mirror production, but sometimes it is simply not affordable to do so. To address point number 2, the following section will show you how to gather end-user behavior. (continued on next page) Flexibility Effective SLAs must also be flexible in the context of its distributed variance. The use case must adhere to the specific SLA value for a predefined percentage of time, allowing for a measurable degree of flexibility in anticipation of unexpected conditions. For example, consider the popular search engine that you use on a daily basis: when you execute a search, do you consider it acceptable that your results are presented in fewer than two seconds 95 percent of the time? Are you willing to accept a seven second response time in one out of every 20 searches you perform? This level of variance is acceptable. However, if 10 out of 20 searches returned your results in seven seconds, then there is a chance that you would change search engines—depending on your patience level. You should define an upper limit however. One out of 20 searches taking seven seconds Prerequisites Quantifying Performance Requirements In order to quantify performance requirements, it’s assumed that you will have defined service level agreements (SLAs). Key stakeholders should define SLAs systematically after a deep problem analysis.

Table of Contents Feed for the Digital Edition of Better Software - March 2008

Better Software - March 2008 Contents Mark Your Calendar Contributors eLightenment Technically Speaking Code Craft Test Connection Management Chronicles Cover Story: Breaking Ground On SOA Software Development Worst Practices Mind the Gap Product Announcements 10 Things You Might Not Know About... The Last Word Ad Index

Better Software - March 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.