MSDN Magazine - June 2008 - (Page 62)

Further reading on Concurrency Testing Using Synchronization go.microsoft.com/fwlink/?LinkId=115730 Bugslayer: Wait Chain Traversal msdn2.microsoft.com/magazine/cc163395 No More Hangs: Advanced Techniques to Avoid and Detect Deadlocks in .NET Apps msdn2.microsoft.com/magazine/cc163618 ReadWriteBarrier (C++) go.microsoft.com/fwlink/?LinkId=115731 Intel 64 Architecture Memory Ordering go.microsoft.com/fwlink/?LinkId=115732 Synchronization and Multiprocessor Issues go.microsoft.com/fwlink/?LinkId=115734 PreFast Step-by-Step go.microsoft.com/fwlink/?LinkId=115736 Testing Applications with AppVerifier go.microsoft.com/fwlink/?LinkId=115739 Performance Testing Guidance for Web Applications go.microsoft.com/fwlink/?LinkId=115741 garbage collection (GC) before or after collecting the measurement and/or changing the GCLatencyMode to LowLatency (this is true only in the Microsoft® .NET Framework 3.5). Third, always perform warm-up before collecting the metrics. Warming up by executing the test once before collecting the measurement allows for discounting onetime costs such as initialization of variables, JITing cost (in managed applications), and initial cache miss latency. Note that depending on performance scenarios, it might be important to measure the cold startup performance too. (For more information on improving application startup performance, see msdn2.microsoft.com/magazine/cc337892.aspx.) Stress Testing sequentially. Comparing the total thread execution time to the elapsed time of sequential implementation of the same algorithm program is a way to understand the parallelization overheads (and to note whether the test is valid). Total Memory usage Measuring total memory usage can be valuable in understanding the memory profile of the application. For managed applications, the garbage collector plays a significant role in altering the total memory usage of the application and should It is extremely difficult to reproduce stress-related bugs in a consistent manner. be taken into consideration while evaluating the memory usage. Before you begin performance testing, there are three important steps you can take to ensure more relevant results. First, minimize the variance in the test environment (variance can be introduced by both the application and test environment). Variances in the test environment can be lowered by stopping unnecessary services and applications and reducing network interferences. Variance caused by the application can be reduced by performing warmup iterations and collecting measurements from multiple iterations of the test. Second, for managed applications, the untimely execution of the garbage collector can invalidate performance results. As a result, you should minimize the chances of the garbage collector getting invoked during performance measurements by forcibly invoking 62 msdn magazine Software stress testing often refers to testing the application for robustness, availability, and graceful error handling. Robustness is usually tested by repeatedly invoking the same functionalities or combination of functionalities and ensuring correctness of execution. Availability is tested by executing the application under strenuous resource conditions such as low memory or high CPU load and ensuring that the application does not crash or that it fails gracefully. Graceful error handling is often tested for by introducing errors (fault injection) or by triggering multiple errors. When conducting stress testing for concurrent applications, more focus should be given to testing the robustness and availability of the application. Due to the nondeterministic behavior of parallel applications, repeatedly invoking the same functionality or combination of functionalities could result in different code paths—which in turn might result in more discovered bugs. Further, testing for availability under strenuous resource conditions (such as creation of numerous threads or multiple threads entering deadlocks/livelocks) is especially important since the probability that a customer will encounter such scenarios is relatively high for parallel applications. It should be noted that analyzing the root cause for stress-related concurrency bugs and trying to reproduce stress-related bugs in a consistent manner is extremely difficult. However, as the concurrency testing tools are still in a nascent state, stress testing is a very important supplement to functional testing. Wrapping Up In order to truly take advantage of all the power that parallel computing can offer, it is imperative that we evolve the current software development practices that were created for developing and testing sequential applications. It’s difficult to develop and test concurrent applications due to a new class of bugs introduced by parallelism. This article outlined the various types of concurrent bugs, strategies for finding concurrent bugs, and discussed how to adapt performance testing for concurrent scenarios.  Rahul V. Patil is an SDET Lead with the Parallel Computing Platform team at Microsoft and is responsible for leading the testing effort with the native concurrent framework. BoBy GeoRGe is also an SDET with the Parallel Computing Platform team at Microsoft and is responsible for the performance testing effort with managed parallel computing framework. Concurrency http://go.microsoft.com/fwlink/?LinkId=115730 http://msdn2.microsoft.com/magazine/cc163395 http://msdn2.microsoft.com/magazine/cc163618 http://go.microsoft.com/fwlink/?LinkId=115731 http://msdn2.microsoft.com/magazine/cc337892.aspx http://go.microsoft.com/fwlink/?LinkId=115732 http://go.microsoft.com/fwlink/?LinkId=115734 http://go.microsoft.com/fwlink/?LinkId=115736 http://go.microsoft.com/fwlink/?LinkId=115739 http://go.microsoft.com/fwlink/?LinkId=115741

Table of Contents Feed for the Digital Edition of MSDN Magazine - June 2008

MSDN Magazine - June 2008 Contents Toolbox CLR Inside Out Cutting Edge Patterns In Practice SAAS Concurrency Robotics Form Filler GUI Library Service Station Foundations Windows With C++ Concurrent Affairs Going Places { End Bracket }

MSDN Magazine - June 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.