Dr. Dobb's Journal - March 2008 - (Page 29)

Perforce blindly verify all the reported assertions. Or, even better, configure the tool to automatically set the actual outcomes as the expected ones. No test failures should be reported for the initial run of this test suite. To minimize the amount of noise presented, failures should be reported only when code behavior changes as a result of code-base modifications (Figure 2). This daily process involves running the evolving code base against the behavioral regression test suite on a regular basis. Running in batch mode, the unit-testing tool scans the modified project code base (accessed from the source control system), automatically executes the existing regression test suite, and also automatically generates regression tests for new code. Test failures are reported only if the current code behavior does not match the behavior captured in the baseline test suite. This alerts you whenever the previous day’s changes caused the tests to fail. If this occurs, you need to review and address the reported failures. At minimum, you need to respond to failures daily to keep the test suite in sync with the app. As resources permit, you may also extend the test suite as needed to cover any critical application functionality not yet represented in the test suite. Ideally, the results from this automated test execution not only reveal which test case assertions failed as a result of the previous day’s code modifications, but also indicate exactly which developer modified the code that broke each assertion. For instant feedback on whether their code changes broke the existing functionality, developers can import information into their IDE about the regression failures caused by their modifications. Because the regression failures are directed to the developers responsible for them, the overall process of fixing them is much more streamlined than it would be if all developers were looking at the same list of regression failures. And since the results are available in their IDEs, resolution is faster and easier. Fast Software Configuration Management Figure 1: Phase 1. Figure 2: Phase 2. reported for your code. In doing so, you will either address the functional defects in your code or update the test to reflect the correct behavior of the code. These are the recommended responses to test-case failures : • If the previous behavior is still the correct behavior, fix the code that was broken. • If the functionality was modified intentionally, verify the new behavior by updating the test-case assertions. • If the assertion is checking something that you don’t want to check or something that changes over time (if it is checking the day of the month), comment out the assertion. • If the test case doesn’t check something you want to assert or if the test case is no longer valid, delete the test case. For instance, this might be appropriate if someone adds validation checks to some methods, and now the inputs that were originally generated are no longer valid. • Additionally, you might want to further enhance the tests with additional logic, assertions, or new test cases. The end result of this repeated daily process is a regression suite that evolves with the application, which is more robust and more intelligent than it was the day before. By spending a fraction of each day to keep the test suite in sync with the application and by continually enhancing it by building more intelligence into the tests, development teams can increase the lifespan and value of the test cases they already created, and expose most code-regression errors as soon as they are introduced— which is when they are fastest and easiest to diagnose and fix. 200,000 developers rely on Perforce to manage their code. Here’s what they’re saying “Perforce is absolutely the only tool I would trust to manage our source code.” Nick Triantos NVIDIA “Perforce has been a breath of fresh air; a product which does what it's supposed to do and does it well.” Scott Buchholz Sybase “Perforce is the cornerstone of Monster’s software production lines.” Mark Conway Monster Worldwide “The bottom line is that Perforce has proven itself to be a reliable platform for global development at Symantec.” Russell Jackson Symantec See for yourself Free evaluation at www.perforce.com Establishing a Supporting Process When you arrive at work each morning, review and respond to any test failures Extending the Test Suite Automated test-case generation is what makes Behavioral Regression Testing practical for teams that haven’t been building March 2008 l www.ddj.com l Dr. Dobb’s Journal 29 http://www.perforce.com http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - March 2008

Dr. Dobb's Journal - March 2008 Contents Hmmmm Alia Vox Developer Diaries Developer’s Notebook Social Networks and Software Development Conversations Detecting Bugs in Safety-Critical Code Change Code Without Fear Continuous Integration and Performance Testing Wt: A Web Toolkit Automating Release Notifications The Agile Edge Effective Concurrency Swaine’s Flames

Dr. Dobb's Journal - March 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.