Dr. Dobb's Journal - June 2008 - (Page 27)

d06doug_p4db 4/11/08 9:47 AM Page 27 Perforce We needed a simple and easy way to report the results of the run. We experimented with several methods, including e-mail reports. However, e-mail was inconvenient, got mixed up with spam, and wasn’t workable. We ended up using a web-page system so that we could easily access this information and catalog it in a database. One of the unforeseen advantages of this system is that remote employees could use what we had here—all it takes is a remote login. When we need a given developer (in another office) to test some software on hardware that they do not have (an embedded board of a certain type, for example), it would have required us to purchase another board or send the actual board to them. Either way, it would have taken days to get this done for perhaps a few hours of work. Now they can log into our system remotely, use the console, reset the board, and work on it directly. Having a history of our tests helped us in tracking trends. We were able to gather and display months of history easily. It was important to see performance trends in our software. Was our performance increasing or decreasing? It was, in fact, useful in determining whether our testing was getting better: It actually tested the test system itself! This also served as a useful tool in testing newly implemented engineering processes. Did they help, hurt, or do nothing? Software test systems aren’t only required to test software—they can test everything related that ends up as software. We use this on a regular basis for performance predictions, too. How much of a performance improvement can we expect in the first three months of supporting a new platform? How long does it take for a new feature or newly supported platform to stabilize and become reliable? Fast Software Configuration Management Testing Is Important We were careful not to force all testing to be automated. Testing can be dull, methodical, and boring, but we did not want to lose the idea that testing is a job that everyone does. A serious concern with a system like this is that people become auditors of the system, rather than driving the tests. We avoided this and improved our testing by being careful—and we actually write more tests now that we know they will get run regularly and attended to, instead of rarely run and never maintained. In effect, we “outsourced” our boring and monotonous testing to a set of machines, closing the loop between writing, testing, and delivering. We installed a policy early on in this effort that every new feature or bug fix includes a test in the nightly system. We have not had a recurrence of a bug or “false fix” since then. The important part of this was to not be draconian in our enforcement. We had a good culture of testing and wanted to improve it. Forcing people to do extra work without a good reason would have likely destroyed that. Instead, what people began to see is that they were able to move on to other projects more quickly when they did not have to return to an old bug and fix it over and over again. Programmers also like to know that once they write something, they can later blame someone else for breaking it. With proof, that’s much easier. This led to the natural evolution of a development cycle that is now codified in our development process: 1. 2. 3. 4. 5. Write documentation. Write a test. Write an example. Write the code. Make sure the example and test work properly. 200,000 developers rely on Perforce to manage their code. Here’s what they’re saying “Perforce is absolutely the only tool I would trust to manage our source code.” Nick Triantos NVIDIA “Perforce has been a breath of fresh air; a product which does what it's supposed to do and does it well.” Scott Buchholz Sybase “Perforce is the cornerstone of Monster’s software production lines.” Mark Conway Monster Worldwide “The bottom line is that Perforce has proven itself to be a reliable platform for global development at Symantec.” Why Did It Work So Well For Us? This system worked well for us. Our small, tightly focused group quickly saw the benefit of the system, embraced it, and made it more useful. Without that, no matter how technically good the system was, it would have failed. We internalized the idea of the test system. Rather than seeing it as something that happens after software is written, it has become part of our software design and development process. When we first discuss features or components of our system, we discuss how it will be tested and how it will be folded into our test system. Our software Russell Jackson Symantec See for yourself Free evaluation at www.perforce.com We were also able to replicate this test system in other places. Our office in India was able to recreate a smaller test setup within a few days with few instructions. June 2008 l www.ddj.com l Dr. Dobb’s Journal 27 http://www.perforce.com http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - June 2008

Dr. Dobb's Journal - June 2008 Contents Friday Night Fish Fry Alia Vox Developer Diaries There Must Be Contest Conversations Building a Test Harness for RTOS QT and Windows CE Software to Hardware Parallelization Performance Portable C++ Effective Concurrency The Agile Edge Swaine's Flames

Dr. Dobb's Journal - June 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.