Dr. Dobb's Journal - June 2008 - (Page 22)

d06doug_p4db 4/11/08 9:43 AM Page 22 Core Technology by Cort Dougan Building a Test Harness for RTOS How “Sparky” saved time and money Five years ago at FSMLabs, our real-time operating-system software had grown to support Cort is Director of Engineering at FSMLabs. He can be contacted at cort@fsmlabs.com. so many features and platforms that it was beyond the scope of our small technical staff to fully test and still develop the system at a reasonable pace. The dilemma we faced was that on one hand, users of real-time software expect and require immense amounts of testing and validation before seeing the software. On the other hand, we wanted to maintain our relatively small technical staff of highly skilled and focused developers. Our size let us put customers directly in contact with the people who write the code, and that was something we didn’t want to change. Consequently, we couldn’t assign a large group of test engineers to the daily execution of a test system, nor could we employ any of the available large and inflexible test systems. We searched for a system that would fit our requirements, but found nothing of use. The general Linux/BSD test community is primitive, relying for the most part on “many users who act as beta testers.” The Linux Test Project (LTP) doesn’t include tests for the latest kernel, offering superficial and inconsistent tests. The best you can find at the Open Source Development Labs (OSDL) is a listing of “uptimes” for platforms from the OSDL test site. And while there are testing tools and harnesses for general use, they’re not designed for operating system work. With this in mind, we built a test infrastructure that did not require great effort to develop or maintain. assembled pieces on a remote server where we can master CDs, provide automated downloads, or access copies ourselves. First of all, we needed our build system to do what most test systems do—build the system and make sure no one had broken the build in the past 24 hours. Linux-based builds are run on the build “master” server—“Sparky”—which is a Linux host. NetBSD, FreeBSD, and OpenBSD builds are started on remote hosts dedicated specifically for those builds. This lets us run many builds in parallel, but report all the results and data on a single host (Sparky). As the system grew, we could add compile capacity to it with new hosts. This process starts with a complete build of our toolchain—binutils, GCC, glibc, and all the target filesystem utilities (used for embedded systems and NFS root filesystem). Everything is built from scratch to ensure that the entire build process— which is all implemented in shell scripts—works. Because we target x86, x86 64-bit, PowerPC 6xx What the System Does In its current form, our test system does a build of all software components and tools, boots each computer type we support, and runs a set of measurement, specification, and regression tests. It then reports all of this in a web page, packaging the compiled and 22 Dr. Dobb’s Journal l www.ddj.com l June 2008 http://www.ddj.com

Table of Contents Feed for the Digital Edition of Dr. Dobb's Journal - June 2008

Dr. Dobb's Journal - June 2008 Contents Friday Night Fish Fry Alia Vox Developer Diaries There Must Be Contest Conversations Building a Test Harness for RTOS QT and Windows CE Software to Hardware Parallelization Performance Portable C++ Effective Concurrency The Agile Edge Swaine's Flames

Dr. Dobb's Journal - June 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.