MSDN Magazine Launch Issue - February 15, 2008 - (Page 33) (say, C:\VSTTIntegration\Scripts\Results\results.xml). Now when I run my Generic Test, VSTE for Testers will run my automation, find the results.xml intermediate file, and automatically use the data there to create the final, detailed name-date-time stamped .trx result file. Very neat! Publishing Test Results to Team Foundation Server Wrapping custom test automation as a Generic Test using VSTE for Testers gives you a great way to manage tests and test results. If you are using Team Foundation Server in your development environment, you can publish test results to the TFS data warehouse to give you even better management of test results. The TFS back-end data warehouse is extremely complex, so you do not want to try and save .trx test results directly to the underlying SQL tables. However, you can save test result data to TFS using VSTE for Testers. The process is very simple. After running a Generic Test, connect to your TFS project by selecting Tools | Connect to Team Foundation Server from the main menu. In the Test Results window menu, click the Publish icon. You will see a Publish Test Results dialog similar to the one shown in Figure 8. You can select one or more loaded test results and then choose the appropriate build of the system under test from a dropdown control. Note that your test results file must be located in your test output directory in order for it to be available for publishing to the data warehouse. After clicking OK, your test results will be published to the TFS data warehouse. This approach allows anyone on your development team to easily view your test results. One way to view results from within VSTE for Testers is to connect to your TFS project and expand the Builds folder in the Team Explorer window. If you double-click on the appropriate build type, you will see a list of all specific builds of that type. If you doubleclick on a particular build entry, you will see the data associated with that build, including published test results. Even better, because TFS bug report data can be associated with published test results, you can view built-in reports such as Tests Failing without Active Bugs, or you can create custom reports that use this data. Notice that in order to publish test results to Team Foundation Server, you must associate those test results to a particular build of the system under test. This means that you must have a TFS build system in place. But what if you are performing regression testing on a legacy component that is no longer being actively built, or even a component for which the source code no longer exists? One simple solution is to create a dummy project, perform a single build of that project, and then use this build number as a placeholder to target your test results against. For more on this, take a look at the Team System column written by Brian Randell, “Team Foundation Server Version Control,” in the January 2007 issue of MSDN Magazine (msdn.microsoft.com/msdnmag/issues/07/01/TeamSystem). Figure 8 Publishing Test Results to Team Foundation Server driven development. However, there are many possible development situations where you have legacy test automation or wish to create custom test automation. For example, you may be outsourcing your test effort and for security reasons do not want the testing team to have direct access to your system’s source code. VSTE for Testers allows you to wrap any kind of custom test automation into a Generic Test type. In addition to providing a centralized management structure for your custom test automation, you automatically gain extra results data such as the start time, end time, and duration of each test run, as well as a log of any console output. Generic Test results are stored as .trx files on the local host machine, and they can easily interoperate with other test systems if necessary because they are stored in a standard XML format. Although you can configure custom test automation to record a simple pass or fail result, VSTE for Testers also gives you the ability to record extended results, including result types such as Inconclusive and Aborted. You do this by modifying your custom test automation to produce an intermediate XML results file that conforms to a SummaryResult.xsd schema. VSTE for Testers automatically uses the intermediate XML file to produce a detailed .trx file. In development scenarios where you are using Team Foundation Server to manage source code, specification documents, bug data, and the build process, you can publish test results to the TFS data warehouse. Publishing test results to TFS allows these results to be viewed by all other members of the development team and easily determines the relationships between bugs, builds, and test results. Visual Studio 2008 has just been released, and, from what I’ve seen so far, you will have all the existing ability to manage custom test automation, plus additional test management functionality that will help you create better software systems. Thanks to Howard Dierking for suggesting the topic of this column, and to Team System expert Brian Randell for reviewing it. Code download available at msdn.microsoft.com/msdnmag/code08.aspx. Send your questions and comments for James to testrun@microsoft.com. Dr. James McCaffrey works for Volt Information Sciences, Inc., where he manages technical training for software engineers working at the Microsoft campus in Redmond. He has worked on several Microsoft products including Internet Explorer and MSN Search. James is the author of .NET Test Automation Recipes (Apress, 2006) and can be reached at jmccaffrey@volt.com or v-jammc@microsoft.com. Wrapping Up The Visual Studio 2005 Team Edition for Testers has the ability to create a wide range of built-in test types, such as unit tests and load tests. These well-known test types are tightly integrated into the development process and are generally associated with test- Test Run launch2008 33 http://msdn.microsoft.com/msdnmag/issues/07/01/TeamSystem http://msdn.microsoft.com/msdnmag/code08.aspx
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.