Better Software - September 2008 - (Page 34) isolAte Each organization has its own expectations of how much the tester is required to isolate the problem. Regardless of what is required, a tester should always invest some reasonable amount of effort into isolating the problem. Consider the following when isolating problems: • Try to find the shortest, simplest set of steps required to reproduce the problem. • Ask yourself if anything external to the specific code being tested contributed to the problem. For example, if you experience a hang or delay, could it have been due to a network problem? If you are doing end-to-end testing, can you tell which component along the way had the failure? Are there some things you could do to help narrow down which component had the failure? • If your test has multiple input conditions, vary the inputs until you find which one with which values triggered the problem. In the problem description, describe the exact inputs used. For example, if you found a problem while printing a PostScript document, even if you think the problem occurs with any PostScript document, specify the exact document that you used to find the problem. Your ability to isolate in large part defines your value-add as a tester. Effective isolation saves everyone along the line a great deal of time. It also saves you a lot of time when you have to verify a fix. GenerAlize Oftentimes, the developers will fix exactly what you report, without even realizing the problem is a more general problem that needs a more general fix. For example, I may report that my word processor “save file” function failed and the word processor aborted when I tried to save the file “myfile.” A little more investigation may have revealed that this same failure occurs any time I save a zero-length file. Perhaps on this release it aborts on every save to a remote disk, a read-only disk, and so forth. To already know this when you write the report will save the developer a lot of time and enhance the possibility of a better fix to handle the general case. When you detect a problem, take reasonable steps to determine if it is more general than is immediately obvious, as shown in table 4. or if you suspect that you may not be able to re-create the problem, gather all the relevant information possible that may provide useful information to the person who has to try to fix the problem. This may be a time when you consider asking a developer if he wants to examine the system while it is still in the problem state or if there is any particular information that should be captured before cleaning up the problem state and restoring the system. Don’t assume that it can be re-created if you haven’t verified that it can be re-created. If you cannot or have not re-created the problem, it is important to note that in the defect remarks. imPACt What is the impact if the bug were to surface in the customer environment? The impact of some bugs is self-evident: “Entire system crashes when I hit the Enter key.” Some bugs are not so obvious. For example, you may discover a typo on a window. This may seem trivial unless you point out that every time someone uses your product this is the first thing he sees, and the typo results in an offensive word. In this case, even though it is just a typo, it may be something that absolutely must be fixed prior to shipping the product. Make your best judgment. If you think it is possible that this defect will not get sufficient priority, then state the potential impact and sell the importance of the defect. Don’t oversell, but make sure the readers of the defect report have an accurate understanding of the probable impact on the customer. re-CreAte Some bugs are easy to re-create, and some are not. If you can re-create the bug, you should explain exactly what is required to do the re-create. You should list all the steps, include the exact syntax, file names, and sequences that you used to encounter or re-create the problem. If you believe that the problem will happen with any file, any sequence, etc., then mention that but still provide an explicit example that can be used to do the re-create. If in your effort to verify that the bug is re-creatable you find a shorter and more reliable means of re-creating, document the shortest, easiest means of re-creation. If you cannot re-create the problem, Table 4: Generalize example 34 BETTER SOFTWARE SEPTEMBER 2008 www.StickyMinds.com http://www.StickyMinds.com
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.