Better Software - December 2008 - (Page 17)

Test Connection designed to expose the problems that you’ve anticipated, checking off existing risks or tests, while keeping your eyes and mind open to new risk ideas. This approach can help drive coverage toward problems that matter. Our own hypotheses about risk are valuable, but we’re all likely to be limited and constrained by our biases, so work with other people and use bug taxonomies or cheat sheets (see below) to generate fresh ideas. Ongoing testing may suggest that we’re well-defended against certain risks and highly exposed to others, so revisit, review, and revise the risk list frequently to identify what’s been covered and what hasn’t. Completely cover some defined corner of the domain space. Doug Hoffman’s story of the integer square root function on the MASPAR processor is a case in point. He considered a number of coverage models to reduce the number of tests to run, and then it occurred to him: Why not try all 4,294,967,296 possible integer inputs? Using automation, he prepared a test that covered the entire input domain for that particular function in a few minutes. This wasn’t complete test coverage for the whole processor, nor even for all of the possible risks for that function (like stress or flow or performance problems), but he did cover the entire map of its input values. Map operational models, use cases, or tasks. Use cases or business process workflows can be useful in identifying places where we need to test. Whether you’re provided with a list or develop one yourself, you can devise tests to cover the list. On the other hand, it’s important to question use cases. I’ve seen a lot that are very tidy and heavily idealized, but I’ve never seen one that describes how people actually work in practice. Things are rarely as messy in a use case as they are in the real world. Use a set of heuristic guidewords or test ideas. James Bach’s Heuristic Test Strategy Model, Elisabeth Hendrickson’s Test Heuristics Cheat Sheet, and Michael Hunter’s You Are Not Done Yet models are all excellent checklists for guiding exploration of some aspect of some model of the program. Use these or develop your own models. Vary the product elements you look at, the quality criteria you look for, and the test techniques you perform. Plot tests against coverage ideas as you … Work from a test matrix. Prepare a spreadsheet. Scribble a list of test ideas down the y-axis and list aspects of some test coverage model—product elements, quality criteria, platforms, test techniques—across the x-axis. As an experiment, create multiple sheets using the same tests, but with a different coverage model on each sheet, and observe how a single test can provide coverage in a number of different dimensions. For a given coverage model, denser coverage of the matrix suggests (but does not prove) deeper coverage of the particular set of ideas on that table. Quantitative measures of coverage can be troublesome because they are so easily subject to reification error— treating conceptual things like test cases or requirements statements as though they were units instead of containers for ideas. When we apply models, though, we begin to enter a qualitative world. Qualitative evaluation of coverage can be troublesome, too, because quality— “value to some person,” in Jerry Weinberg’s definition—is subjective, indefinite, and uncountable. But test completeness is always a subjective and arbitrary concept. Any map tells you something you might want to know, but no map can tell you everything. So we need to develop diversified sets of ideas about coverage and how we map it. Then we check, explore, and compare them to confirm what we believe we know, to guide discovery of what we don’t, and to help tell the story of where we’ve been and what we’ve done. {end} What do your maps look like? How do you describe coverage to clients and your project community? Follow the link on the StickyMinds.com homepage to join the conversation. www.StickyMinds.com DECEMBER 2008 BETTER SOFTWARE 17 http://www.StickyMinds.com http://www.rallydev.com/bsm http://www.rallydev.com/bsm http://www.StickyMinds.com

Table of Contents Feed for the Digital Edition of Better Software - December 2008

Better Software - December 2008 Contents Mark Your Calendar Contributors eLightenment Technically Speaking Code Craft Test Connection Management Chronicles What's a Manager to Do? Six Thinking Hats for Testers The Key to Good Interviewing 2008 Salary Survey Product Announcements 10 Things You Might Not Know About … The Last Word Ad Index

Better Software - December 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.