Better Software - December 2008 - (Page 36)

It is better to know some of the questions, than all of the answers. – James Thurber Recently, Lee Copeland and I completed a test process assessment for a major financial services company. Our assignment was to understand this organization’s current testing process and make recommendations for improvements. The foundation of any successful assessment is interviewing a diverse cross section of the staff. We interviewed team members who were involved in current software testing projects. We spoke with people from different project types including new development, product updates, and emergency releases. We interviewed people working with different technologies such as mainframe, clientserver, Web, and voice technologies, and we met with team members from different office locations. Listening to this varied group helped us build a picture of this organization’s software testing process. Although Lee and I have different professional experiences, our elicitation techniques are quite similar. We interviewed team members, carefully listening to their stories, and took detailed notes. Through their responses, we were able to piece together a clear and realistic picture of our client’s testing process. As we reviewed our notes to analyze the situation, it occurred to us that we could also analyze our questions to learn more about how we perform interviews and how we could improve our own elicitation processes. additional stories and any important, omitted details. We asked interviewees to share stories about recent critical incidents. We sought examples of excellence or demonstrations of failings: “Tell a story about a project that worked really well”; “Tell us a story about a project that failed miserably.” We also asked for stories of typical project experiences. These stories helped us identify several objects of testing, including plans, documents, reports, test cases, and other artifacts. Once we understood the objects of testing, we could relate them to the actions of testing. “What do you do with these objects?” “How do you create them?” “How are they processed?” Often the nouns of the narrative told us what the objects of the testing project were. Often the verbs of the narrative told us what actions were done to these objects. We inquired about the sequence of actions and the evolution of objects, so we were able to compare and contrast process flow between similar stories from different sources. Differences led to rich comparative questions, which helped us ask test leads and managers “why?” After analyzing the client’s current practices and writing test process improvement recommendations, we returned to our interview notes and undertook a thorough analysis of our questioning styles. We made a list of the questions we had asked, organized them into categories, and sought to discover commonalities and flows. The following describe the types of questions we found effective. cussion. We made sure they knew why we were there. We explained our commission from their management, what approach we were using, and what we were trying to learn from the information we collected. We also explained why they were there—that they had been selected because of their in-depth knowledge of the organization’s processes. During the interview, we took notes as the interviewees related their stories. At the beginning of the interview we mentioned that we were going to take notes so that we could capture and remember the details of their stories. We also stated that they could and should speak freely about their organization— that their names would not be attached to any negative comments they made. It is absolutely vital that this promise be strictly honored. Our handwritten notes were of two types—directly related to the test process improvement model we were using and free-form notes recording anything interesting we learned. Rob also used mind maps to create a visual outline of their stories. Mind maps allowed us to create a visual understanding of the testing process described by each interviewee and helped us detect missing actions, objects, inconsistencies, and asymmetries in each story. See the StickyNotes for more information on mind maps. We often began the interview with these questions: “Do you know who we are and why we are here?” and “Do you know why you are here?” Interviewees would tell us what they thought we were doing, then we discussed and clarified our purpose before diving into the interview. We wanted to show respect to the interviewees, so before starting in with questions, we thanked them for taking time out of their chaotic schedules to meet with us. Our Elicitation Process While we had a formal list of topics we needed to ask about, most of our questioning was exploratory. We wanted to learn about the client’s testing realities, so we captured information without bias or interpretation. As we learned, we tried to uncover more of the narrative. We asked further questions based on our findings: Open-ended questions led to rich stories; focusing questions closed gaps capturing important, missing details; and closing questions helped wrap up the interview. We wanted to identify 36 BETTER SOFTWARE DECEMBER 2008 setting the stage Interviewees need comfort; the appearance of outside consultants may appear threatening. Interviewees are not sure why they have been “summoned to appear,” and they may be worried that the things they say will be used against them at some future time. We began by putting the interviewees at ease by setting the stage for our diswww.StickyMinds.com building raPPort “I didn’t expect a kind of Spanish Inquisition” is a classic line from a Monty Python skit. We didn’t want interviewees to feel they had been dropped into the http://www.StickyMinds.com

Table of Contents Feed for the Digital Edition of Better Software - December 2008

Better Software - December 2008 Contents Mark Your Calendar Contributors eLightenment Technically Speaking Code Craft Test Connection Management Chronicles What's a Manager to Do? Six Thinking Hats for Testers The Key to Good Interviewing 2008 Salary Survey Product Announcements 10 Things You Might Not Know About … The Last Word Ad Index

Better Software - December 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.