Chief Learning Officer - January 2008 - (Page 49)

business intelligence cracked the code.” However, upon reading the research study in more detail, I may have some “attempts to crack the code” to share. My response is tailored to the way in which this study defines “return on investment in learning,” specifically measured by desired behavioral change of the organizations’ members; improved organizational performance measured both financially and in terms of organizational capability; and in creating and sustaining a “learning organization.” We are currently working on several evaluation projects to support our learning functions in demonstrating and maximizing the value of their interventions. The evaluation strategies are developed using a hybrid of Kirkpatrick’s Four Levels of Evaluation and Brinkerhoff’s Success Case Method. For example, to evaluate a manager development program, we are employing the following strategy: • Standard L1 survey aimed at capturing quantitative and qualitative data relevant to the participants’ experience in the program, as well as a few leading indicators for how they might apply what they’ve learned on the job and any anticipated challenges to performance. We ask for an estimation of the extent to which they have gained new knowledge/skills, and to what extent they intend to apply these on the job. • L2 (assessment of learned knowledge/skills) happens informally during the training program. This process is made possible and deliberate through the careful development of learning and performance objectives for each module. The facilitators of the program are then “trained” to monitor performance through each activity during the learning experience. • L3 comes in the form of an electronic survey. This is sent out 30-days post-class, our rationale being that the participants would have had sufficient time to apply the learned skills on the job. Again, we capture both quantitative and qualitative data relevant to the participants’ experience applying new knowledge/skills on the job. We ask for self-assessment feedback about the extent to which they’ve been able to apply what they learned and for specific examples of their application. Those who leave us with a “success story” are encouraged to leave their name for follow up. • Success Case Method — Interview: We have developed two interview protocols to support consistent and detailed follow up with those participants who have asked to be contacted on the L3 survey. The first protocol is intended to dive deeper into the success cases, getting at specific behaviors, enablers in the work environment and seen impact on the operation (at the individual, team or departmental level). The second protocol is intended to gain insight into those who have indicated non-success, or a challenge in applying the new skills. The focus here is on barriers in the work environment. We intend the qualitative data from these interviews to yield tangible stories that demonstrate the impact of our learning interventions. We’ll be able to trend best practices for how the operation can support participants once they leave the training environment. Additionally, past participants will be invited back to future classes to share best practices for overcoming challenges and applying new skills with other participants (supporting organizational capability). I communicate our efforts in the previous example as an attempt, in part due to the fact that we have not yet employed the entire evaluation strategy. Therefore, I’m not certain how the data will be accepted in terms of being seen as valid and convincing. The impact of our efforts will be fully realized when we’re able to report and communicate results back to the stakeholders. 49 January 2008 I www.clomedia.com I Chief Learning Officer http://www.clomedia.com

Table of Contents Feed for the Digital Edition of Chief Learning Officer - January 2008

Chief Learning Officer - January 2008 Editor’s Letter Table of Contents Imperatives Selling up, Selling Down Strategies Take Five Environment Sips of Knowledge at E. & J. Gallo Winery CLO Profile Productivity UST Global: Opening Employees’ Eyes to New Learning Tactics Applying CRM Concepts to E-Learning Human Capital Capital One: Experiences in Innovation Learning Solutions Macy’s: Using Feedback to Develop One Leader at a Time Case Study Business Intelligence Advertisers’ Index Editorial Resources In Conclusion

Chief Learning Officer - January 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.