Chief Learning Officer - January 2008 - (Page 50)

business intelligence However, there are practices we are beginning to employ on the other side of the evaluation coin, during needs assessment. We have built into our process a collaboration with key stakeholders to identify what type of information they would need in order to view our evaluation efforts as such, valid and convincing. For example, when engaged by a client to develop a new training program, we are working to identify the “felt need” in terms of impact on the business. Where is the “pain”? How is it affecting your business? What will be different if we implement a training program? What will people be doing differently? and customize to meet our particular needs. Standards were created using Kirkpatrick, Phillips, Shrock and Coscarelli, Norton and Kaplan, Hodges-DeTuncq and Dana Gaines Robinson, to name a few. We use an applied social research methodology with both quantitative and qualitative data analysis. Our tests face rigorous development standards following the Shrock and Coscarelli methodology. We use Level 1 evaluation data to report not only training effectiveness but also leading indicator data on knowledge acquisition and on-the-job application of skills learned. Our senior management is very interested in the knowledge acquisition scores as insight into talent management. This is a cost-effective and simple method to track learning indicator data. We have also reported for over three years across our key business lines using a balanced scorecard following the Norton and Kaplan methodology. In particular, we report on business partner satisfaction, and employee engagement and satisfaction of our training investment. We are in a review cycle for reporting “next generation” impact data. We have tested Level 3 data collection on specific strategic initiatives. We are currently testing facilitator follow-up calls post training to capture both quantitative and qualitative data to measure the impact the training is having on the job for management-level employees. We are currently developing an enterprise scorecard for leadership and management development measuring Level 1-4 impact data. Historically, we tested ROI forecasting and an ROI in our retail line. We strategically moved away from having Level 5. We determined the amount of resource effort and dollar spend was not adding value and chose to measure what we can control — Levels 1-3 and, selectively, Level 4. Our strategy now focuses on determining with our business partners what “impact” data is and measuring against that. 9. Electronics Company This continues to be a difficult hard-dollarquantifiable measure. We have a strong “Leaders as Teachers” culture and will be further developing and deploying that methodology into the middle-manager ranks. At the executive levels, the “faculty” (C-level and their direct reports) continue to use anecdotal follow up. The majority of our classes have an action item that’s followed up on at a pre-stated interval: 30-4560-90 days later. At that time, the participants themselves supply supporting documentation and statements to indicate what was achieved. We use this along with Level 1 evals for the faculty to review. Over time, the behavior changes begin to be noted in performance appraisals, development plans and observations by faculty who actually teach the classes and continue to interact with the senior leaders. This year, we are looking to do surveys and slowly reintroduce a multirater feedback option. Three years ago, I presented to the executive team and faculty how we could approach doing detailed studies. The overwhelming support was to keep with the anecdotal and trend it. We are now beginning to do that. In fact, we have begun second generation, and in some cases a third generation, of a course in areas such as customer service, ethics, strategic thinking, operational excellence, etc. The enthusiastic support by the senior leaders and reception (registration) of the participants reflects the tone from the top. Of course, we’d like to have more dashboard metrics, and we do provide some of the basics. We too would like to know how others can do it. Warren Wilhelm is president of the Global Consulting Alliance and Wilhelm Consulting Partners. He was chief learning officer for AlliedSignal Corp., and head of organization and management development for Amoco Corp. His latest book, Learning Architectures: Building Organizational and Individual Learning is in its second printing. He can be reached at editor@clomedia.com. January 2008 I www.clomedia.com I Chief Learning Officer 50 11. Manufacturing Company Our company university has conducted one success case study that has been accepted by our senior leadership as valid, rigorous and supportive of further investment in learning. Our immediate plans call for two additional studies to be conducted, which are supportive of our long-range evaluation strategy. Our success cases follow Rob Brinkerhoff’s methodology. 10. Financial Company We incorporate standard measures into all strategic training initiatives. A measurement department was created in 1997 to research and adapt the best practices in the industry http://www.clomedia.com

Table of Contents Feed for the Digital Edition of Chief Learning Officer - January 2008

Chief Learning Officer - January 2008 Editor’s Letter Table of Contents Imperatives Selling up, Selling Down Strategies Take Five Environment Sips of Knowledge at E. & J. Gallo Winery CLO Profile Productivity UST Global: Opening Employees’ Eyes to New Learning Tactics Applying CRM Concepts to E-Learning Human Capital Capital One: Experiences in Innovation Learning Solutions Macy’s: Using Feedback to Develop One Leader at a Time Case Study Business Intelligence Advertisers’ Index Editorial Resources In Conclusion

Chief Learning Officer - January 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.