Elearning - April/May 2008 - (Page 25)

assessment responses, communicate learning gaps to learners and deliver targeted “tune-up” content designed to close identified gaps. The immediate and targeted distribution of tune-up content clearly has a positive effect on learning transfer data. Level III evaluation, focused on measuring behavior change back at work, required a slightly different solution for our public seminar and corporate businesses. In our public seminar business, we normally do not have direct access to the organization to effectively measure Level III. We can ask learners — through surveys, for example — how they think they are bringing the learning back to the office. Our goal is to strengthen Level III evaluation in a public seminar environment by connecting with the learner’s manager and asking questions to determine the level of behavior change. Level II evaluation for our corporate solutions is much easier to measure. We have access to the organization, so we can determine with our clients how we’ll approach Level II evaluation. Once the methodology is finalized, we can deliver content in the modality that meets the client’s expectations. RETURN ON EXPECTATIONS The AMA identifies Level IV evaluation using the most recent definition, which is focused on measuring return on expectations. You might be concerned if you were asked to provide return on investment data (the formerly accepted definition of Level IV) after a training intervention. Rightfully so, many outside variables can effect how a team performs. Was it the training? Was it a recent bonus? Was it acknowledgement from the CEO? It is a monumental task to “prove” the change resulted solely from training. Return on expectations is a much more manageable and effective definition. For corporate clients, we can certainly identify key expectations before the intervention begins. We can agree on how we’ll measure success and how we’ll identify success. Level IV is not used for our public seminar clients. In a public seminar setting, the learning experience is focused on the individuals’ development. We certainly identify expectations for the learner at many points in the learning experience but the collec- tive, organizational data is not available. The biggest challenge in creating and delivering a solid measurement approach is not the creation of assessments, validating assessments, building and tuning up content, implementing a LCMS and LMS or designing reports; it is getting learners to complete the assessments and content. Measurement requires that the learners, managers and leaders take ownership and responsibility for each level of evaluation. Learners and managers must complete pre- and post-learning surveys and assessments. Leaders must clearly identify expectations, success and evaluate the data to determine what worked and what needs more focus and attention. —Nicholas Igneri is Director of Learning Technologies for the American Management Association. Contact him via e-mail at nigneri@amanet.org. Elearning! April / May 2008 25 http://www.the-lc.org http://www.the-lc.org

Table of Contents Feed for the Digital Edition of Elearning - April/May 2008

Elearning - April/May 2008 Editor's Note Contents News Targeting Africa Locating Courseware People in the News Upcoming Events Deals Trendlines Web Services Building a Learning Culture Measuring the Learning Experience Making Learning Fun and Social LMS's 2008: What You Need To Know Learning Leader: CA Case Study: AVI Case Study: IBM Case Study: QualComm WBS Case Study: National Center for State Courts Case Study: Grundfos Pumps Case Study: Virginia Tech Pop Quiz Last Word

Elearning - April/May 2008

For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.