Elearning! Fall 2007 - (Page 22) effectivee-learningassessments Research, Inc. When Thalheimer asked group of about 100 e-learning professionals the highest level of assessment they did, 11 percent said they conducted no evaluation and none said they measured return-on-investment. “Even after all the hot air expelled, ink spilled and electrons excited in the last 10 years regarding how we ought to be measuring business results, nobody is doing it,” Thalheimer says. “We’re in a dark fog about how we’re doing, so we have massively impoverished information to use to make improvements.” A LONG HISTORY? “Assessments have been around as long as e-learning — going on 20 years, back to floppy disks in late ‘80s,” says Marketing Manager Brian McNamara of QuestionMark. “Most authoring systems and LMS’s have some kind of component to create questions or quizzes. The level of sophistiThey must also be “medium neutral,” so that they can be used in different publication formats, like the Web, or paper-andpencil tests. And they must fit in available standards and specifications. According to Kathleen Scalise of the University of Oregon and Bernard Gifford of the University of California at Berkeley, a potential limitation for realizing the benefits of computer-based assessment is in designing questions and tasks with which computers can effectively interact while still gathering meaningful measurement evidence. Multiple-choice questions, Scalise and Gifford say, “limit the computer platform’s potential for rich and embedded assessment.” Fully constructed responses like a traditional essay “can be a challenge for computers to meaningfully analyze, even with today’s sophisticated tools,” they add. According to Mark Nichols of the Bible College of New Zealand, an effective assessment system can: >> encourage deep approaches to learning; >> correct student misconceptions; >> pace student progress; >> reflect on and improve course materials in terms of their effectiveness; >> motivate students to explore the subject matter further; “With the stakes so high, it’s important that businesses make sure they are properly assessing e-learning.” —Brian McNamara, QuestionMark cation varies.” In other words, the e-learning industry still has room for improvement. “The industry has come a very long way in the last 10 years,” says Hank Riehl, Director of Competency Management Solutions at SkillSoft. “There’s a wide range of competency libraries and batteries of criteria that form the foundation of these assessments and the basis of these models. A dozen have pretty widespread traction and market recognition.” More sophistication is definitely needed because companies are, more and more, making assessments for regulatory compliance. “An ability to document training and education, along with an audit trail, is important,” notes McNamara. “Also, more companies are establishing internal certification systems. So with the stakes so high, it’s important that businesses take all the 22 Fall 2007 Elearning! necessary steps to make sure they are properly assessing their e-learning efforts.” But organizations don’t need to reinvent the wheel. “They should look at assessment models based on the nature of the nature of the population being catalogued,” suggests Riehl. “Some assessment models are stronger than others in sales and marketing; others are stronger in information technology domains. Some competency models focus on particular disciplines; others are more broadly based, dealing with interpersonal and behavioral competencies but stay out of the harder domainspecific skills.” ASSESSMENT BEST PRACTICES Among other characteristics, effective assessment models must be flexible, formal, reusable, complete and reproducible.
For optimal viewing of this digital publication, please enable JavaScript and then refresh the page. If you would like to try to load the digital publication without using Flash Player detection, please click here.