Questions for Kirkpatrick: Why Measure Training?
October 02, 2008
Author and speaker Don Kirkpatrick first defined a four-level model of training evaluation in 1959. Here, he answers questions about his model from Training readers.
By Don Kirkpatrick
Q: Are there times when education is just intrinsically valuable, and outcomes cannot be measured? For example, a company has hundreds of service delivery personnel around the world. They have to know about many products. Clearly, training plays a large role in preparing them and has a tremendous value. Why measure?
— Michelle Lavoie, EMC
DK: It is important to measure all training for at least two reasons: Was the training effective in teaching the information on the many products—in other words, did the trainees learn the important information about the products? And, if they didn't, why not and how can the training be improved?
In addition, how do you know "training has a tremendous value" unless you measure change in behavior and results because of the training?
Q: Please talk about your methodology vs. Phillip's ROI Methodology—do they compliment one another?
—John R. Mattox, II, Ph.D., KPMG
DK: Jack Phillips has adopted my four levels with two changes in wording. First, he calls Level 3 "Application" and Level 4 "Impact." Unfortunately, although these words mean the same as the ones I use, "Behavior" and "Results," it tends to confuse people.
He then created his approach to ROI and calls it Level 5. I originally included ROI in Level 4, "Results." Regarding ROI, I am afraid that many trainers have picked up the term; learned a little about it; and think it is the answer to the question, "How effective is the training?" And there are many aspects of ROI that are questionable.
My son, Jim, and I adopted the term, ROE, "Return on Expectations." This means we start with the jury stating what they expect to result from the training and work backwards to determine what to teach. Then we measure the effectiveness against the expectations.
Q: How do you suggest dealing with the constraints of confidentiality and personally identifiable information and yet still implement a rigorous evaluation that upholds the standards for evaluation and measurement?
—John R. Mattox, II, Ph.D., KPMG
DK: First of all, in evaluating Levels 1 and 2, all the information is anonymous. There should be no attempt to evaluate each trainee's evaluations.
In measuring Level 3, the matter of trust is important. For example, when a person from the training department interviews or surveys a trainee and /or the manager to measure change in behavior, the trainer must:
a. Tell the interviewee to be honest and don't be afraid to say that no behavior change has taken place.
b. Convince the interviewee that the information will be kept anonymous.
If you'd like your question answered online by Don Kirkpatrick, please e-mail it to Training Editor-in-Chief Lorri Freifeld at lorri.freifeld@nielsen.com. If you'd like to contact Kirkpatrick directly, he can be reached at Dleekirk1@aol.com.
|
RSS
