The Official Blog of Benifys HR Solutions

Delivering a training program is not enough, the cycle does not end there. Measuring the effectiveness of training programs gives a realistic idea of how effective the training program really has been, it’s technical preciseness nowithstanding. Today, we look at one of the most widely used tool for this purpose, which is Donald Kirspatrick’s four level model, for evaluating the impact of training programs on short term or long term goals of the organization.

The need to measure training effectiveness

• To gauge the effectiveness of the current ongoing training programs in an organization
• To identify gaps in training output and use the information to improve the training initiatives
• To evaluate whether the budget allocated for the training activity is justified, and whether the program has a good ROI.

Donald Kirkpatrick and his Four Level evaluation model

Donald L Kirkpatrick, Professor Emeritus, University Of Wisconsin (where he achieved his BBA, MBA and PhD), first published his ideas in 1959, in a series of articles in the US Training and Development Journal. Kirkpatrick then went on to write a book (first published in 1975) called “Evaluating Training Programs” in which he devised a new model that he called the four levels of evaluating training program.

Now widely recognized as the authority on measuring training effectiveness, his four level theory has since then gained worldwide popularity and is arguably the most widely used model for the evaluation of training and learning programs. Kirkpatrick’s four-level model is now considered an industry standard across the HR and training communities.

According to Kirkpatrick, each one of these four levels is equally important and every level has an impact on the next level. Also, as you move along one level to the next one, the process progressively becomes more difficult and time consuming, but it also provides more valuable information.

The Four Levels – A Brief Overview

Level 1 : Reaction

Kirkpatrick likes to call this step a measure of customer satisfaction, which is because this step involves gathering feedback from the trainees about they the felt about the training program, not much unlike how customers are asked for feedback about services. This step is important because it evaluates the engagement level and hence the receptivity of your audience. It is an understatement that without receptivity, the entire training exercise would be meaningless.

In his own words – “It is important not only to get a reaction but to get a positive reaction. The future of the program depends on positive reaction. In addition, if participants do not react favorably, they probably will not be motivated to learn. Positive reaction may not ensure learning, but negative reaction almost certainly reduces the possibility of its occurrence.”

Level 2 : Learning

By learning, Kirkpatrick means measuring the extent of change in the competency of the participant in terms of positive attitude change, improved knowledge and/or increase in skills as a result of attending the program. A lot depends on the use of an appropriate learning channel and the consideration paid while designing the training program to the needs of different learning styles of trainees (i.e., visual learners, kinesthetic learners, etc).

As Kirkpatrick puts it himself – “Some trainers say that no learning has taken place unless change in behavior occurs. Learning has taken place when one or more of the following occurs: Attitudes are changed. Knowledge is increased. Skill is improved. One or more of these changes must take place if a change in behavior has to occur”

Level 3 : Behavior

It is one thing to learn and absorb information and quite another to apply it to actual situations, this step is intended to measure the degree to which the trainee has internalized the learning. Some trainers tend to bypass the first two levels, i.e., reaction and learning, and get straight to measuring changes in behavior. This is a serious mistake. The problem is that the obvious conclusion that will be drawn will be that the training was ineffective and that it should be discontinued. This solution may not be very accurate, because even though the training might have been technically flawless and does well on the first two levels, it may not guarantee a change in behavior.

According to Kirkpatrick, four conditions are necessary to bring about behavioral changes:

1. The person must have the desire to change
2. The person must know what to do and how to do it
3. The person must work in the right climate
4. The person must be rewarded for changing

If no change in behavior is observed after training, the reaction and learning should be re-evaluated to check whether the training delivery or content was effective, if it was, then maybe the climate is not conducive to change. Trainers should actively find out what kind of climate the trainees will face after the completion of the training; a failure to provide the right climate is a major reason why most training programs don’t achieve their set objectives.

This is what Kirkpatrick says about providing the right climate – “One way to create a positive job climate is to involve bosses in the development of the program. It is suggested that that bosses should be asked to help determine the needs of sub-ordinates. Such involvement helps to ensure that a program teaches practical concepts, principles, and techniques. Another approach is to present the training program, or at least a condensed version of it, to the bosses before supervisors are trained”

Level 4 : Result

The final evaluation of the training program is measuring the final results that occurred because the participants attended the program. The various metrics it includes could be increase in production, decline in costs, reduction in employee turnover, higher profits. It is on these results that the scope and future of the training department depends on. It is important to mention here how pertinent it is to set objectives before the commencement of the training program, if this is not done right, it becomes very difficult to determine just how effective the training was.

In the words of the Master himself – “It is difficult if not impossible to measure final results for programs on such topics as leadership, communication, motivation, time management, empowerment, decision making, or managing change. We can state and evaluate desired behaviors, but the final results have to be measured in terms of improved morale or other nonfinancial terms. It is hoped that such things as higher morale or improved quality of work life will automatically result in tangible results”

A word of caution – implementing the Kirkpatrick model requires engagement and sponsorship of the executive management. Very often, the feedback derived from an assessment of the first two levels is positive, but the third and fourth levels leave a lot to be desired. Two primary reasons, as elaborated above, come in the way of training activities achieving their potential – lack of a conducive environment, and lack of a direct correlation between business results and the training activity. It is important for organisations to a) set clear objectives, b) obtain executive level approval of the content, c) provide concrete and visible support to the training activity and a playground to enable the participants apply the newly learnt skills at the workplace, and d) establish clear monitoring and reporting mechanisms so that the influence of non-training variables is minimized in the fourth level assessment.