The Kirkpatrick model and Learning Effectiveness Measurement (LEM) are two of the most well-established methods to evaluate results of educational investments. The methods has both strengths and weaknesses, however in this article’s second part I will describe third relatively new method, STUDENT (Enterprise Learning Evaluation of Value), that to a large extent solve the deficits that Kirkpatrick and LEM have.
Kirkpatrick model contains four levels of evaluation, namely Reaction, Learning, Application and Outcome. A research study on the practical use of the method, which involved seven multinationals, including Coca-Cola Europe, BP and Vodafone, shows that the four different levels is given different priority with a fading attention from the first level and onwards. This characterised all companies in the study. In many cases, the evaluation ended at the first level – Reaction. The analysis thereby was limited only to the question of whether the student had participated or not. In other cases, there was a limited analysis of level 2-3, Learning and Application, while the fourth level, the outcome of education, for the most part was ignored completely. This implies that the training effect and economic outcomes was not analyzed at all, consequently the evaluation did not give any basic information for future educational investments.
In general the Kirkpatrick model too much is floating on the surface. The analysis assumes that the evaluator evaluate course by course without systematics. With this conclusion and the potential of new technologies in fresh memory, it is appropriate to question this evaluation method, which to a large extent has shaped particularly large enterprise education departments since the introduction 1959.
Learning Effectiveness Measurement (LEM)
Edward de Bono writes: “The past can be analyzed. The future must be created.” An effective evaluation method should therefore contain both these ingredients. The focus is then not set on defending what has been accomplished in the past, but to analyze and develop the values that will guide future investments. All these conditions are fulfilled with the LEM method that was developed by Dr Dean Spitzer at IBM.
The LEM method turns the evaluation chain upside down. The focus is set on the beginning of the course rather than after its completion. The main part of the evaluation-force thereby is directed on what should be achieved based on a number of key criteria. The evaluation is thus a continuous process which is directly interconnected with the learning process. This provides great opportunities for education officials to change the course objective and focus during the time of the training. Since the focus of evaluation is set on the organisations real-time demand a continuous comparison between the requirements and outcomes is created. In practice the educational investment is transformed from a course with a traditional curriculum to a project for competence development.
Methods to evaluate education
However … this requires a considerable and qualitative counselling and evaluation function, which is both time-consuming and costly. What the organization wins on the swings it looses on the carousel. Because of these shortcomings we instead continue to the third of the surveyed methods. The British consulting firm eLearnitys evaluation model Enterprise Learning Evaluation of Value (STUDENT), which combines the best of Kirkpatrick and LEM. A slightly revised version of the STUDENT method is described in the second part of this article that will be published in the beginning of the next week.