Employee training and development 5th chapter 06

Chapter 6
Training Evaluation
McGraw-Hill/Irwin

Copyright © 2010 by the McGraw-Hill Companies, Inc. All rights reserved.


Introduction
 Training effectiveness - the benefits
that the company and the trainees
receive from training.
 Training outcomes or criteria measures that the trainer and the
company use to evaluate training
programs.

6-2


Introduction (cont.)
 Training evaluation - the process of
collecting the outcomes needed to
determine if training is effective.
 Evaluation design - collection of
information, including whom, what, when,
and how, for determining the
effectiveness of the training program.

6-3


Reasons for Evaluating Training

Companies make large investments in
training and education and view them as a
strategy to be successful; they expect the
outcomes of training to be measurable.
Training evaluation provides the data
needed to demonstrate that training does
provide benefits to the company.
It involves formative and summative
evaluation.
6-4


Reasons for Evaluating Training
(cont.)
Formative evaluation - takes place
during program design and development.
It helps ensure that the training program is
well organized and runs smoothly, and
trainees learn and are satisfied with the
program.
It provides information about how to make the
program better; it involves collecting
qualitative data about the program.
6-5


Reasons for Evaluating Training
(cont.)
Formative evaluation
 Pilot testing - process of previewing the

training program with potential trainees and
managers or with other customers.

6-6


Reasons for Evaluating Training
(cont.)
Summative evaluation - determine the
extent to which trainees have changed as
a result of participating in the training
program.
It may include measuring the monetary
benefits that the company receives from the
program.
It involves collecting quantitative data.

6-7


Reasons for Evaluating Training
(cont.)
A training program should be evaluated:
To identify the program’s strengths and
weaknesses.
To assess whether content, organization, and
administration of the program contribute to
learning and the use of training content on
the job.
To identify which trainees benefited most or

least from the program.
6-8


Reasons for Evaluating Training
(cont.)
A training program should be evaluated:
To gather data to assist in marketing training
programs.
To determine the financial benefits and costs
of the program.
To compare the costs and benefits of:
training versus non-training investments.
different training programs to choose the best
program.

6-9


Figure 6.1 - The Evaluation
Process

6-10


Table 6.1 - Kirkpatrick’s Four-Level
Framework of Evaluation Criteria

6-11

Outcomes Used in the Evaluation
of Training Programs
 The hierarchical nature of Kirkpatrick’s
framework suggests that higher level
outcomes should not be measured unless
positive changes occur in lower level
outcomes.
 The framework implies that changes at a
higher level are more beneficial than
changes at a lower level.
6-12


Outcomes Used in the Evaluation
of Training Programs (cont.)
 Kirkpatrick’s framework criticisms:
Research has not found that each level is
caused by the level that precedes it in the
framework, nor does evidence suggest that
the levels differ in importance.
The approach does not take into account the
purpose of the evaluation.
Outcomes can and should be collected in an
orderly manner, that is, measures of reaction
followed by measures of learning, behavior,
and results.
6-13

Table 6.2 - Evaluation Outcomes

6-14


Outcomes Used in the Evaluation
of Training Programs (cont.)
 Reaction outcomes
It is collected at the program’s conclusion.

 Cognitive outcomes
They do not help to determine if the trainee
will actually use decision-making skills on the
job.

 Skill-based outcomes
The extent to which trainees have learned
skills can be evaluated by observing their
performance in work samples such as
simulators.

6-15


Outcomes Used in the Evaluation
of Training Programs (cont.)
 Return on investment
Direct costs - salaries and benefits for all
employees involved in training; program
material and supplies; equipment or

classroom rentals or purchases; and travel
costs.
Indirect costs - not related directly to the
design, development, or delivery of the
training program.
Benefits - value that the company gains from
the training program.
6-16


Determining Whether
Outcomes are Appropriate
Criteria
Relevance

The extent to which training outcomes are related to the
learned capabilities emphasized in the training program.
Criterion contamination - the extent that training
outcomes measure inappropriate capabilities or are
affected by extraneous conditions.
Criterion deficiency - the failure to measure training
outcomes that were emphasized in the training objectives.

Reliability

The degree to which outcomes can be measured
consistently over time.

Discrimination

The degree to which trainees’ performance on the
outcome actually reflects true differences in performance.

Practicality

The ease with which the outcome measures can be
collected.
6-17


Figure 6.2 - Criterion Deficiency,
Relevance, and Contamination

6-18


Figure 6.4 - Training Program Objectives
and Their Implications for Evaluation

6-19


Evaluation Designs
 Threats to validity - factors that will
lead an evaluator to question either the:
Internal validity - the believability of the study
results.
External validity - the extent to which the
evaluation results are generalizable to other
groups of trainees and situations.

6-20


Table 6.7 - Threats to Validity

6-21


Evaluation Designs (cont.)
Methods to Control for Threats to Validity
 Pretests and Posttests
A comparison of the posttraining and pretraining
measures can indicate the degree to which trainees
have changed as a result of training.

 Random assignment - assigning employees
to the training or comparison group on the
basis of chance.
Helps to reduce the effects of employees dropping
out of the study, and differences between the
training group and comparison group in ability,
knowledge, skill, or other personal characteristics.
6-22


Evaluation Designs (cont.)
Methods to Control for Threats to Validity
 Using a comparison group - employees who
participate in the evaluation study but do not

attend the training program.
Helps to rule out the possibility that changes found
in the outcome measures are due to factors other
than training.

6-23


Table 6.8 - Comparison of
Evaluation Designs

6-24


Types of Evaluation Designs
Time series - training outcomes are
collected at periodic intervals both before
and after training.
It allows an analysis of the stability of
training outcomes over time.
Reversal - time period in which
participants no longer receive the training
intervention.
6-25