Evaluation Instrument Design Services

An effective evaluation strategy is a critical component of a successful training intervention, particularly where a competency-based solution is mandated.

Four levels of evaluation are generally recognized in the training industry, based on a model pioneered by Donald Kirkpatrick in 1959. These levels are incremental in complexity, difficulty of implementation, and cost.

A Level 1 evaluation does not measure learning but captures feedback about how participants felt about the training. Level 1 evaluations are fast and easy to design and implement.

A well-designed Level 1 evaluation can provide valuable information on the effectiveness of the training delivery. It captures participant opinions on:

• course delivery (pacing, style)
• course content (sufficient content, appropriate content, clear explanations, etc.)
• training facilities and equipment
• course materials (participant guides, handouts, visuals, etc.)

 

A Level 2 evaluation measures the learner's success in acquiring the skills and knowledge taught in a lesson or module. It identifies which subjects or tasks the learner has mastered, and which ones require more review and practice.

An effective Level 2 evaluation should measure all the learning objectives for the lesson or module.

Evaluation activities should also align as closely as possible to the learning objectives, particularly when the objectives are performance-based. For example, if the objective is to have a learner log on to a computer or put out a small kitchen fire, then the evaluation activity should be to actually log on or put out a fire.

 

A Level 3 evaluation measures how the learner's on-the-job behavior has changed as a result of a training intervention. It identifies how well the learner has applied his/her new knowledge and skills to the real-world tasks and environment.

Level 3 evaluations are understandably more challenging to design and implement, requiring systematic appraisal of on-the-job performance before and after training. The post-training appraisal should take place only after learners have had sufficient time to master their new skills and incorporate them into their on-the-job activities.

 

A Level 4 evaluation measures the impact of the training on the organization's business goals by tracking and quantifying indicators such as costs, production rates and quality, customer satisfaction, and rates of employee turnover or absenteeism.

Because other factors can affect these indicators, it is almost impossible to isolate the effect of training on business goals. Nonetheless, informal and anecdotal evidence of changes can provide valuable insights into the training program's effectiveness.

 

Evaluation instruments such as tests, quizzes, questionnaires, surveys, and interviews can be used to measure:

• subjective learner response to instructors, facilities, and training materials
• employees' abilities prior to training, in order to identify individualized training needs
• employees' abilities upon completing a training intervention
• the success of an organization's overall training plan

 

Organizations should consider developing evaluation instruments whenever a business issue or performance gap can be traced to insufficient employee skills and knowledge.

A diagnostic evaluation instrument can help identify specific performance gaps, which can then be addressed through training or other performance support strategies.

Post-training evaluation instruments can help an organization measure whether desired training outcomes have been achieved.

At the very least, a well-designed Level 1 evaluation at the end of a training program can help assess instructor performance and measure learner satisfaction with course content, pacing, and other variables.

Effective evaluation instruments flow directly from your training objectives, and should be updated whenever there is a change in an organization’s business processes. This ensures that potential new performance gaps are identified and addressed in a timely and cost-effective fashion.

 

BrainWave team members have decades of experience in the design and development of paper and electronic evaluation instruments as part of larger training interventions.

We can design a range of customized evaluation instruments for your organization, depending upon your budget and business objectives. These include:

• Levels 1, 2, and 3 evaluations
• pre-course diagnostic questionnaires
• observation surveys/ behaviour scorecards
• interview guides and scripts

Tell me more about your overall client portfolio...

E-mail us for more information...