When determining whether or not to continue, retire, or modify training programs, a formal evaluation using a model, such as Kirkpatrick’s Four Levels of Evaluation, can illuminate the effectiveness of training and potential areas of improvement. My competency in the challenge area of “Implement summative evaluation plans” can be seen in my EDCI 577 – Evaluation and Instructional Design evaluation plan project. This project was completed in partnership with another graduate student and showcases our combined efforts in establishing a summative evaluation plan, including evaluation instruments specific to the training program evaluated. Please note that the company name and other identifying details have been changed or redacted. Additionally, all evaluation data is hypothetical, as course timing did not allow for the gathering of actual data.

The importance of evaluating training programs is critical to understanding the value it brings to the organization and learners. The overall understanding of a program’s effectiveness allows organizations to determine their next steps, whether that is continuing with the program as-is, retiring it due to ineffectiveness, or revising it to address areas of concern that were highlighted by the summative evaluation plan. Evaluating training programs can contribute to better learning and performance outcomes.

In this project, my partner and I used Kirkpatrick’s Four Levels of Evaluation as our guiding framework. We utilized learner surveys generated through SurveyMonkey for Level 1 (Course Reaction) and facilitator interviews to gauge initial impressions of the training program. To evaluate the Content Learning (Level 2) of learners, several methods were used: a summative assessment in the final eLearning module (with a required passing score), a rubric assessing learners’ workshop activities, and surveys and interviews. Level 3 (Behavior Impact) was evaluated via surveys, interviews, and a Required Driver package that included supporting documentation, supervisor monitoring and feedback, and a recommended peer community. Training Results (Level 4) was conducted through workplace satisfaction survey results, focus group discussions, and applicable human resources data. To complete the course project, fictitious data was used. However, if the data showed what the fictitious data did, which was learner satisfaction, positive content learning results, workplace behavior modified after applying the workplace roles, and demonstrable positive impacts to workplace survey results and HR data metrics, the course would have been deemed effective and would have continued being offered to the target audience.

This project represented a new skill for me. I did not have much prior experience with the evaluation process. I believe this is due, in part, to my work history. I have worked as an instructional designer contractor for the majority of my career. In this capacity, I am often engaged on a contract during the design, development, and implementation phase. However, evaluation is often carried out after conclusion of my contract. However, becoming familiar with the evaluation process will make me a stronger and more skilled instructional designer. Understanding the evaluation process and possible metrics that will be used to evaluate a program allows me to design the program so it is aligned with the learning objectives and evaluation metrics, promoting a cohesive and focused learning experience through all stages of the ADDIE process (the most common model used in my experience).

The full evaluation plan uses the Kirkpatrick model effectively, as evidenced by the instruments designed and their (hypothetical) application to the learning program. A key growth area for me as a designer is in the area of learner summative assessment. Through this process, I came to understand assessment as more than verifying knowledge acquisition. My thinking evolved to view assessments as an integrated component of evaluating the effectiveness of training, which includes behavior change and verifiable results, which impact each learner as well as the organization as a whole. As I continue to practice instructional design, my efforts will continue to focus on effective assessment as an organic part of effective instruction and evaluation.