6 Best Training Evaluation Models in 2024: Part 2

In part one, we explored three foundational models for evaluating training effectiveness: The Kirkpatrick Model, The Phillips ROI Model, and The CIPP Model. These models provided a comprehensive framework for assessing various aspects of training programs, from participant reactions to long-term results.

However, the world of training evaluation offers even more valuable tools to help you gain deeper insights. In this part two, we'll delve into three additional models that tackle training effectiveness from distinct angles: the DACUM Model, the Dreyfus Model, and the Brinkerhoff Model.

The Developing A Curriculum or the DACUM Model is a competency-based model that works best for job and task analysis, curriculum development, and training program design. It is a highly effective method that relies on the expertise of skilled workers to define the tasks and competencies required for a specific job or occupation.

The DACUM Model involves a panel of six to eight highly skilled employees who analyze the duties and tasks that make up their jobs under the guidance of the neutral facilitator. Here is the algorithm:

Panel members brainstorm and list all the tasks they perform in their jobs. This can include daily, weekly, and occasional tasks. Each task is recorded on index cards or sticky notes for easy organization and modification.

For each task, panel members provide a detailed description that includes the specific actions performed, the tools and equipment used, the conditions under which the task is performed, and the expected outcomes. They also identify the skills, knowledge, and attitudes required to perform each task effectively.

Panel members discuss and identify natural groupings of tasks which are typically further organized into broader categories called duty areas or job functions.

The members establish a hierarchical structure where tasks are categorized under relevant duty areas, creating a clear and organized framework.

Panel members share the organized task list with additional subject matter experts or stakeholders who were not part of the initial workshop to ensure the accuracy, relevance, and completeness of the task list through external validation. They collect feedback and revise the task list, making necessary adjustments to improve accuracy and completeness.

For each task, panel members formulate specific performance objectives that describe the desired outcomes and standards of performance. They should ensure that the performance objectives align with the identified competencies and the overall goals of the training or curriculum development.

As a result, they create a set of well-defined performance objectives that can be used as a basis for curriculum development, training program design, and performance assessment.

Thanks to expert opinions, using this model grants an accurate, thorough, and detailed understanding of job requirements and competencies. Another benefit of the model is its flexibility: it can be applied to various industries, occupations, job roles, and purposes, from performance assessments to designing training courses.

However, reaching a consensus among panel members can be challenging, especially in diverse occupational areas.

• Curriculum development:Educational institutions use DACUM to design competency-based curricula that align with industry standards and job requirements. The DACUM chart determines the development of course content, learning objectives, and assessment methods.

• Training Program Design:Businesses use DACUM to create targeted training programs that address specific job competencies. The chart helps identify training needs, design instructional materials, and evaluate their effectiveness.

• Job Analysis and Design:DACUM provides a detailed understanding of job roles, which can be used to develop job descriptions, performance standards, and recruitment criteria. It helps organizations streamline job functions and improve workforce planning.

• Performance Assessment:Employers use the DACUM chart to evaluate employee performance based on clearly defined competencies and tasks. The model provides a basis for developing performance appraisal tools and identifying areas for employee development.

The Dreyfus Model of Skill Acquisition, developed by brothers Stuart and Hubert Dreyfus in the 1980s, describes how individuals progress through different levels of skill acquisition from novice to expert. This model is widely used in education, professional development, and training to understand and facilitate skill development.

According to the model, a person goes through the following stages during their learning journey: novice, advanced beginner, competent, proficient, and expert.

At this stage, learners have little or no experience with the skill or domain. They rely heavily on rules and guidelines provided by others and have difficulty understanding the broader context. A bright example of a novice is a new employee learning to use a specific software by following step-by-step instructions.

Training programs for novices should provide clear, context-free rule-based instructions and feedback.

Learners start to gain some experience and can recognize recurring situational elements. They still rely on rules but begin to apply them to specific situations. A medical student who can perform basic procedures but still needs supervision and support is an example of an advanced beginner.

Training programs for these learners should include practice with real-world scenarios and guided experiences to help them solidify their understanding.

At this stage, learners can plan and manage their work. They understand and apply rules within context and can handle more complex situations. They develop problem-solving skills and can prioritize tasks. An example of a competent employee is a teacher who can design and deliver lessons effectively, manage a classroom, and adapt to different student needs.

Essential parts of training programs for competent employees should be goal-setting, strategic planning, and more independent practice.

Proficient learners see situations holistically rather than in terms of individual aspects. They can intuitively understand the context and make decisions based on experience. They develop a deeper understanding and can apply it flexibly. An example of a proficient employee is a software developer who can quickly understand and debug complex code, design efficient algorithms, and mentor junior developers.

To further enhance these employees' proficiency, training programs should involve reflective practice, mentoring, and opportunities for complex problem-solving.

Experts operate intuitively and fluidly, often without relying on explicit rules. They have deep, tacit knowledge and can recognize patterns and nuances that others may miss. They innovate and drive best practices in their field. An example of an expert is a surgeon who performs complex surgeries with ease, develops new surgical techniques and trains other surgeons.

It may seem that experts have overgrown the need for training, but this is not so. They will benefit from continuous learning through innovation, teaching others, and engaging in cutting-edge practices and research.

The greatest strength of this model is a customized learning system. It discards the one-size-fits-all approach, suggesting a framework for developing personalized learning experiences that match the learner's current level of expertise. The Dreyfus model helps track skill development over time, making it easier to identify when learners are ready for more advanced challenges.

Following this model, educators and trainers can better design and deliver instruction that meets the needs of learners at different stages.

However, assessing an individual's skill level can be subjective and may vary based on the evaluator's perspective. Besides, providing the appropriate level of support and instruction at each stage can be resource-intensive for educators and organizations.

Let's take a software development training program as an example.

During training, provide introductory courses on programming languages, coding standards, and development tools. Use hands-on exercises with clear, step-by-step instructions.

After finishing the course, assess the ability to write simple code snippets and follow basic coding guidelines.

Now you can introduce small projects and code reviews in the training course. Encourage learners to debug simple errors and make minor modifications to existing code. Evaluate learners' ability to apply basic coding concepts to real-world scenarios and solve simple problems.

Assign larger projects that require planning, designing, and implementing software solutions. Include collaborative projects and code documentation.

Assess the ability to manage software development projects, write maintainable code, and collaborate effectively with team members.

While training proficient software developers, focus on advanced topics such as software architecture, design patterns, and performance optimization. Provide opportunities for peer mentoring and code refactoring exercises.

Evaluate learners' ability to design robust software systems, optimize code, and mentor junior developers.

Encourage participation in cutting-edge projects, open-source contributions, and innovation challenges. Provide opportunities for leadership roles in development teams.

Assess their ability to lead complex software projects, innovate in software development practices, and contribute to the broader community.

The Brinkerhoff Model, also known as the Success Case Method (SCM), is an approach to evaluating training programs developed by Robert O. Brinkerhoff. This model focuses on identifying and understanding the most and least successful cases within a training program to determine its effectiveness and improve future training initiatives. The Brinkerhoff Model is particularly valuable for its practical and action-oriented approach to evaluation, providing insights that can directly inform decision-making and training improvements.

1. Focusing and Planning

The goal of this stage is to define the scope and objectives of the evaluation and to plan the evaluation process. It is also essential to determine the critical success factors for the training program.

2. Creating an Impact Model

At this stage, the purpose is to develop a conceptual model that links the training program to desired outcomes and impacts.

First, one should identify the expected changes in behavior and performance resulting from the training.

The next stage is to determine the necessary conditions for these changes and develop a logic model or impact map to visualize these relationships.

3. Designing and Implementing the Evaluation

This stage is to collect data on the training program's success and failure cases. Begin with developing criteria for identifying successful and unsuccessful cases. Collect qualitative and quantitative data through surveys, interviews, observations, and performance records. Select representative samples for success and failure cases. As a result, you should have a dataset that includes detailed information on successful and unsuccessful training experiences.

4. Analyzing and Interpreting Data

The purpose of this stage is to understand the factors contributing to success and failure in the training program.

Analyze the data collected in the previous stage to identify patterns and insights. Compare successful and unsuccessful cases to determine key differentiators.

Identify the factors that enabled success and the barriers that led to failure.

5. Reporting and Communicating Results

The final stage is to share the findings with stakeholders and provide actionable recommendations.

Prepare a comprehensive report summarizing the evaluation findings, highlighting success stories and the conditions that facilitated success. Provide recommendations for improving the training program based on the evaluation insights.

The Brinkerhoff Model provides actionable insights that can directly inform improvements to training programs. It helps identify what works and what doesn't, offering a balanced view of the training's effectiveness.

However, the criteria for success and failure may be subjective and not fully and not fully reflect reality. Findings based on selected cases may not be fully generalizable to all participants.

For example, you are evaluating a customer service skills training program. Start with focusing and planning. This involves defining the evaluation's objective, such as assessing the program's effectiveness in improving customer interactions and satisfaction and outlining key questions, data collection methods, and a timeline. Key questions might focus on identifying critical success factors and specific behaviors the training should improve.

Next, create an impact model by linking training activities to desired outcomes like improved customer satisfaction, reduced complaints, and better communication skills. Necessary conditions for these outcomes include access to feedback data, managerial support, and resources such as scripts and FAQs.

In designing and implementing the evaluation, select a representative sample of customer service reps and use pre- and post-training surveys, interviews, and observations. Collect and analyze customer satisfaction data, looking for significant improvements in satisfaction ratings, positive customer feedback, and better communication and problem-solving skills.

Analyze and interpret the data by comparing satisfaction scores and feedback before and after training, identifying patterns and differences between successful and unsuccessful cases. Determine effective aspects of the training and areas needing improvement.

Finally, report and communicate results by summarizing key findings, highlighting successful cases, and providing actionable recommendations. Share the report with stakeholders to inform future training initiatives.

This exploration of six training evaluation models has hopefully provided you with a comprehensive toolkit for assessing the effectiveness of your training initiatives. Each model offers a unique perspective and excels in specific areas.

The Kirkpatrick Model and the CIPP Model provide a structured approach for evaluating training across multiple levels. The Phillips ROI Model adds a crucial financial dimension, while the DACUM Model ensures training programs are aligned with job requirements. The Dreyfus Model personalizes training delivery based on learner skill levels, and the Brinkerhoff Model offers practical guidance for continuous improvement.

Ultimately, the best training evaluation model depends on your specific needs and objectives. Consider the goals of your training program, the resources available, and the level of detail you require when choosing.

By implementing a well-chosen evaluation strategy, you can gain valuable insights into your training programs' effectiveness, identify areas for improvement, and ultimately maximize the return on your training investment. So, take the next step, choose the model that best suits your needs, and start evaluating your training programs for a more skilled and successful workforce.