Diversity Training Evaluation: How Does Your Initiative Measure Up?

Left without the effective practice of utilizing consistent, time-tested methods of training evaluation for diversity initiatives and processes, diversity can become an easy target for those who do not believe in its value. This article summarizes a diversity training measurement strategy that focuses on return on investment.

There seems to be a myth operating within some communities of practice, which suggest that the outcomes or results created by a diversity implementation process defy measurement or can only be measured in the long-term. In a sense, it’s presented with the fundamental belief that creating an effective diverse, inclusive work environment is something of a complex and mysterious art form. Allegedly, the real value of diversity work can only be judged by those who perform it, those who are truly committed to its purpose or value it as important, etc. Because of poor diversity evaluation practices one article went so far as to say the ‘Diversity’s Business Case Doesn’t Add Up”[i].

When asked, even some diversity practitioners perceive that there is an inherent conflict between what is good for business and what is good for people. Some others believe, like truth, that the real reward is in the work itself. The words often used to describe diversity results include terms such as working better, appreciating differences, understanding each other better, less conflict, getting along, working as a team, and other similar non-measurement specific words. While these are admirable aims in themselves they are not enough. Especially when organizations are looking for strategies to deal with increased competition, options for reducing cost, adding value, adding dollars and increasing productivity to affect the bottom-line.

Many diversity training evaluations use poorly designed ‘smile sheets” and attendance records that have very little to do with the needs assessment, cultural audit, business requirements, or specific course objectives and behavioral outcomes.

So, how does your diversity training efforts measure up? What are you doing to show that the diversity training you conduct adds value to the organization and its bottom-line in real measurable and evaluative terms?

Accountability is a key issue for diversity training as well as any other business unit. Diversity training is only one of several initiatives that are undertaken to achieve an organization’s diversity objectives. Left without the effective practice of utilizing consistent, time-tested methods of training evaluation for diversity initiatives and processes, diversity can become an easy target for those who do not believe in its value. Consequently, the idea of being able to calculate the diversity return on investment (DROItm) of diversity training is mandatory if diversity is to take its place at the strategic partnership table.

Adding to the Alphabet

If the language of business is dollars, then the alphabet is numbers. Business has always been a numbers and bottom-line results game. With the advent of computers, executives have access to data like never before. Human Resources-based initiatives like diversity must report their contribution and performance like any other organizational entity. The lack of measurement practices for diversity sets diversity apart from the rest of the organization. Therefore if we, as diversity professionals, want to be effective communicators of value added to the bottom-line, we must build rapport with our audience. The business case and rationale for diversity must be linked to strategic business objectives and initiative results must be displayed and communicated in financial terms.

Why aren’t practitioners doing this now? One study seems to have a few answers. In one study, Margaret Blackburn White, asked a group of diversity professionals ‘if their organizations are demanding evidence that their investments in diversity are paying off on the bottom-line”. The answers received suggested that the response landscape is varied. There are few demands for diversity to ‘justify its existence” by documenting a direct return-on-investment link.

However, there was agreement that there is a lot of general conversation about the diversity return-on investment topic.[ii]

The only way to determine that diversity training and skills development are having the desired effect is to use formal training evaluation processes and cost benefit analysis methods.

As competitive pressures, globalization, and other issues become more prevalent, measuring diversity results will become a critical requirement for the future. Customers, shareholders and employees will no longer accept ‘business as usual”. With the increased availability and use of diversity metrics, diversity scorecards and the like, organizations will begin to see diversity training evaluation as a requirement, not as ‘nice to have”.

The only way to determine that diversity training and skills development are having the desired effect is to use formal training evaluation processes and cost-benefit analysis methods. The results of these activities can confirm the positive effects of training and development and identify improvements to make it better. Evaluation can contribute to maximizing the organization’s return on training investment.

Evaluation means to measure something in preparation for making a decision: for example, to stop, modify, or expand it to increase its benefits. This implies:

• Knowing what decision the evaluation data will help you make.
• Measuring scientifically, using data collection methods and research designs that separate the effects your program is having from all other influences on your outcome variables.
• Choosing the right measures for what your program is really trying to accomplish.

It is useful first to distinguish result measures from other types of measures commonly used to evaluate human resource-based programs like diversity training. In a famous article, Kirkpatrick observed that training and other programs could be evaluated at one of four levels: reaction, learning, behavior, and results. Building on this model, Jack Phillips added a fifth level: return on investment.[iii] A summary of the model levels is shown on the previous page.

The amount of evaluation that you provide depends on the types of decisions that your organization must make and the information needed to make those decisions.

Caution! The four levels of evaluation should be used in sequence. It is acceptable to use Levels 1 and 2 by themselves. Levels 3 and 4 should not normally be used unless you have positive results from a Level 2 evaluation. Without a Level 2 evaluation, it is difficult to relate the results of Level 3 evaluation back to training. This is because factors other than training can influence on-the-job transfer.

If you do a Level 3 without a Level 2, you need to rule out these other factors before you can assume that training had positive or negative outcomes. With Level 4 evaluation, it is important to have some evidence of training results at both the learning and transfer levels. Once this is accomplished, you can begin to explore the cost-benefit, return-on-investment relationship that ties the diversity training effort to the bottom-line.[iv] [v]

How Much Training Evaluation Do You Need?

Anyone responsible for diversity training is also responsible for evaluation. The amount of evaluation that you provide depends on the types of decisions that your organization must make and the information needed to make those decisions. For example, if your only requirement is to ensure that participants have positive attitudes toward the course, then Level 1 evaluation is sufficient. But, if your goal is to determine whether your diversity course is having a positive effect on job performance, then you will have to do Level 3 evaluation, and this means also doing Level 1 and 2. They provide the basis for determining whether participants want to use what they have learned and have indeed learned the appropriate attitudes and skills.

Where Do You Begin?

Your first step in evaluating diversity training is to determine your major evaluation questions or objectives based upon a needs assessment or cultural audit conducted at the outset. The second step is to plan and conduct the appropriate level of diversity training evaluation. The decision table on the next page may help you identify some of the major questions and methods.

If full utilization of a diverse workforce and diversity is to be a reality in our lifetime, we must use every tool or resource available to fully monitor and communicate the effectiveness of this effort. When we start to show management exactly how much value diversity training programs can contribute to the process of building an inclusive work environment, then diversity-training initiatives will become a strategic requirement.

A true commitment to diversity takes effort, time and money, none of which can afford to be wasted in a competitive marketplace. Utilizing diversity in pursuit of organizational objectives can generate measurable dollars and sense as well as bottom-line performance.

Dr. Edward E. Hubbard is president and CEO of Hubbard & Hubbard, Inc., and author of the books Measuring Diversity Results, How to Calculate Diversity Return on Investment, and The Diversity Scorecard.

End Notes