Published May 2004
Adequate methods to prove the success and business impact of corporate learning programs are on every CLO’s mind today. Based on research and on conversations I have had with CLOs, one insight that has emerged is that metrics and measurement programs are more successful when they come from a deeper business need or reflect the kind of business thinking that already pervades the culture. Measurement or impact programs that are imposed artificially from the outside, but that are not expressions of the way in which the organization already works, may either fail or end up measuring the wrong kinds of things.
For example, Kevin Wilde, chief learning officer for General Mills, notes that when he set out to find the right ways to measure the impact of learning on the performance of General Mills, he drew on what was distinctive about the company as a consumer products company. Early on, said Wilde, “I realized that, to build the credibility and relationships with senior executives in the company, I needed to speak their language. Most of them grew up in a marketing world where they relied heavily on research. They are constantly asking, ‘What are the numbers telling us about what consumers want?’ ”
Realizing that type of data is a key lever in the company, Wilde set out to determine the best data to show that learning programs were having an impact on employee commitment and performance through excellent development programs. The employee survey turned out to be a particularly effective tool. “I have leveraged the heck out of these surveys because that’s the language our leaders appreciate. It helps to show the numbers. So when we doubled the size of the company two years ago with the Pillsbury merger, I used employee survey data to help us understand some of the people issues on both sides. We then provided the right response to smooth many of the merger speed bumps we found. Subsequent surveys indicated solid progress,” he said.
The main point, said Wilde, is that learning and development professionals need to link their work with the business strategy first, then provide differentiating value and periodically show the impact with the right long-term metrics that speak most powerfully to decision makers within the organization.
At Wachovia, the metrics work being done by CLO Ron Garrow and his team was driven by a real business need: the merger between Wachovia and First Union in 2001. Wachovia CEO G. Kennedy Thompson set an extremely high goal for the new organization following the merger: zero customer impact. The training that was needed to ensure that Wachovia employees could perform in a new environment with no service repercussions from the merger was substantial. It involved about 1.6 million hours of training so that the workforce could learn new products, processes and systems and ultimately, so the company could ensure that the entire organization exceeded customers’ expectations for servicing through the entire merger integration period.
In order to meet the goal of zero customer impact, said Garrow, he and his team worked with line-of-business training teams to develop and administer a number of “readiness assessments” before each region or line of business did any type of conversion based on the new processes and systems for the new combined company. “That readiness involved not only Level 1 evaluation—evaluation of the participant and employee satisfaction with the training— but also measurements of their comprehension and retention and even of how well they were able to apply that knowledge. So the business circumstances we were dealing with forced us rather quickly into some solid Level 2 and 3 evaluation data.”
The measurement program put in place to meet the immediate business need of the merger then became the basis for what Wachovia called its “Assessment, Measurement and Evaluation” or AME philosophy. Garrow and his staff have identified AME champions within each of the learning teams that are part of the learning organization. The AME champions are in the process of going through an ROI certification, part of which involves a full-blown return on investment assessment. Wachovia is already seeing some benefits and demonstrated results from this initiative.
For example, for a new executive leadership program the company is developing in conjunction with the Kenan-Flagler School of Business at UNC-Chapel Hill, Garrow and staff assigned executive coaches to each of the participants. By interviewing these coaches and the subordinates of the program graduates, Wachovia intends to collect Level 2, 3 and even Level 4 data about how the program has affected the performance of these leaders and the company as a whole.
Garrow said, “All in all, this data is really going to help show us exactly what these graduates are doing differently as a result of that educational experience.”
These stories underscore the importance of going beyond the “right” external measurement methodology. What’s needed is an approach that operates out of a deep understanding of the business levers of your company and using relevant business events to help drive an approach to measuring the business impact of learning that resonates with your executive leadership team.