Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section A reads “Knowledge of models and methods to evaluate the impact of learning and talent development solutions.”
There are numerous ways to measure the impact of training. The purpose of this article is to share some effective ways to measure impact and some caveats on their use.
As examples, I will use my leadership development work and give specific ideas that show how they work.
ROI Calculations
Depending on the type of training and the level of people being trained, it may be possible to measure the ROI directly by getting pre’ and post-training data from existing sources. These elements might include direct productivity of the unit, turnover rates, grievances, Quality of Worklife surveys, and other extant data that generally reside with Human Resources.
The obvious caveat here is to make sure the shift in performance was a direct result of the training and not some other influence that was going on at the same time.
Course Evaluations
One universally used tool is the end of course evaluation. Simply carve out time at the last session of the course for people to fill out a brief survey. The following is a list of the few questions asked on most end-of-course evaluations:
1. The impact of this course on your leadership skill
2. The skill of the facilitator
3. The pace of the course and use of time
4. Rate the effectiveness of the experiential learning components
5. Degree to which you would recommend this course to others
There are two caveats I can think of with evaluations. First, people may feel tired or rushed if they are put on the spot at the end of an exhausting course. If you elect to send an electronic version of the evaluation, your net return of the surveys will be a significantly lower percentage of the population, and the data will be skewed.
People who had a good experience will take the time to fill out the survey, but people who did not like the course (those with the most significantly helpful data) will often not bother to submit.
Second, the anonymity of the feedback is always in question. If there is a hard copy form, then people might believe there is a secret mark somewhere so that their input is not really anonymous.
Stories
Sometimes you will hear stories about the impact of training. Keep track of these and get as much input as you can. The following story happened to me as a result of some leadership training I did in a city about 200 miles from my home town.
I was called in to meet with the CEO and HR Manager of a metal working firm in another city. When I drove into the parking lot, I noticed that it was only about one third full.
The CEO and HR Manager told me that their business was faltering due to internal squabbles between the various groups and lack of customer focus. They had furloughed many workers and were working partial shifts to get by. I determined that there was a lot of conflict due to low trust, and they lacked focus and alignment with a solid strategic plan.
I worked with the leadership team for a couple days giving them information on how to build a better culture of Trust and developing a strong strategic plan.
Six months later, the CEO called me back down there because they had “a different problem.” When I rounded the corner to the parking lot, I immediately saw that there were no slots available in the entire lot. The CEO wanted to know how they could ramp up faster with staff because they were buried in too many orders for the current team.
Testimonials
Many times, people will volunteer to write a short paragraph on their reaction to the training. These reviews are like gold, because they capture the enthusiasm in the actual words selected by the writer.
I have been collecting testimonials for years, and they are useful. If you use too many of them in your promotions it can have a negative impact.
I will share two testimonials that reflect actual student reactions to my teaching.
“Bob Whipple has been a force of nature in our community when it comes to trust-building and leadership development. He volunteers locally on business ethics boards and committees, writes books and articles, produces videos, and tirelessly trains our local up-and-coming leaders through the local Chamber of Commerce. His thinking is original and powerful. His lifetime of achievements have shown his deep competence in the leadership and strategy domains coupled with his authentic caring for his community and the people he trains.”
“You have an incredible way of teaching others and motivating them to excel. I have learned more from your insight and examples than any book, course, or seminar I have taken thus far. It is so refreshing to meet a leader who not only has the didactic knowledge and requisite experience but also is an expert at its practical application! Thank you for helping me see my potential and for an experience I will never forget.”
Keep in mind that testimonials also can include suggestions for how you can improve your product, so read them carefully and let the writers be your coaches.
Repeat Business
One way to measure the worth of a talent development effort is whether there is a continuum of repeat business. If individuals and groups are seeing real value and progress from your training, they will encourage others to take the course. If a course dies out after one or two cohorts, you need to think about what you are doing wrong.
These examples are just a few ways to measure the impact of a talent development solution. Get creative and see what other methods you find helpful.
Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.