Talent Development 37 Evaluating Impact

May 10, 2021

Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section A reads “Knowledge of models and methods to evaluate the impact of learning and talent development solutions.”

There are numerous ways to measure the impact of training. The purpose of this article is to share some effective ways to measure impact and some caveats on their use.

As examples, I will use my leadership development work and give specific ideas that show how they work.

ROI Calculations

Depending on the type of training and the level of people being trained, it may be possible to measure the ROI directly by getting pre’ and post-training data from existing sources. These elements might include direct productivity of the unit, turnover rates, grievances, Quality of Worklife surveys, and other extant data that generally reside with Human Resources.

The obvious caveat here is to make sure the shift in performance was a direct result of the training and not some other influence that was going on at the same time.

Course Evaluations

One universally used tool is the end of course evaluation. Simply carve out time at the last session of the course for people to fill out a brief survey. The following is a list of the few questions asked on most end-of-course evaluations:

1. The impact of this course on your leadership skill
2. The skill of the facilitator
3. The pace of the course and use of time
4. Rate the effectiveness of the experiential learning components
5. Degree to which you would recommend this course to others

There are two caveats I can think of with evaluations. First, people may feel tired or rushed if they are put on the spot at the end of an exhausting course. If you elect to send an electronic version of the evaluation, your net return of the surveys will be a significantly lower percentage of the population, and the data will be skewed.

People who had a good experience will take the time to fill out the survey, but people who did not like the course (those with the most significantly helpful data) will often not bother to submit.

Second, the anonymity of the feedback is always in question. If there is a hard copy form, then people might believe there is a secret mark somewhere so that their input is not really anonymous.

Stories

Sometimes you will hear stories about the impact of training. Keep track of these and get as much input as you can. The following story happened to me as a result of some leadership training I did in a city about 200 miles from my home town.

I was called in to meet with the CEO and HR Manager of a metal working firm in another city. When I drove into the parking lot, I noticed that it was only about one third full.

The CEO and HR Manager told me that their business was faltering due to internal squabbles between the various groups and lack of customer focus. They had furloughed many workers and were working partial shifts to get by. I determined that there was a lot of conflict due to low trust, and they lacked focus and alignment with a solid strategic plan.

I worked with the leadership team for a couple days giving them information on how to build a better culture of Trust and developing a strong strategic plan.

Six months later, the CEO called me back down there because they had “a different problem.” When I rounded the corner to the parking lot, I immediately saw that there were no slots available in the entire lot. The CEO wanted to know how they could ramp up faster with staff because they were buried in too many orders for the current team.

Testimonials

Many times, people will volunteer to write a short paragraph on their reaction to the training. These reviews are like gold, because they capture the enthusiasm in the actual words selected by the writer.

I have been collecting testimonials for years, and they are useful. If you use too many of them in your promotions it can have a negative impact.

I will share two testimonials that reflect actual student reactions to my teaching.

“Bob Whipple has been a force of nature in our community when it comes to trust-building and leadership development. He volunteers locally on business ethics boards and committees, writes books and articles, produces videos, and tirelessly trains our local up-and-coming leaders through the local Chamber of Commerce. His thinking is original and powerful. His lifetime of achievements have shown his deep competence in the leadership and strategy domains coupled with his authentic caring for his community and the people he trains.”

“You have an incredible way of teaching others and motivating them to excel. I have learned more from your insight and examples than any book, course, or seminar I have taken thus far. It is so refreshing to meet a leader who not only has the didactic knowledge and requisite experience but also is an expert at its practical application! Thank you for helping me see my potential and for an experience I will never forget.”

Keep in mind that testimonials also can include suggestions for how you can improve your product, so read them carefully and let the writers be your coaches.

Repeat Business

One way to measure the worth of a talent development effort is whether there is a continuum of repeat business. If individuals and groups are seeing real value and progress from your training, they will encourage others to take the course. If a course dies out after one or two cohorts, you need to think about what you are doing wrong.

These examples are just a few ways to measure the impact of a talent development solution. Get creative and see what other methods you find helpful.

Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.

Leave a Comment » | Talent Development | Tagged: , | Permalink
Posted by trustambassador

Talent Development 16 Surveys That Work

November 8, 2020

Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section B reads, “Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.”

For about 5 years, I taught a graduate course called Experimental Design. The course was part of a curriculum leading to a Masters Degree in Organization Learning and Human Resources Development.

In preparation for writing this brief article, I went back and reviewed my slide deck for the course. It was 200 slides long, and many of the slides were heavy with statistical techniques. Obviously, I will need to skim the surface in this short summary article.

In the course, we studied how well-intended surveys often miss the mark and produce bogus results. We also studied the antidotes.

Why Surveys Fail

There are many reasons why surveys fail. I will list some of the more common reasons here and then describe the typical antidotes.

1. Survey too long and complex – one reason for poor data is because people get turned off by too burdensome and too many surveys. When people are angry about too many surveys, they give responses that do not reflect their true feelings.

2. Changes made are not evident – if leaders do not stress that a change is being made because of an employee survey, people will believe their input was ignored. The common misconception sounds like this, “They make us fill out all these ridiculous surveys, but nothing ever changes.”

3. Survey not valid – incorrectly designed surveys often do not measure the thing they are trying to measure. Surveys must be statistically validated to be useful.

4. Survey not reliable – If you would repeat the survey a second time, you would get a different result?

5. Questionable anonymity – If people believe there is some secret way management can find out who said what, then the instrument will not give accurate results. People will respond in ways they think management wants to hear.

6. No clear objective – When people are asked to fill out a survey, they need to know ahead of time why they are being asked to participate and what to expect.

7. Questions not clear – Often the wording of questions leaves people wondering what is really meant by the questions. In this case, you will get guesswork rather than valid data.

8. Leading Questions – Sometimes the way questions are worded leads to skewed data. For example, a political survey might ask, “Are you frustrated by the lies being spread by my opponent?”

Antidotes

I will list the antidotes to the problems in the same order.

1. Make sure your survey is user friendly. Take the survey yourself and ask if you would take the time to do this instrument justice on your most busy day. A good rule of thumb is to be able to fill out the information in less than 10 minutes.

2. Make sure you get back to everyone who responded with the results of the survey. Also, tie all changes made to the survey results, so people are aware of the connection.

3. Test if the survey is valid. The only way you can tell is a survey is measuring what you are trying to measure is to use a statistical analysis of the data. There are five different types of validity (Content, Construct, Concurrent, Criterion, and Predictive). Get help from someone qualified to measure validity. Don’t just wing it.

4. Test if the survey is reliable. This involves trial runs of the survey with different groups under different conditions. The survey needs to produce consistent information to be reliable. Another method is to use a split-half technique. Again, get help if you are not an expert in this area.

5. Insuring anonymity is tricky – The best method I have found to get people to really believe the survey is anonymous is to select a skeptical person from the population to help reduce the data into usable form. The skeptic will let others know that there was no secret means by which management knew who said what.

6. Clarity of Objective – This is a matter of good survey design. It is not just a simple matter of generating some questions and handing out the survey. It must be done with care and solid logic. The way the survey is introduced (typically with an email or letter) is critical. Otherwise you have garbage in garbage out.

7. Test your questions for understandability – This is usually done in the final design phase. You ask people how they interpreted the question. It is not uncommon for many people to be baffled by the wording. Check it out carefully.

8. Avoid leading questions – do not telegraph the requested answer by the way a question is worded. Like don’t ask “Would you like a yummy pizza?”

These areas are general, but they do show how generating a survey is not so simple as most people believe. If a survey is going to generate valuable information for the organization, it needs to be constructed well and administered correctly.

Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.

1 Comment | Candor creates trust, Communications, Enabling Actions, HR, Interviews, Organization, Transparency, Trust, Trust and breach of trust | Tagged: , , , , , , , , , , , , , , , , , , , , , , , , , , , | Permalink
Posted by trustambassador

Body Language 83 Handshakes Post COVID-19

April 13, 2020

I am having to modify my leadership training material as a result of COVID-19. I do a section on the impact of Body Language on trust between people.

Historically, I have discussed the handshake at length because how you do it impacts the first impression people have about you, which has a huge impact on the trust you can achieve with the other person.

We may get back to shaking hands post COVID-19, but it will likely be quite a while before people are comfortable doing it.

Every culture has some form of touch ritual for people when they first meet. I suspect they will all be impacted by the pandemic we have experienced in 2020.

In western cultures, and several others, the handshake is the preferred method of greeting a person you are just meeting. What are the options, and how will they impact the ability to bond with the other person?

Fist bumps

The fist bump is assumed to be far less contaminating than a full handshake for two reasons. First, the contact area is much less, and second, the duration of the contact is far less. Still, if I am going to be uncomfortable with a full hand shake, I am also going to be a bit leery of a fist bump for quite some time.

Elbow bumps

Having the elbows touch is suggested as an alternative, but it is a really poor one because it is difficult to maintain eye contact when doing it, and the intimacy is destroyed by the awkward position required to do it. When watching two people try to do an elbow bump, I usually see it followed by an awkward kind of laugh as if the whole thing is some kind of joke. This could become less of an issue in the future, but I really doubt it.


Thumbs up

Here you can maintain a good distance from the other person. It is a positive and friendly gesture that sends a good signal. There is no touching at all, so the possibility of contamination is greatly reduced. Unfortunately, the intimacy of the handshake is lost with a thumbs up.

Wave

A cheerful wave may be as good as a thumbs up gesture. Here you can combine a facial expression of gratitude for being able to meet the other person. That is the most important ingredient that made the handshake so valuable in the past.

We have to modify our habitual touch ritual that we learned as children and have been using all our life, up to this point. That’s too bad, because the handshake was a powerful way to show your eagerness to meet the other person. In my programs, I stress that it is possible to plant a seed of trust in the first 10 seconds, and a large part of doing that was a proper handshake.

The substitute greeting gestures are never going to replace the value of a handshake as a way to have two people bond when first meeting. That is an unfortunate reality, which means we will need to work extra hard to demonstrate our emotions without touching in the future, at least for a while.

Pay attention to how you greet new acquaintances in the future and select a method that you feel conveys the right spirit and that you can apply consistently. We may return to the handshake someday in the future based on some kind of immunization program, but I believe the scars left by this huge disruption of COVID-19 will have a long memory in the minds of most people.



This is a part in a series of articles on “Body Language” by Bob Whipple “The Trust Ambassador.”


Leave a Comment » | Candor creates trust, Communications, Enabling Actions, HR, leadership, Transparency, Trust, Trust and breach of trust, Trust and fear | Tagged: , , , , , , , , , , , , , , , , , , | Permalink
Posted by trustambassador

The Leader’s Role in Building Trust

October 31, 2015

CEOOver a period of several decades, I have observed how the trust level in any organization is influenced the most by one single factor. The behaviors of the senior leaders in any organization will have more impact on trust than anything else.

Therefore, if the trust in an organization is not as high as needed, the senior leaders need to take a good long look in the mirror. It is the behaviors of the senior leaders that are almost always the root cause of a trust problem in an organization.

Please do not misunderstand, there will be trust issues evident at all levels of the organization, and often severe untrustworthy behaviors exist at the operational level.

The cold reality is that in most organizations nearly all employees will perform in a trustworthy manner if they are properly led.

Many leaders reject their culpability indicating that it is the workers who are not being trustworthy that account for low trust. Upon closer examination, I find that it is almost always the behavior of the senior leaders that causes employees at various levels to act in a non-trustworthy manner.

The culture of any organization is established from the top. Certainly there are many levels in any organization, and there can be trust issues at any level, but the tone of the environment is created by the behaviors and policies set out by the most senior leader.

Trying to get leaders to step up to this responsibility is one of the most difficult challenges I face in my consulting business. They would much rather blame others, or circumstances, or customers, or the economy, or anything other than themselves as being the cause of the difficulties they face.

Exercise for leaders: Today, ask yourself what behaviors you would need to change in order to begin a new culture within your organization. Think about your role as a leader in establishing the environment in which all employees work. That environment is the creator of either excellence or difficulties in trust.

I rarely meet an executive who will say, “there is a lack of trust in the organization, and since I am the leader here, it must be originating with me.” Occasionally I will run into someone who thinks that way, but it is pretty rare.

The more we can convince leaders of their responsibility in terms of creating the right culture, the more trust we can create in the world.

Here are four “foundational behaviors” leaders can exhibit that will move the culture to one of higher trust along with my favorite quote on each one:
1. Reinforce Candor – make people unafraid to bring up issues. “The absence of fear is the incubator of trust.”
2. Hold people accountable in a balanced way, not just when they have messed up. “Hold people ‘procountable’ rather than accountable.”
3. Extend more trust in the people within the organization. “The First Law of trust: If you want to see more trust, then extend more trust.”
4. Have firm values and demonstrate those values every single day. “Stated values that are not demonstrated by leaders act like nuclear missiles to the fragile trust ecosystem.”

Once leaders can do these four things consistently, then there are hundreds of other behaviors that will take root and begin to accelerate the pace of building trust. I will mention just a few of the behaviors here for the sake of brevity:

1. Do what you say
2. Treat people well
3. Tell the truth
4. Demonstrate respect
5. Be transparent
6. Use Golden Rule
7. Stick up for people
8. Be ethical
9. Admit mistakes
10. Care for other person
11. Adhere to values
12. Listen well
13. Reinforce good behavior
14. Practice humility
15. Be consistent
16. Right wrongs

If you are a leader, step up to your role as the primary force that is creating the culture in your organization. If there are problems, then it is up to you to change the culture to eliminate them.

If you are not the leader, print out this article and put it on the desk of that person. It may have an impact.

The preceding was derived from an episode in “Building Trust,” a 30 part video series by Bob Whipple “The Trust Ambassador.” To view three short (3 minutes each) examples at no cost go to http://www.avanoo.com/first3/517

4 Comments | Candor creates trust, Communications, Enabling Actions, leadership, Trust, Trust and fear, Trust and motivation, Trust and respect | Tagged: , , , , | Permalink
Posted by trustambassador

How Trust Helps Solve Problems

April 18, 2015

Leadership SolutionsThe level of trust in a group has a profound impact on the ease with which they solve problems.

I sit on several Boards of Directors, and one of them is a pretty low trust group. When a problem comes up, it seems the team is always tiptoeing around the interpersonal issues.

Low trust groups often fail to solve the real problem and frequently have to deal with a lot of acrimony, often unrelated to the problem.

This low trust group can discuss things for an hour and not even get close to the real problem at hand. We quite often end up putting “BandAids” on the symptoms hoping the problem will resolve itself. We all know the world does not work that way.

It is very frustrating because we waste a lot of time and energy with low output.

Another BOD I sit on is a particularly high trust group. They solve problems quickly and efficiently because they get to the heart of the issue fast without people playing games with each other. One hallmark of high trust groups is that they solve problems quickly and with high quality solutions while having fun.

The quality of solutions is higher because people are not afraid to voice creative ideas. They don’t need to protect themselves from ridicule. Brainstorming possible actions is spontaneous, light, and often comical.

It is important to assess the level of trust on every team. There are numerous surveys available online if you just do a quick search. As an alternative, I have developed a quick survey that can be very helpful at understanding the level of trust on your team. It is available at the following link

https://www.surveymonkey.com/r/ZZGQVD3 .

Take the time today to do an assessment of the trust level on your team. This is especially important if your team seems to struggle at times. Make sure all members of the team take the instrument and share the data.

If trust is lacking, then get a commitment to do something about it. Here is a link to several articles about trust on my Leadergrow Website

http://www.leadergrow.com/articles/categories/17-trust

Putting up with interpersonal issues that result from low trust is a sign of mediocrity. You can move to excellence simply by investing some time and energy into raising the trust level. It is not impossible, and your team will become much more efficient.

4 Comments | Candor creates trust, Communications, Enabling Actions, HR, leadership, Organization, Teamwork, Trust, Trust and fear, Trust and motivation | Tagged: , , , , | Permalink
Posted by trustambassador

Anti-Stupid Pill for Leaders

April 17, 2011

One of my leadership students laments that some of the decisions the leaders in his organization make relative to policies and size of workforce are just plain stupid. These decisions reflect a misunderstanding of their impact, so the leaders end up doing things that are at cross purposes to their true desires.

I told the student to buy some “anti-stupid” pills for the leaders to take, which will let them know when they do things that take them in the wrong direction. Then I realized that I already had discovered the “anti-stupid” pill several years ago and have taught leaders how to administer this magic potion for quite a while.

Leaders need a way to determine the impact of their decisions on the organization at the time of making those decisions. This knowledge will reduce the number of wrong-headed actions. Picture a leader of 84 individuals. There are exactly 84 people who are capable of telling her the truth about the impact of poor decisions before she makes them. They would gladly do this if the leader had established an environment where it is safe to challenge an idea generated in the mind of the leader. How would a leader go about creating such an environment?

If a leader makes people glad when they tell her things she was really not eager to hear, those people will eventually learn it is safe to do it. They have the freedom to level with the leader when she is contemplating something really dumb. It does not mean that all dumb things the leader wants to do need to be squashed. It simply means that if the leader establishes a safe culture, she will be tipped off in advance that a specific decision might backfire. Sometimes, due to a leader’s perspective, what may seem dumb to underlings may, in fact, be the right thing to do. In this case the leader needs to educate the doubting underling on why the decision really does make sense. Here is an eight-step formula that constitutes an anti-stupid pill.

1. As much as possible, let people know in advance the decisions you are contemplating, and state your likely action.

2. Invite dialog, either public or private. People should feel free to express their opinions about the outcomes.

3. Treat people like adults, and listen to them carefully when they express concerns.

4. Factor their thoughts into your final decision process. This does not mean to always reverse your decision, but do consciously consider the input.

5. Make your final decision about the issue and announce it.

6. State that there were several opinions that were considered when making your decision.

7. Thank people for sharing their thoughts in a mature way.

8. Ask for everyone’s help to implement your decision whether or not they fully agree with the course of action.

Of course, it is important for people to share their concerns with the leader in a proper way at the proper time. Calling her a jerk in a staff meeting would not qualify as helpful information and would normally be a problem. The leader not only needs to encourage people to speak up but give coaching as to how and when to do it effectively. Often this means encouraging people to give their concern in private and with helpful intent for the organization rather than an effort to embarrass the boss.

The leader will still make some stupid decisions, but they will be fewer, and be made recognizing the risks. Also, realize that history may reveal some decisions thought to be stupid at the time to be actually brilliant. Understanding the risks allows some mitigating actions to remove much of the sting of making risky decisions. The action here is incumbent on the leader. It is critical to have a response pattern that praises and reinforces people when they speak the truth, even if it flies in the face of what the leader wants to do. People then become emboldened and more willing to confront the leader when her judgment seems wrong.

A leader needs to be consistent with this philosophy, although no one can be 100%. That would be impossible. Once in a while, any leader will push back on some unwanted “reality” statements, especially if they are accusatory or given in the wrong forum. Most leaders are capable of making people who challenge them happy about it only a tiny fraction of the time, let’s say 5%. If we increase the odds to something like 80%, people will be more comfortable pointing out a potential blooper. That is enough momentum to change the culture.

It is important to recognize that making people glad they brought up a concern does not always mean a leader must acquiesce. All that is required is for the leader to treat the individual as someone with important information, listen to the person carefully, consider the veracity of the input, and honestly take the concern into account in deciding what to do. In many situations, the leader will elect to go ahead with the original action, but she will now understand the potential ramifications better. By sincerely thanking the person who pointed out the possible pitfall, the leader makes that individual happy she brought it up. Other people will take the risk in the future. That changes everything, and the leader now has an effective “anti-stupid” pill.

5 Comments | Candor creates trust, Communications, Enabling Actions, leadership, Organization, Transparency, Trust, Trust and motivation | Tagged: , , , , , , , | Permalink
Posted by trustambassador