Talent Development 16 Surveys That Work

November 8, 2020

Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section B reads, “Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.”

For about 5 years, I taught a graduate course called Experimental Design. The course was part of a curriculum leading to a Masters Degree in Organization Learning and Human Resources Development.

In preparation for writing this brief article, I went back and reviewed my slide deck for the course. It was 200 slides long, and many of the slides were heavy with statistical techniques. Obviously, I will need to skim the surface in this short summary article.

In the course, we studied how well-intended surveys often miss the mark and produce bogus results. We also studied the antidotes.

Why Surveys Fail

There are many reasons why surveys fail. I will list some of the more common reasons here and then describe the typical antidotes.

1. Survey too long and complex – one reason for poor data is because people get turned off by too burdensome and too many surveys. When people are angry about too many surveys, they give responses that do not reflect their true feelings.

2. Changes made are not evident – if leaders do not stress that a change is being made because of an employee survey, people will believe their input was ignored. The common misconception sounds like this, “They make us fill out all these ridiculous surveys, but nothing ever changes.”

3. Survey not valid – incorrectly designed surveys often do not measure the thing they are trying to measure. Surveys must be statistically validated to be useful.

4. Survey not reliable – If you would repeat the survey a second time, you would get a different result?

5. Questionable anonymity – If people believe there is some secret way management can find out who said what, then the instrument will not give accurate results. People will respond in ways they think management wants to hear.

6. No clear objective – When people are asked to fill out a survey, they need to know ahead of time why they are being asked to participate and what to expect.

7. Questions not clear – Often the wording of questions leaves people wondering what is really meant by the questions. In this case, you will get guesswork rather than valid data.

8. Leading Questions – Sometimes the way questions are worded leads to skewed data. For example, a political survey might ask, “Are you frustrated by the lies being spread by my opponent?”

Antidotes

I will list the antidotes to the problems in the same order.

1. Make sure your survey is user friendly. Take the survey yourself and ask if you would take the time to do this instrument justice on your most busy day. A good rule of thumb is to be able to fill out the information in less than 10 minutes.

2. Make sure you get back to everyone who responded with the results of the survey. Also, tie all changes made to the survey results, so people are aware of the connection.

3. Test if the survey is valid. The only way you can tell is a survey is measuring what you are trying to measure is to use a statistical analysis of the data. There are five different types of validity (Content, Construct, Concurrent, Criterion, and Predictive). Get help from someone qualified to measure validity. Don’t just wing it.

4. Test if the survey is reliable. This involves trial runs of the survey with different groups under different conditions. The survey needs to produce consistent information to be reliable. Another method is to use a split-half technique. Again, get help if you are not an expert in this area.

5. Insuring anonymity is tricky – The best method I have found to get people to really believe the survey is anonymous is to select a skeptical person from the population to help reduce the data into usable form. The skeptic will let others know that there was no secret means by which management knew who said what.

6. Clarity of Objective – This is a matter of good survey design. It is not just a simple matter of generating some questions and handing out the survey. It must be done with care and solid logic. The way the survey is introduced (typically with an email or letter) is critical. Otherwise you have garbage in garbage out.

7. Test your questions for understandability – This is usually done in the final design phase. You ask people how they interpreted the question. It is not uncommon for many people to be baffled by the wording. Check it out carefully.

8. Avoid leading questions – do not telegraph the requested answer by the way a question is worded. Like don’t ask “Would you like a yummy pizza?”

These areas are general, but they do show how generating a survey is not so simple as most people believe. If a survey is going to generate valuable information for the organization, it needs to be constructed well and administered correctly.

Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.

1 Comment | Candor creates trust, Communications, Enabling Actions, HR, Interviews, Organization, Transparency, Trust, Trust and breach of trust | Tagged: , , , , , , , , , , , , , , , , , , , , , , , , , , , | Permalink
Posted by trustambassador

Leadership Barometer 58 12 Rules for Success

July 13, 2020

Several years ago I generated a list of rules for success. It is important to write down a set of rules for yourself, just as it is to document your values.

Having a list of rules gives you something to hang on to when there is too much confusion. Another benefit of a list like this is that it helps other people know how you operate much quicker.

I would review this list and my passion for each item whenever inheriting a new group. People appreciated that I made a special effort for them get to know me in this way.

1. The most important word that determines your success is “attitude” – how you react to what happens in your life. The magic learning here is that you control your attitude, therefore, you can control your success.

2. Engagement of people is the only way to business success.

3. Credibility allows freedom to manage in an “appropriate” way (which means if you are not credible, you will be micro-managed).

4. Build a “real” environment – maximize trust – This requires honesty and transparency.

5. Create winners – help people realize their dreams of success (which means, grow other leaders).

6. Recognize and reward results at all levels (reinforcement governs performance).

7. Operate ahead of the power curve (which means, be organized and get things done well ahead of the deadline).

8. Don’t get mired down in bureaucratic mumbo jumbo, negotiate the best position possible, out flank the Sahara. However, feed the animal when necessary (which means pick your political battles carefully).

9. Enjoy the ride – when it is no longer fun – leave.

10. Admit when you are wrong and do it with great delight. Beg people to let you know when you sap them and thank them for it (which means Reinforce Candor).

11. Provide “real” reinforcement that is perceived as reinforcing by the receiver. Build an environment of reinforcement.

12. Keep trying and never give up. You will succeed.

There are many other things that could be mentioned, but if you can master the things above, most other things become subcategories of them.

For example, another bullet might be “Treat people as adults and always demonstrate respect.” That is really a sub item of the second bullet.

Or another bullet might be “Always walk your talk.” That is one thing (among many) you need to do for bullet four to happen.

I believe every leader should have a documented set of beliefs such as the one above. I am not advocating that you adopt my list. Think about it and develop your own list.

Don’t worry about being complete, just start an electronic file and add to it over the years as you grow and encounter new ideas. You will be amazed how this simple task enables you to operate with congruence and grow in your leadership skill.


The preceding information was adapted from the book Leading with Trust is like Sailing Downwind, by Robert Whipple. It is available on http://www.leadergrow.com.

Robert Whipple is also the author of The TRUST Factor: Advanced Leadership for Professionals and, Understanding E-Body Language: Building Trust Online. Bob consults and speaks on these and other leadership topics. He is CEO of Leadergrow Inc. a company dedicated to growing leaders.

2 Comments | Candor creates trust, Communications, Enabling Actions, HR, leadership, Organization, Table stakes, Teamwork, Transparency, Trust, Trust and motivation, Trust and respect | Tagged: , , , , , , , , , , , , , , , , , | Permalink
Posted by trustambassador

What If You Are a Jerk But Don’t Know It?

February 27, 2011

It seems impossible, but you could actually be a jerk. You may think you are a perfectly normal, fun-loving person that other people just love to be around, but you could be dead wrong and not even realize it. People might have low respect for you because of any number of bad habits or insensitive things you do or say.

Let’s have a little fun with this analysis and see where it leads. Let’s suppose there is a great bell-shaped curve in the sky that shows the distribution of official jerks. The center of the bell shaped curve is neither a jerk nor a wonderful person. To the left of the center are increasing levels of jerkiness. Individuals far to the left of the mean would be categorized by most people as jerks. The rest of the population are not necessarily jerks, and the ones to the right of the mean are great people.

Now, we separate out the jerks and put them all in a line. Maybe you’ve seen them at the grocery store in the express line with 20 items in their basket. We bring each person into a room individually and ask the person if he or she is a jerk. Note: At this point I am going to switch to the male pronouns “he” and “him” to avoid awkward construction. Actually, the tendency toward being a jerk is probably gender neutral, but I am not going to get into that!

In a high percentage of the cases, the individual will honestly not believe he is a jerk. Reason: this person knows why he is acting the way he is and believes it is the right thing to do in every case. If he believed something else was right, he would do that. In other words, our friend on the low end of the scale would be deceiving himself that he is not a jerk when he actually is one according to other people.

If a person was at the extreme left on the jerk scale, then he might have a clue that he is really rubbing people the wrong way most of the time. He would know that because of the body language and feedback he gets from others. That still does not stop him from being a jerk; it just means that he knows about it.

Now comes the fun part. We add the element of time. Since we can act like a saint one moment and a devil the next, we may be perceived by others as being a jerk sometimes and not other times. Of course, we normally do not know the difference between these two states, so we figure we are basically OK most of the time. Behind our back, people talk about our “problems” and the fact that very often we act like a jerk. What a conundrum. How can we find out when we are acting like jerks? (Ironically, only those people who aren’t jerks would care!)

Enter Emotional Intelligence (EI). The essence of EI is that people who have high levels of this trait have the ability to see themselves more accurately. These individuals have a special mirror that lets them view their own behaviors as others do. In other words, people with high Emotional Intelligence may act like jerks for some small percentage of the time, but they have the perception to know they are doing it. People with low EI have a huge blind spot and cannot detect when they are acting poorly.

This phenomenon is most easy to see in organizations at the leadership or management levels. Leaders with low Emotional Intelligence believe people are responding to them in a different way from what is actually happening – hence the blind spot. So, one cure for the conundrum is to get a higher level of Emotional Intelligence to eliminate the blind spot. Can you buy that stuff at a drug store? No! So how can you get higher EI?

In my leadership classes, students often ask if EI is basically inherited or if it can be learned. I say EI most definitely can be learned. Why? Well, because teaching EI is my occupation: I see significant results when helping leaders gain higher levels of Emotional Intelligence through training and coaching. One thing anyone can do is read about the science of Emotional Intelligence. Start with Emotional Intelligence by Daniel Goleman. Another highly effective way to gain EI is to obtain a great mentor who is really high on the scale of Emotional Intelligence and is willing to pass on to you what it means and how to interpret the signals coming to you from other people.

Higher EI would mean you become more adept at reading body language and become more openly curious about how people are really reacting to the things you say and do. It would mean building trusting relationships with many people who will do you the great kindness of telling you when you are acting like a jerk. The only way to get people to do that is to reward them when they are honest enough to reflect what you are really doing at any given moment (good or bad). These trusted friends can save you from having a blind spot about your own behavior, which automatically increases your EI. Collectively, they form the surface of the mirror that allows you to see yourself as others do. From that point on, you might still be a jerk for some part of the time, but at least you will know it.

5 Comments | Candor creates trust, Communications, leadership, Organization, Teamwork, Trust, Trust and motivation, Trust and respect | Tagged: , , , , , , , , | Permalink
Posted by trustambassador