Talent Development 16 Surveys That Work

November 8, 2020

Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section B reads, “Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.”

For about 5 years, I taught a graduate course called Experimental Design. The course was part of a curriculum leading to a Masters Degree in Organization Learning and Human Resources Development.

In preparation for writing this brief article, I went back and reviewed my slide deck for the course. It was 200 slides long, and many of the slides were heavy with statistical techniques. Obviously, I will need to skim the surface in this short summary article.

In the course, we studied how well-intended surveys often miss the mark and produce bogus results. We also studied the antidotes.

Why Surveys Fail

There are many reasons why surveys fail. I will list some of the more common reasons here and then describe the typical antidotes.

1. Survey too long and complex – one reason for poor data is because people get turned off by too burdensome and too many surveys. When people are angry about too many surveys, they give responses that do not reflect their true feelings.

2. Changes made are not evident – if leaders do not stress that a change is being made because of an employee survey, people will believe their input was ignored. The common misconception sounds like this, “They make us fill out all these ridiculous surveys, but nothing ever changes.”

3. Survey not valid – incorrectly designed surveys often do not measure the thing they are trying to measure. Surveys must be statistically validated to be useful.

4. Survey not reliable – If you would repeat the survey a second time, you would get a different result?

5. Questionable anonymity – If people believe there is some secret way management can find out who said what, then the instrument will not give accurate results. People will respond in ways they think management wants to hear.

6. No clear objective – When people are asked to fill out a survey, they need to know ahead of time why they are being asked to participate and what to expect.

7. Questions not clear – Often the wording of questions leaves people wondering what is really meant by the questions. In this case, you will get guesswork rather than valid data.

8. Leading Questions – Sometimes the way questions are worded leads to skewed data. For example, a political survey might ask, “Are you frustrated by the lies being spread by my opponent?”

Antidotes

I will list the antidotes to the problems in the same order.

1. Make sure your survey is user friendly. Take the survey yourself and ask if you would take the time to do this instrument justice on your most busy day. A good rule of thumb is to be able to fill out the information in less than 10 minutes.

2. Make sure you get back to everyone who responded with the results of the survey. Also, tie all changes made to the survey results, so people are aware of the connection.

3. Test if the survey is valid. The only way you can tell is a survey is measuring what you are trying to measure is to use a statistical analysis of the data. There are five different types of validity (Content, Construct, Concurrent, Criterion, and Predictive). Get help from someone qualified to measure validity. Don’t just wing it.

4. Test if the survey is reliable. This involves trial runs of the survey with different groups under different conditions. The survey needs to produce consistent information to be reliable. Another method is to use a split-half technique. Again, get help if you are not an expert in this area.

5. Insuring anonymity is tricky – The best method I have found to get people to really believe the survey is anonymous is to select a skeptical person from the population to help reduce the data into usable form. The skeptic will let others know that there was no secret means by which management knew who said what.

6. Clarity of Objective – This is a matter of good survey design. It is not just a simple matter of generating some questions and handing out the survey. It must be done with care and solid logic. The way the survey is introduced (typically with an email or letter) is critical. Otherwise you have garbage in garbage out.

7. Test your questions for understandability – This is usually done in the final design phase. You ask people how they interpreted the question. It is not uncommon for many people to be baffled by the wording. Check it out carefully.

8. Avoid leading questions – do not telegraph the requested answer by the way a question is worded. Like don’t ask “Would you like a yummy pizza?”

These areas are general, but they do show how generating a survey is not so simple as most people believe. If a survey is going to generate valuable information for the organization, it needs to be constructed well and administered correctly.

Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.


Body Language 85 Zoom Boom 2 Lighting

May 18, 2020

This is the second of four short articles highlighting the differences from in-person body language and body language when using a virtual platform.

The topic of this article is the lighting and background that is evident in the picture when you are using your video camera.

A common mistake is to sit between the camera and a window, especially on a sunny day.

You will show up as a dark blob in the foreground, and no facial detail will be available to the other participants.

If you have too much light in front of you, either from a window or the computer screen, it can reflect off your glasses and make it hard for people to see your eyes. The cure for this is to screen out the excess light or purchase anti-glare glasses. Another solution to this problem is to wear contact lenses.

I have made videos using the rims of glasses with no lenses in them. That is a good solution when you are alone and just talking at a screen, but when you want to participate with other people in a meeting, you need to be able to see them clearly.

If you have an overhead light, it can be overpowering and make you look washed out or reflect off a bald head so you look like a light bulb. Here again, the solution is some form of light screen so you are surrounded by indirect, but adequate light.

It is important to experiment with the lighting so that other people in the meeting can see your face. Try to create a professional looking environment rather than an obvious bedroom, basement, or attic when working from home. The same rules apply when you are working in an office setting.

Avoid overly complex or messy backgrounds that distract attention from the facial area. Whether at work or at home, try to avoid having the camera pointing toward a high traffic area behind you.

Sometimes having other people in the background is unavoidable, because you are supporting a meeting while in a coffee shop or at the airport. In these situations, people will understand your dilemma.

Many people choose a virtual background, but these do not work particularly well unless you are using a green screen behind you. The picture you input will show as a still or moving image, but when you move, the shape of your head will be grossly distorted until you remain motionless for a few seconds. This movement can be very distracting, although it sometimes provides some comic relief.

What to do

The best approach is to spend time and energy on your setup so that it shows you in the kind of way that reflects professionalism. Have an area set up with the camera and proper lighting and background. Make provision for having meetings in the morning or the afternoon where the challenge of sunlight can be dealt with easily.

For example, I have a window above my work table. If a meeting is in the morning, the shade I use provides just the right amount of light. In the afternoon, if it is a sunny day, there is too much light, so I have a sheet of feather board I can quickly place in front of the window to block the excess light.

Recognize that not all participants may have access to good quality bandwidth where they are located. Expect that some members of the group will need to call in. In these cases, you will have to go by tone of voice and the words that are used to determine the mental state of the individual. There is no video image.

In some cases people will have a picture but use the phone for audio.  Remember to assign the phone number to the appropriate breakout room or the person will not be heard in the breakout.

Consider also your attire. You want to dress as you would if you were at work in a meeting. If you would not wear a colored polo shirt to a meeting in your office, then don’t show up in one for a virtual meeting. If you dress down just because you are working from home, it does not reflect well on you for business discussions.

Even though the Zoom environment seems more informal, you always want to look your best and display a professional demeanor.


This is a part in a series of articles on “Body Language” by Bob Whipple “The Trust Ambassador.”


Leadership Barometer 17 Optimize Communication

September 23, 2019

There are a few leading indicators that can be used to give a pretty good picture of the overall quality of your leadership. Here is one of my favorite measures.

Optimize communication

All of us communicate all of the time. When you add the body language to what we say, there is a steady stream of communication all day, every day. So why does communication always surface in the top 2 of every employee satisfaction survey as the most significant problem facing an organization?

The sad fact is that most leaders are not that good at communicating, even though they work very hard at it. Let’s first look at the symptom from two vantage points.

The leader feels nearly overwhelmed with the need to communicate. In fact, the leader is communicating from the moment she logs on in the morning until she turns out the light exhausted at the end of the day.

All work is a steady stream of explaining what is happening, reinforcing good work, explaining how poor attitudes are not helping, discussing the new product roll out, etc. So it is a frustration when people feedback that there is “never any communication” going on. Wow, what a slap in the face.

From the worker’s perspective, the signals that are coming through are not consistent and often incomprehensible. They long for information in a format and frequency that computes to them.

The messages heard are not consistent with the messages sent by the leader. There are frequent surprises where a vacuum in communication is followed by a “gotcha” announcement.

The battle for excellent communication rages every day in every organization. Let’s take a look at some of the root causes of poor downward communication to uncover some opportunities for improvement.

1. Frequency – the span between communication on key issues is more tricky than meets the eye. The old rule of “the more the merrier” is really not the best policy.

When you constantly say the same message in the same format, eventually people tune it out, and you might better not have said anything because nobody is listening anymore. Yet, the other extreme is worse, if your touch points are so infrequent that people have forgotten the context of the message, then they will listen and hear, but not understand.

So what is the antidote? How do leaders find the sweet spot? You need to let feedback from people be the volume control on your outgoing communication. Most of this feedback comes in body language – often in group settings.

2. Boring Message – I have seen really good leaders who tend to drone on in a monotone style that puts everybody to sleep. So, all the information is given, but everyone is zzzzd out, and there is poor communication.

The best way to avoid this is to watch for the MEGO effect (short for My Eyes Glaze Over). When people get that look, you need to stop and ask a question. Get the audience back with you.

Change the cadence, even use 5 seconds of silence to get the group conscious again. Get people up on their feet or engaged in a question for discussion among small groups. The energy needs to be on a conscious level for people to grasp meaning.

I know the CEO of an organization that communicates with a deck of 50-100 PowerPoint slides. After the third slide, everybody in the audience is politely staring at the screen with the facial appearance of listening when in reality they are absorbing none of the information.

The antidote here is to get the CEO some basic training on PowerPoint no-no’s and make sure he doesn’t sleep through the class.

3. Not what I said – Some people hear what they think you are going to say, even if you say something else. Their predisposition leaves them incapable of absorbing the actual words and meaning.

It reminds me of the old Archie Bunker quote, when he says to his wife, Edith, “The reason you don’t understand me, is because I’m talking in English and you’re listening in Dingbat!” During any presentation, test with your audience if you are getting through the fog. You may need to learn to speak in “dingbat.”

4. Too complex – In an effort to be complete with communications, many leaders are their own worst enemy. People can only absorb and internalize so much information at one time.

Exactly how the information is conveyed has a lot to do with how much can be presented at any one time. Make sure each communication effort has only two or three key points and these are repeated at least three times in the presentation.

Test afterward if people really understood those three key points. Use illustrations when possible, and consider the different learning styles of your audience.

5. Management Speak – Leaders often talk in a kind of language I call “management speak.” They need to understand that the average shop floor person does not relate to ROI or references to Maslow. Make sure your communication is on a level where people can readily grasp the message.

However, be very careful to not “talk down” to people on the shop floor. They are not dumb; in fact they are incredibly smart. They just use different words, and you need to use their language as much as possible when communicating messages to them.

Resist the temptation to “dumb down the message” so they can understand. Instead think of using the right language. Just because they do not know Latin is no reason to treat them as ignorant.

6. Shifting messages – It is not a static world, so a valid message on Wednesday may be the wrong one on Friday. The problem here is that leaders are cognizant of what transpired as the current message morphed into something different.

Unfortunately, the shop floor people are not up to speed on the shifting sands. All they experience is a confusing message that is not consistent. Actually, this problem is more pervasive than leaders recognize, and it is a key reason why there is such a disconnect.

The antidote is for leaders to be extremely cognizant of any small change in the message over time. Make sure you bring people up to speed on the background for the change if you want them to grasp the true meaning.

7. Electronic Communication – Leaders have shifted to a much higher percentage of communication virtually. It is not in the scope of this short article to go over all of the gremlins in this mode of communication. It took me 300 pages in a book to describe how leaders fail to navigate the minefield of successful online communication.

Suffice to say this is an area of great peril. Unfortunately, most leaders think there is little difference between communicating face to face versus online. There is a huge difference (I outline 8 major differences in my book).

An example may help here. Most people view an e-mail like a conversation. You have information coming in, you process it, and then send information out. Just a conversation, right? Wrong! When we talk to people face to face, we are constantly modifying the message, cadence, body language, and the words based on the real-time feedback we are getting.

Online, there is no feedback while the message is being sent. It is all blind, and we have no way to correct things if we are off track. Thinking of online communication like a conversation is extremely dangerous.

8. Communicating at the head level – Good communication does not occur at the “head” level. Sure, we use the mouth to speak, the ears to hear, the brain to interpret, the eyes to see, etc. Real communication is deep in the gut.

When you have internalized the message fully, it goes well into the body. Don’t make the mistake of thinking you have communicated with someone because you have talked and they appear to have heard it. Verify what was taken in at the gut level.

Those are just 8 ways of improving communication. Actually there are hundreds of them, this article only scratches the surface. But, if you focus on these few important considerations, you can really improve your communications with people.

Bob Whipple is CEO of Leadergrow Inc., a company dedicated to growing leaders. He speaks and conducts seminars on building trust in organizations. He can be reached at bwhipple@leadergrow.com or 585-392-7763.


Communication Complexities

October 14, 2012

Most of us have played the campfire game where a bunch of kids sit around the fire and pass a message from one to the other. It never fails that the message coming out at the end bears little resemblance to what was started.

The same kind of phenomenon is going on when two people try to communicate. There are many steps in the communication process, each of which might be pictured as an individual cub scout sitting around the fire. Here are ten steps that happen each time we say something to someone else:

1. I have a thought that I want to convey to you.

2. I decide how I am going to convey that message to you with my choice of words.

3. I send the message according to my interpretation of how my words will translate my true intent. (I will discuss tone and body language below.)

4. The information goes out from me through the air in sound waves.

5. You then pick up some portion of those waves depending on your level of attention and your physical ability to receive them. You never get them all.

6. You process the information based on your interest in what I am saying and your current level of distraction.

7. You make an interpretation of the information based on your biases and filters about how you perceive the world and what you were expecting me to say.

8. You make a decision how to translate the input into reaction thought patterns in your brain.

9. You make a determination about what you are going to do with the information.

10. You then give some external reaction, comment, or action based on your thoughts.

In each of these steps, there is the potential for tiny modifications of the original thought. Each modification may seem insignificant, but just as in the case with the campfire game, if you add up all of the minute changes, the final meaning may be quite different from the original one.

If the communication is reasonably good, then the thought in my head would be planted in your head roughly intact. If one step in the process modifies the input slightly, the starting point for the next step will be different, and a significant distortion in the final received message is likely.

When you add in the infinite variety of signals included in tone of voice and body language, the complexity goes up exponentially. The complexity involved in getting the words right is a significant challenge, but studies show that the words contain only a tiny fraction of the meaning we get. In 1967, Albert Mehrabian measured that when talking about feelings or emotions, only about 7% of the meaning is contained in the words we use. The remaining 93% of content is in the tone of voice and body language.

If I say to you, “You couldn’t have been any better in that meeting this morning,” the message you will receive is highly dependent on my voice inflection and body language. The same words can have very different, even opposite, meanings.

Body language is so complex because we send signals on many different levels subconsciously. The meaning you get will be colored by my skill at accurately projecting the intent behind the communication and also your skill at picking up the signals and decoding them correctly. There may be cultural differences as well that can make the interpretation even more complex. That is why knowledge of and appreciation for the complexities of body language are essential for good communication.

When you consider the complexity of this process, it is not shocking that a fair percentage of meaning in direct communication does not even hit the target area, let alone accomplish a bulls-eye. I think it is amazing that we get as close as we do.

When miscommunication happens, it is a natural reaction to become frustrated and even angry. We may jump to conclusions about the worthiness of our partner in communication. We say things like, “You are not speaking so I can understand your message,” or “You never listen to me,” or “You just don’t pay attention to what I am saying.” All of these scape-goating expressions may make us feel better by putting the blame on the other person, but they do not identify or rectify the root cause.

What is needed when message content becomes garbled is a sense that the inevitable straying off message has occurred. It is not necessarily the fault of either person. It just may take more than one attempt to communicate a message. To mitigate the problem, we need to patiently verify the message internalized is the same as the message sent. That takes a verification step, either verbally or with body language. Since the original communicator is 100% sure of what he or she thinks was said, it seems redundant to go through a verification ritual, but it is really necessary, especially for important messages.

When communicating with another person, keep in mind the complex process that is going on. Use your powers of observation to detect possible visual or verbal cues that the communication did not work as intended. Try to not blame the other person, because the truth is, it is a system problem, and you are also part of the system. Work on improving your own system both on the sending side and the receiving side.