Talent Development 16 Surveys That Work

November 8, 2020

Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section B reads, “Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.”

For about 5 years, I taught a graduate course called Experimental Design. The course was part of a curriculum leading to a Masters Degree in Organization Learning and Human Resources Development.

In preparation for writing this brief article, I went back and reviewed my slide deck for the course. It was 200 slides long, and many of the slides were heavy with statistical techniques. Obviously, I will need to skim the surface in this short summary article.

In the course, we studied how well-intended surveys often miss the mark and produce bogus results. We also studied the antidotes.

Why Surveys Fail

There are many reasons why surveys fail. I will list some of the more common reasons here and then describe the typical antidotes.

1. Survey too long and complex – one reason for poor data is because people get turned off by too burdensome and too many surveys. When people are angry about too many surveys, they give responses that do not reflect their true feelings.

2. Changes made are not evident – if leaders do not stress that a change is being made because of an employee survey, people will believe their input was ignored. The common misconception sounds like this, “They make us fill out all these ridiculous surveys, but nothing ever changes.”

3. Survey not valid – incorrectly designed surveys often do not measure the thing they are trying to measure. Surveys must be statistically validated to be useful.

4. Survey not reliable – If you would repeat the survey a second time, you would get a different result?

5. Questionable anonymity – If people believe there is some secret way management can find out who said what, then the instrument will not give accurate results. People will respond in ways they think management wants to hear.

6. No clear objective – When people are asked to fill out a survey, they need to know ahead of time why they are being asked to participate and what to expect.

7. Questions not clear – Often the wording of questions leaves people wondering what is really meant by the questions. In this case, you will get guesswork rather than valid data.

8. Leading Questions – Sometimes the way questions are worded leads to skewed data. For example, a political survey might ask, “Are you frustrated by the lies being spread by my opponent?”

Antidotes

I will list the antidotes to the problems in the same order.

1. Make sure your survey is user friendly. Take the survey yourself and ask if you would take the time to do this instrument justice on your most busy day. A good rule of thumb is to be able to fill out the information in less than 10 minutes.

2. Make sure you get back to everyone who responded with the results of the survey. Also, tie all changes made to the survey results, so people are aware of the connection.

3. Test if the survey is valid. The only way you can tell is a survey is measuring what you are trying to measure is to use a statistical analysis of the data. There are five different types of validity (Content, Construct, Concurrent, Criterion, and Predictive). Get help from someone qualified to measure validity. Don’t just wing it.

4. Test if the survey is reliable. This involves trial runs of the survey with different groups under different conditions. The survey needs to produce consistent information to be reliable. Another method is to use a split-half technique. Again, get help if you are not an expert in this area.

5. Insuring anonymity is tricky – The best method I have found to get people to really believe the survey is anonymous is to select a skeptical person from the population to help reduce the data into usable form. The skeptic will let others know that there was no secret means by which management knew who said what.

6. Clarity of Objective – This is a matter of good survey design. It is not just a simple matter of generating some questions and handing out the survey. It must be done with care and solid logic. The way the survey is introduced (typically with an email or letter) is critical. Otherwise you have garbage in garbage out.

7. Test your questions for understandability – This is usually done in the final design phase. You ask people how they interpreted the question. It is not uncommon for many people to be baffled by the wording. Check it out carefully.

8. Avoid leading questions – do not telegraph the requested answer by the way a question is worded. Like don’t ask “Would you like a yummy pizza?”

These areas are general, but they do show how generating a survey is not so simple as most people believe. If a survey is going to generate valuable information for the organization, it needs to be constructed well and administered correctly.

Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.


Body Language 91 Ready to Make a Deal?

August 24, 2020

Body Language is relevant in all aspects of our life. The topic is particularly useful in the field of sales. Highly skilled sales professionals are trained to look for many different body language shifts that indicate the prospect has crossed the mental chasm from skeptical to sold.

This article highlights a few of the signs you can observe in this dynamic.

Deep Breath

If a person has been breathing shallow slow breaths and all of a sudden takes in a huge breath and lets it out slowly, that signals a change in mental attitude. It is likely that the person is expressing the lowering of overall tension.

The technique is known as a cleansing breath because the impact is to acknowledge the tension going out of the body.

Feet to Floor

If the buyer is sitting with crossed legs suddenly uncrosses his or her legs and puts both feet on the floor, it is usually a sign the person is ready to sign on the dotted line.

Unbutton Jacket

If a man in a business suit that has been buttoned suddenly unbuttons it and pushes it out to the sides, thus exposing the solar plexus, it is a very good sign. You are scoring points.

Open Palms

If a person who has been sitting at a table or desk with closed fists and knuckles on the surface, turns hands over with palms upward, it shows a transition from being resistant to being open.

Dilated Pupils

If you observe the pupils of the person are larger than normal, you have an indication of anticipation that is generally a good indication of a deal. Note, the other person has no way of observing his or her own pupils, so you have a significant advantage by checking for that clue.

Increased Blinking Rate

This signal can go either way, so be careful. If the other person is irritated by the negotiation, the result may be an increase in blinking rate.

However, an increase in blinking rate could also signal anticipation of closing the deal. If you observe increased blinking rate, check for other signs to understand which direction is operational.

Inward Lean

If a person who has been sitting back in the chair suddenly leans in, that is a signal that the person is ready to close the deal.

Increased Eye Contact

If a person who has had difficulty maintaining good eye contact, all of a sudden increases eye contact to the level of 60% to 70%, you have likely made the sale.

The Opposite Signal

If a person has his or her notebook open on the table in front of him, then suddenly closes his book and folds his hands on top of it, that gesture means no sale is likely today. The person has just shut down the negotiation for this session.

There are many other body language gestures that can help you identify when a person is ready to make a deal. Many of these have to do with facial expressions, such as skeptical versus a satisfied smile. Stay alert when negotiating for another person over anything, from which food to order to buying a house. Knowing these signals will help you come out with a better result.

This is a part in a series of articles on “Body Language” by Bob Whipple “The Trust Ambassador.”



Body Language 79 Skeptical

June 14, 2020

There are many different ways we can express skepticism without using any words. This article will highlight some of the typical body language gestures that can be seen if a person is skeptical.

There are numerous facial cues you can use to identify a skeptical person and also some telltale hand gestures. We will start by observing the eyes.

Eyes

A skeptical person will often look at you with a sideways glance. The message is “do you expect me to believe this?”

Alternatively, the person may be squinting at you like what you are saying is painful or just does not compute.

A third option with the eyes is having them wide open in a somewhat surprised stance or looking over the rims of his glasses.

Eyebrows

The eyebrows will often be raised as the person contemplates what as just said. The connotation is – really?? Sometimes the eyebrows will be pulled toward the bridge of the nose as an indication of confusion, concern, or disbelief.

Head tilt

Often you will see a tilted head when observing a skeptical person. The message being conveyed is that the person is thinking something is definitely wrong with what you just said or did but cannot quite figure out what it is.

Mouth

The most often mouth gesture for a skeptical person is a kind of pout. Alternatively, you might see the mouth pulled slightly to one side and either be open or shut. The connotation is that the person is straining to believe what you just said.

Hand gestures

There are many different hand gestures associated with a skeptical person. A common one is stroking the chin area. The person is trying to rationalize what was just said, so he is pondering the meaning.

Another common hand gesture is with arms extended and the hands palm up and open. It is like the person is trying to feel the weight of what you just said.

You might see an extended index finger pointing at you or even a “time out” signal with the tips of one hand touching the palm of the other hand.

What to do

If you see a cluster of these kinds of gestures, you can be pretty certain the other person is skeptical about what is going on. The best approach is to invite dialog with a question. Here are a few examples of questions that may draw the other person out.

Do you find this hard to believe?
You seem doubtful – what’s wrong?
Can you tell me how you feel about what I just said?
Does this seem right to you?
Is there another way of looking at this?

Then, pay particular attention to the response you get and try to avoid getting defensive. The other person is entitled to his or her opinion, and you need to handle the conversation with tact in order to start rebuilding any lost trust.

This is a part in a series of articles on “Body Language” by Bob Whipple “The Trust Ambassador.”