Talent Development 16 Surveys That Work

November 8, 2020

Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section B reads, “Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.”

For about 5 years, I taught a graduate course called Experimental Design. The course was part of a curriculum leading to a Masters Degree in Organization Learning and Human Resources Development.

In preparation for writing this brief article, I went back and reviewed my slide deck for the course. It was 200 slides long, and many of the slides were heavy with statistical techniques. Obviously, I will need to skim the surface in this short summary article.

In the course, we studied how well-intended surveys often miss the mark and produce bogus results. We also studied the antidotes.

Why Surveys Fail

There are many reasons why surveys fail. I will list some of the more common reasons here and then describe the typical antidotes.

1. Survey too long and complex – one reason for poor data is because people get turned off by too burdensome and too many surveys. When people are angry about too many surveys, they give responses that do not reflect their true feelings.

2. Changes made are not evident – if leaders do not stress that a change is being made because of an employee survey, people will believe their input was ignored. The common misconception sounds like this, “They make us fill out all these ridiculous surveys, but nothing ever changes.”

3. Survey not valid – incorrectly designed surveys often do not measure the thing they are trying to measure. Surveys must be statistically validated to be useful.

4. Survey not reliable – If you would repeat the survey a second time, you would get a different result?

5. Questionable anonymity – If people believe there is some secret way management can find out who said what, then the instrument will not give accurate results. People will respond in ways they think management wants to hear.

6. No clear objective – When people are asked to fill out a survey, they need to know ahead of time why they are being asked to participate and what to expect.

7. Questions not clear – Often the wording of questions leaves people wondering what is really meant by the questions. In this case, you will get guesswork rather than valid data.

8. Leading Questions – Sometimes the way questions are worded leads to skewed data. For example, a political survey might ask, “Are you frustrated by the lies being spread by my opponent?”

Antidotes

I will list the antidotes to the problems in the same order.

1. Make sure your survey is user friendly. Take the survey yourself and ask if you would take the time to do this instrument justice on your most busy day. A good rule of thumb is to be able to fill out the information in less than 10 minutes.

2. Make sure you get back to everyone who responded with the results of the survey. Also, tie all changes made to the survey results, so people are aware of the connection.

3. Test if the survey is valid. The only way you can tell is a survey is measuring what you are trying to measure is to use a statistical analysis of the data. There are five different types of validity (Content, Construct, Concurrent, Criterion, and Predictive). Get help from someone qualified to measure validity. Don’t just wing it.

4. Test if the survey is reliable. This involves trial runs of the survey with different groups under different conditions. The survey needs to produce consistent information to be reliable. Another method is to use a split-half technique. Again, get help if you are not an expert in this area.

5. Insuring anonymity is tricky – The best method I have found to get people to really believe the survey is anonymous is to select a skeptical person from the population to help reduce the data into usable form. The skeptic will let others know that there was no secret means by which management knew who said what.

6. Clarity of Objective – This is a matter of good survey design. It is not just a simple matter of generating some questions and handing out the survey. It must be done with care and solid logic. The way the survey is introduced (typically with an email or letter) is critical. Otherwise you have garbage in garbage out.

7. Test your questions for understandability – This is usually done in the final design phase. You ask people how they interpreted the question. It is not uncommon for many people to be baffled by the wording. Check it out carefully.

8. Avoid leading questions – do not telegraph the requested answer by the way a question is worded. Like don’t ask “Would you like a yummy pizza?”

These areas are general, but they do show how generating a survey is not so simple as most people believe. If a survey is going to generate valuable information for the organization, it needs to be constructed well and administered correctly.

Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.


Body Language 73 Coy

March 27, 2020

How can you tell if another person is being coy? In this article I will give some tips to recognize the body language gestures associated with being coy and give some ideas to deal with the situation.

I will start with being coy in a business setting and then cover the same topic from a social point of view. The former usually involves someone being somewhat evasive while being coy in a social setting runs the gamut between deception all the way to overt flirtation.

What is being coy?

The definition of coy is “pretending to be shy or modest.” Another definition from Merriam-Webster’s 11th Collegiate Dictionary is..”marked by cute coquettish or artful playfulness.” A third perspective of being coy includes a reluctance to make a commitment or give details about something regarded as sensitive.

Eyebrows

When a person is being coy, he will usually have one eyebrow noticeably raised. You do not see both eyebrows raised, because that gesture would normally be associated with the emotion of surprise or even shock.

The clandestine look is what gives the gesture the appearance of some mystery. That can be what creates the playfulness of the gesture. The uncertainty of what he is thinking enhances the effect.

Eyes

A coy person will usually be looking sideways. He does not look up or down, but the eyes are noticeably looking to the side. It is an evasive kind of movement, like he is making an effort to hide something.

He rarely will make direct eye contact when being coy because that would reduce the mystery effect.

Cheeks

When a person is being coy, you usually see only one cheek as he will turn his head and not look at you straight on. He may also have a tilted head to accentuate the expression.

Mouth

There is often a slight pulling back of the one cheek in an effort to make a mysterious smile. Alternatively, the mouth can also form a sober frown, as in the picture.

Chin

Often the chin is lowered a bit as if to hide something. It would be unusual to see a coy person with his head held high. Part of the secret look is to lower the chin.

In a Business Setting

Being coy in a business setting means that there is something a person wants to express but prefers to only hint at until the other person begins to get a message. It can also mean reluctance to give information the other person wants.

The desired outcome is rarely sexual, as in social expressions of being coy. Many of the body language signals will be the same, as is shown in the picture above of a male being coy in a work setting.

Perhaps he knows something, but he does not trust the other person enough to reveal it. Perhaps he is playing some kind of game where he wants to stall for time for some reason. This is not necessarily negative, since a person can have valid reasons for keeping something private.

One approach to break the tension is to ask the person if he is uncomfortable at the moment. If he agrees, then you have the opportunity to make open ended questions in an attempt to find out the root cause of the discomfort.

Being coy is also used in marketing when an organization wants to build anticipation for a new product but is not quite ready for the grand announcement.

The art of being coy is often seen in political situations where a candidate is not ready to announce his or her true intentions. This application often comes across as artful dodging, which tends to lower trust because people recognize they are not getting the full truth.

Being coy in a social setting

The most well-known examples of being coy occur in a social setting, rather than a business or political setting, as in the attached picture.

The gestures are usually intended to be provocative and involve another person. People would not make these movements if they were alone.

The smile may be slightly pulled to the side reminiscent of the Mona Lisa look. The half-smile is an indication of potential pleasure. She may also exhibit pursed lips as if in a mock kissing gesture.

Conclusion

When a person (male or female) is being coy, you have to recognize that there is some kind of agenda going on. Be careful to not misinterpret the coyness as attraction. There could be a number of things going on. The best advice is to remain unsure until you have other indications for why the person is making these gestures.

Look for the opportunity for some dialog. Observe more and try to ask open ended questions for more data. If the person begins to open up, then you can improve the accuracy of your understanding and not make unwarranted assumptions.


This is a part in a series of articles on “Body Language” by Bob Whipple “The Trust Ambassador.”