Section 2.8 in the CPTD Certification program for ATD is Evaluating Impact. Section B reads, “Skill in creating data collection tools, for example questionnaires, surveys, and structured interviews.”
For about 5 years, I taught a graduate course called Experimental Design. The course was part of a curriculum leading to a Masters Degree in Organization Learning and Human Resources Development.
In preparation for writing this brief article, I went back and reviewed my slide deck for the course. It was 200 slides long, and many of the slides were heavy with statistical techniques. Obviously, I will need to skim the surface in this short summary article.
In the course, we studied how well-intended surveys often miss the mark and produce bogus results. We also studied the antidotes.
Why Surveys Fail
There are many reasons why surveys fail. I will list some of the more common reasons here and then describe the typical antidotes.
1. Survey too long and complex – one reason for poor data is because people get turned off by too burdensome and too many surveys. When people are angry about too many surveys, they give responses that do not reflect their true feelings.
2. Changes made are not evident – if leaders do not stress that a change is being made because of an employee survey, people will believe their input was ignored. The common misconception sounds like this, “They make us fill out all these ridiculous surveys, but nothing ever changes.”
3. Survey not valid – incorrectly designed surveys often do not measure the thing they are trying to measure. Surveys must be statistically validated to be useful.
4. Survey not reliable – If you would repeat the survey a second time, you would get a different result?
5. Questionable anonymity – If people believe there is some secret way management can find out who said what, then the instrument will not give accurate results. People will respond in ways they think management wants to hear.
6. No clear objective – When people are asked to fill out a survey, they need to know ahead of time why they are being asked to participate and what to expect.
7. Questions not clear – Often the wording of questions leaves people wondering what is really meant by the questions. In this case, you will get guesswork rather than valid data.
8. Leading Questions – Sometimes the way questions are worded leads to skewed data. For example, a political survey might ask, “Are you frustrated by the lies being spread by my opponent?”
I will list the antidotes to the problems in the same order.
1. Make sure your survey is user friendly. Take the survey yourself and ask if you would take the time to do this instrument justice on your most busy day. A good rule of thumb is to be able to fill out the information in less than 10 minutes.
2. Make sure you get back to everyone who responded with the results of the survey. Also, tie all changes made to the survey results, so people are aware of the connection.
3. Test if the survey is valid. The only way you can tell is a survey is measuring what you are trying to measure is to use a statistical analysis of the data. There are five different types of validity (Content, Construct, Concurrent, Criterion, and Predictive). Get help from someone qualified to measure validity. Don’t just wing it.
4. Test if the survey is reliable. This involves trial runs of the survey with different groups under different conditions. The survey needs to produce consistent information to be reliable. Another method is to use a split-half technique. Again, get help if you are not an expert in this area.
5. Insuring anonymity is tricky – The best method I have found to get people to really believe the survey is anonymous is to select a skeptical person from the population to help reduce the data into usable form. The skeptic will let others know that there was no secret means by which management knew who said what.
6. Clarity of Objective – This is a matter of good survey design. It is not just a simple matter of generating some questions and handing out the survey. It must be done with care and solid logic. The way the survey is introduced (typically with an email or letter) is critical. Otherwise you have garbage in garbage out.
7. Test your questions for understandability – This is usually done in the final design phase. You ask people how they interpreted the question. It is not uncommon for many people to be baffled by the wording. Check it out carefully.
8. Avoid leading questions – do not telegraph the requested answer by the way a question is worded. Like don’t ask “Would you like a yummy pizza?”
These areas are general, but they do show how generating a survey is not so simple as most people believe. If a survey is going to generate valuable information for the organization, it needs to be constructed well and administered correctly.
Bob Whipple, MBA, CPTD, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, Leading with Trust is Like Sailing Downwind, and Trust in Transition: Navigating Organizational Change. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations.
I work with leaders on a regular basis, and most of them wish they were better at delegating. I have yet to meet a person who believes delegating is a bad thing to do.
Granted, it is possible overdo the technique and get into trouble, just as one can overdo any good thing, but for most of us, we would be far more effective if we did more delegation rather than less.
The reason for not delegating stems from each person’s desire to have things done well. We want things to be done the way we would do them, and are afraid that some other person will not live up to the standards we have for ourselves.
The excuse often given is “it is much easier to just do it myself than to teach the other person to understand how I want it done and make sure he does it that way.” That thinking sounds like just being honest, but it is not a helpful way to think.
The fear is not just about getting the work done the “right” way. It is also a sociological fear that if we need to have the work redone, then we have made an enemy or at least have to do some coaching to calm the other person down.
The dread of having to deal with the consequences of a failed attempt and the rework involved is very real and makes us feel like the time is better spent just doing the job ourselves. That approach will also prevent the time pressure if there is an urgency to the task.
You cannot use the “Law of Leverage” to multiply your good influence in the world until you let go of the idea of perfection and grab onto the concept of “excellence by influence.”
By trusting other people to figure out the best way to do something and leaving them alone to do it “their way,” you unleash the power of creative thinking and initiative in other people. They will often surprise you by delivering work and solutions that are far better and arrive sooner than you would have done yourself.
To have subordinates perform as you wish, it is first important to ensure you have defined the desired outcome. Make sure they can recite the objective back to you before they go off to accomplish the task.
This is also a great time to verify they have the resources needed to accomplish the work. Many managers fail to provide the time, money, or other resources that will be needed to do the job and then become frustrated when an employee tries to improvise a sub-optimal solution.
A typical problem is that managers have a preconceived idea of what the ideal solution will resemble. When we see the result of the work done by a creative and turned-on individual, it just does not look like the solution we envisioned, so the “not invented here” syndrome takes over, and we send signals that the work is not good enough.
It is hard to admit that the solution we are presented with is, in many cases, a superior one. Here are some ideas that can help you lower this rejection reaction and be more accepting of the solutions others present.
1. Does it do the job?
In every task there are countless ways to achieve a result that actually does the job intended. When you see the work of another person, try to imagine that the solution you see is one of hundreds of alternatives, including the one you had in mind.
2. Did it help the other person grow?
Our job as managers and leaders is not only to get everything done according to some standard. Our primary purpose is to help people grow into their powerful best, which means putting higher value on what the person is learning than on the particular solution to a specific task. Even if the solution turns out to be flawed, it still is a success in terms of helping the person learn and grow.
3. Are you making a mountain out of a molehill?
We often get so intense about how things are being perceived by our own superiors that we lose sight of the bigger picture. By showing high trust and enabling more people to leverage their skills, you are going to be perceived very well, even if there is an occasional slip.
4. Who is the judge for which is the best solution?
Clearly if you have a preconceived idea of what the solution looks like, you are not in a position to be objective. You are already biased in the direction of your vision.
5. What kind of culture do you want?
To have an engaged group, you need to empower people by giving them tasks and trusting them to use their initiative and creativity to find their own solutions. If you want everything done “your way,” you will end up getting what most organizations typically do, which is roughly 30% of the discretionary effort that is available in the workforce. You end up with compliance rather than excellence.
6. What are you really risking?
When you stop and think about it, the risks involved are really quite small. Even if something does not work out, it will be of little consequence in a week or two. The risk is even lower if people are becoming more engaged in the work and more skilled over time through trial and error.
7. What is the best for you?
Realizing that you have a choice to micromanage or not and choosing to be an empowering rather than stifling manager lets you sleep a lot better at night. That is a huge advantage and well worth having to endure a serviceable solution that is not exactly what you had in mind.
The benefits of good delegation are well documented. Few people would vote for less delegation by any manager, so why not learn to set good objectives and trust people to come up with good solutions? You will find it is not as hard as you imagine, and your overall performance will go up dramatically as you leverage resources better.
Bob Whipple, MBA, CPLP, is a consultant, trainer, speaker, and author in the areas of leadership and trust. He is the author of: The Trust Factor: Advanced Leadership for Professionals, Understanding E-Body Language: Building Trust Online, and Leading with Trust is Like Sailing Downwind. Bob has many years as a senior executive with a Fortune 500 Company and with non-profit organizations
OD is short for Organization Development. This is not a new term. Behavioral scientists have been writing about Organization Development for over 40 years. The science has evolved into many different approaches all aimed at the same objective: to enable massive improvements in organizational performance through specific and planned interventions.
I have been involved with hundreds of OD efforts over the past decades. Some of these have resulted in the desired improvement. Some have not. In this article I will reveal some green lights, some caution (yellow lights), and some things to stop doing, or red lights.
Let’s review four major types of OD interventions (there are others, but they are usually variations or combinations of these four):
Although the objective of each of these techniques is the same, the viewpoint and methodology for each is different. I will give my personal views of the strengths and problems with each method from my experience. All of these methods can work. The trick is to match the leadership style and organization culture so that the one selected has the best chance of success in a particular case.
Most OD work is performed with the assistance of trained facilitators. They have the professional training to lead groups through the chaos of change to arrive at the objective. Managers who attempt a “do it yourself” approach to OD work often create more turmoil and make things worse. This is especially true if the leadership dynamic is part of the problem (which is usually the case).
OD work is tricky. It requires the skill of someone trained in this field. Headstrong managers who decide to undertake massive organization change without help are like critically ill patients trying to remove their own appendix. It is not a smart strategy. The flip side is that the effort needs to be owned by the manager rather than the consultant. Leaders who abdicate their responsibility to be the spiritual leader of the organization pay for it with lower trust.
Most organizations contemplating an OD initiative, do so because they are not satisfied with how things are going. If the current trajectory of business is meeting or exceeding goals, there is little impetus for change. The Action Search approach takes on a somewhat negative spin from the outset. The idea is to determine what is wrong and fix it quickly.
The first stage is to gather data. What areas of the business are falling short? How can these be changed to perform better? Unfortunately, many efforts using this technique become “witch hunts” where management looks for scapegoats. The process becomes one of uncovering ugly issues, followed by defensive tactics by those in charge.
Most of us have participated in this type of intervention. It takes place on a regular basis in some companies. Ask yourself how successful these programs have been in your experience. Do they produce positive change, or simply mask more underlying issues while creating interpersonal chaos? My experience indicates this technique should be used only under very tight constraints with ground rules supporting solid values. That does not happen very often. Hence, using Action Research has a real potential to backfire if not managed extremely well.
This approach is the mirror image of the “action research” technique. The process starts by asking what is working well. Groups focus on what is going right rather than what is going wrong. The idea is to find ways of doing more of the right stuff, thus providing less reinforcement for doing the wrong stuff.
This is a much more pleasant process. It feels good to focus on strengths. It also provides a benchmark for improvement. The danger is that groups who are failing miserably can deceive themselves into thinking all they need do is clone the few bright spots to succeed.
I witnessed an example of this, years ago, and it was ugly. One business unit was on the verge of extinction, so they did a three-day exercise in appreciative inquiry. By the end of the exercise, they were celebrating, dancing, and singing about their wonderful opportunities while they were actually going out of business. Six months after the crepe paper, helium balloons, high fives, and “jive dancing,” they were all looking for new jobs.
I believe appreciative inquiry can be much more powerful than action research, but it needs to be tempered by reality. A combination of both methods can avoid a kind of “Pollyanna” view of reality.
In this process, the focus is on the vision rather than the current state. The idea is to get groups engaged in defining a compelling view of the future. When compared to the present, this allows clarification of the gaps between current practices and organizational goals. Outstanding vision is the most powerful force for all individuals and organizations. Here are some comments on vision from my book (Whipple, 2003, p27).
Without a well-defined vision, the organization has no true direction. It is like a ship without a rudder, sailing around at the mercy of the wind, hoping to find a safe port with little chance of reaching one. Creating vision is absolutely essential for any group because it gives a common direction and provides a focus for energy.
Not all vision statements are helpful. Some are relegated to plaques on the wall and ignored. This is a tragedy because an uninspiring vision breeds apathy and is worse than no vision at all. If people point to the vision statement on the wall and say, “that is where we are supposed to be going but they don’t act that way,” you are in trouble.
Getting a great vision is not a 15-minute exercise. Some groups spend months working on developing a good vision statement. The process can get convoluted and burdensome if not handled correctly. If you are adept at facilitating group discussions, you may conduct this yourself.
If not, a professional facilitator would be worth the investment. As the leader, even if you feel qualified to lead the discussion, you still may want to hire an outside person so you can become one of the people developing this material. The danger if you lead the discussion is that you could influence it too heavily.
In general, if a leader brings in a consultant to facilitate a discussion or to assist with a particular instrument or skill set, there is usually a high value.
If the consultant is brought in to get into the trenches and do the dirty work of leadership, it is often a disaster because the consultant can undermine the leader. The leader calls in a consultant and says, “Things are a mess around here and I’m under a lot of pressure. Performance is horrible recently and morale is way down. I haven’t time to fix the problem because I am overloaded just trying to run the business and I have to attend all these management meetings. I need you to assess what is wrong and recommend a program to get back on track. If my team buys into your recommendations, we will let you handle the program.”
This leader probably has lost the ability to lead the organization effectively. As the consultant mucks around trying to understand problems, significant negative energy is unearthed but the consultant doesn’t have the authority to fix these issues. Meanwhile, the leader is “busy running the business,” and being micro-managed by superiors. Morale and performance go down even further until, finally, the leader is simply forced out.
This is why it is important for the leader to be the driving force in creating a vision for the organization. It cannot be delegated to a consultant or even a high-ranking lieutenant. The leader is responsible for making sure the vision statement is clear, compelling, memorable, actionable, and real.
Key ideas for developing a good vision statement:
• Most importantly, make sure your vision tells everyone where the organization is going. A nice sounding phrase that doesn’t have pull makes a poor vision. For a football team “We will be number one in the league within 3 years” is a better vision than “We will improve our position in the rankings every year until we become the top team in the league.”
• Avoid grandiose sweeping statements that are too broad. “We will become the best in the world at computer technology” would be too general and vast for a good vision statement. A better example might be “Our superior microchips will gain 90% market share with computer manufacturers in 5 years.”
• Make sure people can connect their everyday activities to the vision. “Every interface is a chance to bestow great customer service” would allow everyone to view daily activities with customer service getting top billing.
• Keep it short and powerful. Avoid long lists of items that sound good but don’t create a picture. For example, being “trustworthy, loyal, helpful, friendly, courteous, kind, obedient, cheerful, thrifty, brave, clean, and reverent” may be a good motto for the Boy Scouts, but it would make a terrible vision statement.
• Select colorful words that inspire rather than describe. “Our greeting cards melt the heart and transform the soul” would be superior to “Our greeting cards are better because they make people feel great.”
• Keep it short. The fewer words the better. “Absolutely, positively overnight” is better than “Our packages are guaranteed to arrive by the next day or your money back.”
• Use special words to emphasize your most significant point. “We will never, ever, run out of stock” is better than “We promise to keep our customers needs met by always having stock on hand.”
• Don’t try to be abstract or cute in order to grab attention. “We have the softest software in the nation” might be a slogan helpful on Madison Avenue, but it makes a lousy vision. Instead try “Software delivered on time, every time!”
The initial thoughts often contain the seeds of the eventual finished product. Craft these thoughts into words and images. Sometimes a picture or logo can be enough to communicate a vision, like the Rock of Gibraltar for Prudential Insurance. Other times, it can be a slogan, such as Wegmans Market’s “Every day you get our best” or General Electric’s “We bring good things to life.” The expression needs to have “pull”; it must provide forward momentum.
Communicate the organization’s values and vision to everyone in it. Do this well and often, as it forms the basis of everything to come. Frequently demonstrate your alignment with the vision by naturally working it into conversations. You might say, “Well, let’s call the customer and tell them about this situation. After all, our vision is to put the customer first.”
Whole System Intervention
This is a kind of zero-based approach to OD. In this case, the activities of the organization are viewed through a “systems” approach. The emphasis is on getting a critical mass within the organization to redefine the business. Processes become the focal point for redesign efforts. This is less threatening than the action research technique because of focuses on the “what” and “how” rather than the “who.”
The challenge with a systems approach is that can get pretty complicated. In systems thinking, we try to understand the interrelations between things. This is opposed to the usual linear way of thinking – If we do one thing it results in an effect. In systems thinking we need to understand not only the direct effect of actions but also the side effects. If leaders are unhappy with performance, they need to look at their system because it is perfectly designed to give exactly the result they are getting. Trying to untangle what is hurting the system and streamline the process for a better result can get convoluted.
The four OD interventions described in this article are the cornerstones for organizational improvement. They need to be applied with care and judgment to be effective. When OD activities go awry, the “cure” is often worse than the “disease.” With the health, or even survival, of the organization at stake, it is important to do this work carefully with the assistance of an expert.
The preceding information was adapted from the book The TRUST Factor: Advanced Leadership for Professionals, by Robert Whipple. It is available on http://www.leadergrow.com.
Robert Whipple is also the author of Leading with Trust is like Sailing Downwind and, Understanding E-Body Language: Building Trust Online. Bob consults and speaks on these and other leadership topics. He is CEO of Leadergrow Inc. a company dedicated to growing leaders.
I originally intended to have the title for this article read “How Do You Know When You Are Biased.”
The more I thought about it, the more I realized that it is impossible to be totally unbiased on an issue. The most we can do is be conscious of our biases and factor that knowledge into our deliberations. We can also seek to replace hubris with humility.
To be totally objective, it would put us in the category of a machine or computer. Unless you are Mr. Spock, you have emotions and cannot separate your logical reasoning from your emotions entirely.
You also carry a set of beliefs that are made up of the sum of all the experiences you have had to date. You cannot detach yourself from your unique mindset any more than the earth can detach itself from the solar system.
With herculean effort you may be able to change your orbit a perceptible amount, but you will always be subject to the laws of physics in your corner of the universe. Notwithstanding another “big bang,” you are going to orbit around the sun forever.
One place to observe bias is when managers try to measure accurately the performance of people who work for them. Imagine a manager trying to write an objective performance appraisal.
Because the manager is a human being, he or she will observe performance through the lens of how he or she feels about the employee. It would be impossible to factor out personal biases, but by recognizing that there is the certainty of being biased, the manager can take that into account.
One tool is to use a correlation process where several managers review the appraisals each one has written.
If you have an environment of trust, groups of managers can discuss the objective observations about an individual without getting defensive. In this open discussion one particular manager’s biases can become more visible. This practice reduces the problem of favoritism and enhances the level of trust in an organization.
Another area where we struggle to be objective is when thinking about political issues. We are bombarded by information that is presented with strong biases already marinated in.
Most of us prefer to listen to the “news” that is slanted in the direction we habitually lean. That gives us a kind of affirmation that our biases are valid.
Just for fun I often listen to the news on a network known for having the opposite bias to my own. It is a kind of jarring exercise as I quickly see the how their biases are strikingly “wrong” only to realize that it could be mine that are so far off base.
One thing is for sure, when interpreting political forces, there is no such thing as objectivity.
Your opinion is a very personal thing, and the good news is that you can never get your opinion wrong. The bad news is that your opinion will never be totally objective, so factor that conundrum into your decisions and relationships with other people.
One tool to do this is to take off the “I AM RIGHT” button you wear every day and replace it with a button that says, “I have an opinion on that – what’s yours?”
Does your strategic plan clarify or complicate? When organizations do a strategic plan, a bunch of specific words are used to describe the various pieces, but you would be surprised how those words are often used incorrectly. Many people hate to work on strategy because it is either eternal or terrifying, like going to the dentist. This problem fascinates me, because I do a lot of strategic work with corporations, not-for-profit groups, and educational institutions. I also teach strategic thinking at two universities. One cure for confusing strategic plans is to use the jargon correctly.
For example, it is common to have the mission and vision statements mixed. I have written about that problem and given some typical examples in another article entitled “Mission and Vision Essentials.” Another common sticking point is getting the strategy separated from the tactics. Strategy is the overarching way you are going to move from the current situation to the vision, and tactics are the detailed actions you will take to accomplish the strategy.
Most facilitators have an order they prefer when helping groups with strategy. I believe it is not essential to have a rigid pattern, but I generally prefer to start out with the values. Reason: Values are a kind of foundation upon which the other elements rest. To me, putting values late in the process feels like digging the foundation after the house is already constructed.
A key element in most strategic work is a SWOT analysis. SWOT stands for Strengths, Weaknesses, Opportunities, and Threats. This work is the basis for determining an intelligent strategy. It consists of two views of the organization. In the first view, we look through a “microscope” at the internal strengths and weaknesses of the current organization. The second view is looking through a “telescope” at the environment around the organization to determine the opportunities and threats.
The SWOT analysis can be a time consuming and very boring process. It does not need to be. Facilitators can move through this exercise by breaking up a large group into four subgroups for the exercise. The use of creative techniques, like giving a prize for the most novel idea, can keep the atmosphere light. Of course, like in any brainstorm activity, it is essential to have a “safe” environment where the ideas are just captured, but not critiqued during the session.
One technique that I like to use is a “two wave” approach to the ideas. Let’s suppose we just completed a 10 minute discussion of the “Opportunities” part of SWOT. I then will say something like this, “That is a really great list of opportunities. We could stop here, but I want to challenge the group.
Most of these ideas came quickly and were from the top of your minds. I am sure there are additional creative and dynamite ideas still lurking in the corners of your brains. Let’s take another 10 minutes and see if we can double the number of opportunities on our list.” That process brings out some highly creative ideas, because all the obvious ones have already been mentioned.
I do not use this technique for all sections of the SWOT, as that would get old. It works best for the opportunities section.
After doing a SWOT, it is possible to identify the overarching strategy and tactics. A mistake made by most organizations is to have too many strategic thrusts in the analysis. The reason for a strategic plan is to focus effort on the vital few activities. If you have 32 high priority strategies, you will have trouble making much progress. I encourage groups to narrow the analysis down to three strategies: perhaps four.
One additional activity that is extremely important, but often left out by groups, is to document the behaviors we expect of team members. Without specific behaviors stated in advance, it is difficult to hold people accountable for doing them.
I use a story to illustrate what the jargon on a strategic plan means. Sometimes this helps groups focus on the work and not get muddled up in the terminology. Here is a typical story I use for that.
I liken the strategic process to taking a trip. I want to go from New York City to Toronto. My mission is to have a safe and enjoyable trip. I am considerate and make sure people on the other end are aware of my plans (values). Reaching Toronto is my vision; I can see the skyline in my mind.
I now look at my resources: my late model car is a strength; the fact that the tires are almost bald is a weakness. I see on the map there are some excellent highways (opportunities) but also there is some potential bad weather on the way (threats). I need to select the route and timing wisely.
I decide which day to leave and the route to take (my strategy). The plan is to stay in Toronto three nights, because I have two days worth of business to conduct. My goal is to drive there in 10 hours. I know it is not possible to get there in 9 hours, and I am willing to accept up to 12 hours if there is construction or other delays. There are contingency plans associated with potential problems.
Then I figure out what things to pack, decide what time to leave, and buy two new tires (tactics). I monitor my progress and determine my gas mileage along the way (measures). For example, I know it is necessary to reach Buffalo by 1 pm to make my timing goal. I drive within the speed limit, am courteous to other drivers, and I stop frequently enough to not get over tired (behaviors). I have a very good chance of having a good trip, which was my original mission.
Now if I can only get those SOBs in Toronto to sign my contract, I will be fine. Hold on a minute; maybe that is worth some planning as well. Maybe my vision in the first place should have been more about a signed contract than about seeing the Toronto Skyline. For that, I need to make sure my strategy achieves the true purpose for the trip, and make sure all parts of the plan align with that objective. In this case, I would have been wise to state the vision was to get a signed contract, and the trip to Toronto was one of the strategies. Now my strategic plan would stand a better chance of getting me what I really need.
The process of creating a strategic plan is fairly straightforward, yet many groups get tripped up with all these strange words, and come up with a plan that looks good on paper but does not work well in the real world. That is a colossal waste of time. Make sure you have someone who knows what he or she is doing lead the activities when creating your strategic plan.