W. Edwards Deming had a lot to say about how managers use data incorrectly and waste the resources of an organization. It was part of his philosophy of quality which he called “profound knowledge.” He stressed a number of mistakes typically made by managers when handling data. Here are some of the problems along with the antidote for each misuse.
Mistake 1 – Assuming variation is a result of special cause variability when it is really due to common cause variability.
Common cause variability is when a system is in statistical control with small random type variation going on. The only way to tell if a system is in control is to consider all the data, usually by plotting it, and find out if the data variability is within certain defined bounds. If it is, then for managers to ask people to explain the variation is simply a waste of time. People will dutifully go off and try to find out what caused the variation, but the answer will be only a guess and not valid information.
When one or more data points go outside the bounds of normal variability, then there is a special cause. In these cases it is not only possible but vital to determine what caused the variation so it can be controlled and eliminated in the future.
Most managers fail to determine if a signal is due to special cause variation when they ask underlings to explain what happened. This causes a large waste of effort and time.
Mistake 2 – Assessing the capability of a process based on the most recent data point.
It is tempting to react to the most recent data and ask people to take corrective action based on that. At home, we might say, it’s cold in here, why not turn up the heat? But just because it is cold at the moment does not mean the system needs to be adjusted. It may be the low point of the cycle that is in common cause variation. In which case, if we turn up the thermostat, we are doing what Deming called “tampering.” Tampering is defined as moving the set point of a system experiencing common cause variation in an attempt to reduce the variability. In fact, it can be demonstrated that “chasing” the perfect setting will result in a large increase in the variation of the process. It is better to leave things alone.
Many of us have experienced this when sitting in a meeting. All of a sudden someone will say, “Whew, it is very warm in here” and turn down the thermostat. Ten minutes later people in the room are reaching for their sweaters because they are chilled, so up goes the thermostat. All day long people fiddle with the darned thermostat and swear at the heating system. The problem resides in the fingers of the people playing with the setting, not the furnace control. They are tampering, which results in roughly double the temperature variation than if they just left things alone.
Mistake 3 – Interpreting two points as a trend
This flaw is ingrained so deeply into the fabric of our thinking that we rarely even realize how stupid most statements of movement really are. Every day we read in the paper or hear on the news something like the earnings for Company X are up by 20%. We think that is a good thing. Rubbish! All it means is that in comparison to four quarters ago the earnings are 20% higher. It says nothing about the actual trend of the data. For knowledge of how the company is doing we need to plot the data and consider the quarterly earnings over something like 8 consecutive quarters. Then we can know what is going on.
Many advertisements for products are based on the faulty logic that two points make a trend. When we hear that interest rates on mortgages is down by ½ point, that is a symptom of two points equaling a trend. We really cannot use that data to imply what has been happening to interest rates in the past or is likely to happen in the future.
Mistake 4 – Looking for blame rather than root cause
When something goes wrong, managers often focus on who messed up and why rather than what aspect of the system was the root cause so it can be fixed. They think if they can pinpoint the culprit and punish him or her that will eliminate problems in the future. Actually, the reverse is true. By trying to find a scape goat, people tend to hide the truth and work to pin blame on other people to protect their own interests. That leads to infighting and other disruptive behavior.
Mistake 5 – Too much automation of process data.
This issue is counter intuitive. One would think that data plotted and interpreted by computers would be superior to that plotted by hand. In fact, data where people have been involved in the process is more useful because people have the ability to spot peripheral issues and correct them where a computer will just keep logging rubbish. When people rely on the machine always being right, there can be disastrous results because at the root of it the machines are controlled by people, but once programmed, people tend to rely too much on the machine and forget to check for sanity. That is how pilots occasionally fly into the side of a mountain, because they rely too much on the dumb auto pilot and forget to watch where they are going.
These 5 mistakes are the most common ones. There are other symptoms of how managers use data incorrectly to the detriment of their organization and the people. The antidote for each of these problems is to make sure managers are educated on these flaws and modify their behaviors to avoid the pitfalls.
Great post Robert.
We have conditioned ourselves to think in terms of special causes being only negative factors. At times, they can be positive – something to build into the system, not eliminate. Either way, we must learn from special causes
Reblogged this on News & Notes on LEADERSHIP for LEARNING.