It is entertaining and discouraging to watch the media’s reaction to events. Entertaining in that they react to common cause as if it were special and special cause as if it were common. Discouraging in that people and organizations react in the same way.
Common cause are events, ocurrences and points of reference that are considered normal variation. Special cause are unusual events that don’t fall into the category of common cause. The reaction to common vs. special cause are different otherwise we increase more of each and that is costly.
Lets say you drive the same road to work every day and on average it takes you 30 minutes to get to work. Then one day it takes you one hour to get to work because of an accident on the road. The common cause is normal traffic flows. The special cause is abnormal traffic flows as a result of an accident. Would you permanently change the route you drive to work because of one accident that ocurred on one day? Not likely.
Yet every day government and organizations change the rules because they think special cause are common cause and vis versa. In doing so they simply add cost and reduce productivity because of an over reaction to special causes of variation.
Now Big Data Will Create More Reactions
The dialog around Big Data is growing and the reactions vary as expected.
Because of the internet more data has been created in the last ten years then since the beginning of time. The quest to create meaning from the data has increased the demand for minds that understand data. Consider:
- 30 Billion pieces of content shared on Facebook every month
- 40% projected growth in global data generated per year vs. 5% growth in global IT spending
- 235 terabytes data collected by the US Library of Congress by April 2011
- 15 out of 17 sectors in the United States have more data stored per company than the US Library of Congress
- $300 Billion potential annual value to US health care—more than double the total annual health care spending in Spain
- 60% potential increase in retailers’ operating margins possible with big data
- 140,000–190,000 more deep analytical talent positions needed in the US today
- 1.5 million more data-savvy managers needed in the United States by 2015
There is an old saying “Data lies and liars use data“. Data can be manipulated to mean anything but not everything has meaningful value. To gain meaningful value from data there must be a purpose. The purpose usually is to improve value. Value comes from reducing the friction created by common causes and to understand and reduce the reactions greated by special cause. Otherwise all this BIG DATA will only create BIGGER PROBLEMS.