Much has been made about the imminent demise of market research thanks to the emergence of big data over the last 10 to 15 years. Indeed, big data is a hot topic for the business world at present; with everyone salivating at the thought of greater profits from the insights that big data can provide. However, I am doubtful that big data can really live up to the promise to cure all the business world’s revenue and marketing ills.
Like anything, big data has its usefulness but also has its limitations. It is these limitations that will ensure that market research continues to have its place within business strategy. Accordingly, those voices that have heralded the death of market research are akin to newspapers that prematurely print an obituary for a famous person that is still alive and kicking.
The limitations of big data
- How much is too much data?
Having access to data is not new for businesses; the difference is that there is now a great deal of it. Having such vast data is only useful if a business has the expertise to makes sense form all that data. If a business cannot effectively mine the ‘diamonds’ from the vast amount of data then they risk making poor decisions or failing to make timely decisions, both of which could potentially hand competitors the advantage.
- When significant results are not significant
Big data, by its definition, involves very large data sets. Statistical theory dictates that to increase statistical power, and limit the occurrence of Type II errors (i.e. a false negative), one must aim for large sample sizes when conducting research. Big data aligns to this theory very well, however a limitation with very large data sets is that every small difference turns out to meet the statistical significance definition (p=<0.95). This can create problems for businesses in terms of what they respond to and what decisions they make in response to so many significant results.
Market research has a clear role to play in assisting business to understand what is really significant and what they should respond to as opposed to what significant results may not be relevant for their business.
- False positives
Another limitation with big data is its potential to link two outcomes that have absolutely nothing to do with each other. An example of this is provided in a blog by Allen Fromen, entitled ‘The Promise and Perils of Big Data’ (source greenbookblog.org) in which he highlights the near perfect correlation between margarine consumption and divorce rate in the American state of Maine. This example illustrates the potential for a business to identify a relationship between two parameters from their data that are not actually related to each other in any way. Making business decisions based on false positive relationships could have catastrophic effects for a business.
Traditional market research has a clear role to play in better understanding the relationships identified within big data.
- Big data cannot tell you why
Big data is effective in telling a business that something occurred, but is unable to tell us why it occurred. A good example of this was provided in an article by Jeff Fraser, in which he recounts a story where data analysis identified a huge increase in drink sales (Marketing Magazine). Looking at the data alone could have led the business to think that the increase in drink sales was due to an advertising campaign or a change in preference of consumers. However, through the use of market research the reason for the spike in drink sales was identified to be due to a proposed strike by liquor store workers and that consumers were responding to this by stocking up on drinks. This is a great example of how bid data can identify that something has occurred but market research is needed to tell a business WHY something has occurred.
Perhaps this limitation is best explained by watching the short you tube clip of Beth Schneider, Director, Corporate Customer Market Insights, INTUIT.
- Bid data; big assumptions
The last limitation of big data is maybe its biggest and one that has played out many times in human history when we think we might have found the magic bullet. This limitation has to do with the assumptions that we have made of big data. As noted by Steve Needel in his 2012 blog entitled ‘Confessions of a Big Data Blasphemer: What If Big Data Doesn’t Work?’ (Source greenbookblog.org), it is assumed that big data:
- Has sufficient data to predict a person’s behaviour with some level of statistical rigor;
- People’s behaviour is sufficiently consistent and that the determinants of that behaviour will be consistent across individuals of some cohort;
- That it is possible to build a model of that behaviour and use it for prediction.
- Is able to determine the casual factors involved in people’s decisions;
These are big assumptions to make when businesses are making key strategic decisions.
Perhaps why market research is still relevant in the age of big data is best summed by Allen Fromen, in his 2014 blog entitled “Why big data will never replace market research” (source greenbookblog.org) when he said that “My point is Big Data is no panacea. It can tell us what has happened in the past, and perhaps infer future events, but it has limited ability to explain WHY something has happened. Without understanding the WHY, Big Data is not particularly actionable.”
Market research provides the WHY for businesses and this is why it is and will continue to be alive and kicking!!
Do you think Market Research still has a future?
Do you like market researchers calling you at dinnertime or would you prefer businesses simply predicted what you want from their data?