Market Research is alive and kicking! – A Rebuttal to the RIP: MARKET RESEARCH blog

Much has been made about the imminent demise of market research thanks to the emergence of big data over the last 10 to 15 years. Indeed, big data is a hot topic for the business world at present; with everyone salivating at the thought of greater profits from the insights that big data can provide. However, I am doubtful that big data can really live up to the promise to cure all the business world’s revenue and marketing ills.

Like anything, big data has its usefulness but also has its limitations. It is these limitations that will ensure that market research continues to have its place within business strategy. Accordingly, those voices that have heralded the death of market research are akin to newspapers that prematurely print an obituary for a famous person that is still alive and kicking.

The limitations of big data

  1. How much is too much data?

Having access to data is not new for businesses; the difference is that there is now a great deal of it. Having such vast data is only useful if a business has the expertise to makes sense form all that data. If a business cannot effectively mine the ‘diamonds’ from the vast amount of data then they risk making poor decisions or failing to make timely decisions, both of which could potentially hand competitors the advantage.

  1. When significant results are not significant

Big data, by its definition, involves very large data sets. Statistical theory dictates that to increase statistical power, and limit the occurrence of Type II errors (i.e. a false negative), one must aim for large sample sizes when conducting research. Big data aligns to this theory very well, however a limitation with very large data sets is that every small difference turns out to meet the statistical significance definition (p=<0.95). This can create problems for businesses in terms of what they respond to and what decisions they make in response to so many significant results.

Market research has a clear role to play in assisting business to understand what is really significant and what they should respond to as opposed to what significant results may not be relevant for their business.

  1. False positives

Another limitation with big data is its potential to link two outcomes that have absolutely nothing to do with each other. An example of this is provided in a blog by Allen Fromen, entitled ‘The Promise and Perils of Big Data’ (source greenbookblog.org) in which he highlights the near perfect correlation between margarine consumption and divorce rate in the American state of Maine. This example illustrates the potential for a business to identify a relationship between two parameters from their data that are not actually related to each other in any way. Making business decisions based on false positive relationships could have catastrophic effects for a business.  

Traditional market research has a clear role to play in better understanding the relationships identified within big data.

  1. Big data cannot tell you why

Big data is effective in telling a business that something occurred, but is unable to tell us why it occurred. A good example of this was provided in an article by Jeff Fraser, in which he recounts a story where data analysis identified a huge increase in drink sales (Marketing Magazine). Looking at the data alone could have led the business to think that the increase in drink sales was due to an advertising campaign or a change in preference of consumers. However, through the use of market research the reason for the spike in drink sales was identified to be due to a proposed strike by liquor store workers and that consumers were responding to this by stocking up on drinks. This is a great example of how bid data can identify that something has occurred but market research is needed to tell a business WHY something has occurred.

Perhaps this limitation is best explained by watching the short you tube clip of Beth Schneider, Director, Corporate Customer Market Insights, INTUIT.

https://www.youtube.com/watch?v=H-FsyW3l9Kg

  1. Bid data; big assumptions

The last limitation of big data is maybe its biggest and one that has played out many times in human history when we think we might have found the magic bullet. This limitation has to do with the assumptions that we have made of big data. As noted by Steve Needel in his 2012 blog entitled ‘Confessions of a Big Data Blasphemer: What If Big Data Doesn’t Work?’ (Source greenbookblog.org), it is assumed that big data:

  • Has sufficient data to predict a person’s behaviour with some level of statistical rigor;
  • People’s behaviour is sufficiently consistent and that the determinants of that behaviour will be consistent across individuals of some cohort;
  • That it is possible to build a model of that behaviour and use it for prediction.
  • Is able to determine the casual factors involved in people’s decisions;

These are big assumptions to make when businesses are making key strategic decisions.

Perhaps why market research is still relevant in the age of big data is best summed by Allen Fromen, in his 2014 blog entitled “Why big data will never replace market research” (source greenbookblog.org) when he said that “My point is Big Data is no panacea. It can tell us what has happened in the past, and perhaps infer future events, but it has limited ability to explain WHY something has happened. Without understanding the WHY, Big Data is not particularly actionable.”

Market research provides the WHY for businesses and this is why it is and will continue to be alive and kicking!!

Do you think Market Research still has a future?

Do you like market researchers calling you at dinnertime or would you prefer businesses simply predicted what you want from their data?

Advertisements

14 thoughts on “Market Research is alive and kicking! – A Rebuttal to the RIP: MARKET RESEARCH blog

  1. Great article!

    Those ‘unrelated correlations’ are SO interesting! I think you are spot on, the potential is there – but the data isn’t so useful just yet. I think traditional market research has a future for now, but possibly not for long.

    I definitely do not like market researchers calling at dinner time, but it is such an archaic method that has NO place in this day and age. I assume that method started long before the internet and was one of the only ways of reaching out to customers. I am more likely to answer an email survey, than not hang up on a telemarketer at 6pm. So far, i don’t mind companies predicting my needs based on historical data i have provided, but i don’t know how successful it is… it may very well be working on me and i don’t even know it yet!

    Like

    • HI Enewington, Thanks for you reply. I don’t like the telemarketing calls either, to be fair at any time. You mentioned that market research might have a future at the moment, but maybe not for long. Do you think that data will have the capability to anticipate human behaviour and decisions at sometime in the future to make engaging with humans irrelevant?

      Like

  2. First of all… Great work on the post!!!
    Secondly, Yes, is the answer the answer to all of the questions above mentioned in the post. Big Data revolves and plays around the set of information provided, but what about customer needs, that still has to be collected from the customer itself. This is where Market Research or the researchers involved come into play. For example, a small scale pharmaceutical industry working on its new drug’s effects, cannot just rely on Big Data, but has to get actual insights and views from its consumers. People have got into a wrong notion of just following Big Data because its new, i would not say it is completely driving us to the wrong path, but a decision without market research would just be a detour followed by an assumptive destination.

    Like

    • Thanks very much for your support Ajayvinayan. It is an interesting question about whether big data will replace the need to interact directly with humans to gain more insight into their behaviour and decisions. In some respects, what we are contemplating in the two blogs is the development of artificial intelligence that has the capability to accurate anticipate and predict human behaviour and decision making. I am not sure that I feel totally comfortable with that potential, which is why I think and would like market research to always be relevant.

      Like

  3. Leave me along to eat my dinner. Happy for researchers to do what they want with the big data if they leave me along at dinner time. Thanks for the blog – clearly some research behind the analysis. What came to mind in reading your blog was context – the example of the analysis in the increase in liquor sales is a good one. The data itself is only useful if it has context and I am yet to be convinced that big data as we know it can help us there.

    I also note with interest the use of big data to enable mathematical predictions of consumer (not human) behaviour. The data is mostly from the consuming habits of people or which is only a part of human interaction and decision making. What can big data tells us about issues to guide public policy or voting preferences or similar. These lines of inquiry to my mind still require the application of other market research techniques.

    Finally is the use of big data to make predicative models of the future limited in relying on past to predict the future. Most of the big data collected is past behaviour – market research techniques still retain the ability to gauge future intentions and what may motivate a specific kind of change. While I can see big data providing some information – ‘old fashioned’ market research may be with us for awhile yet.

    Like

    • Hi Sharpym, thanks for your thoughts in response to this blog and the RIP Market Research blog. I really like your comment that Big Data uses past behaviour to predict future behaviour, whereas traditional market research is about asking people what their future behaviour would be without any reference to their past behaviour. It would be interesting to understand which method gave the more accurate results. What do you think?

      Like

  4. Was wondering when we gather data on large scale through quantitative or qualitative techniques, is the data usefull to find out relevant information for niche businesses? because on large scale the research data will be showing the data on large scale and the results will be blurred.
    With regard to unrelated co-relations it is very common to see such co-relations, in one of my units Global trade and Markets, we were shown such data to be cautious of such irrevelant co-relations, I have found some examples of such co-relations to enhance the understanding:
    example 1. US spending on science, space and technology was highly corelated with suicides by hanging, strangulation and suffocation.
    Example2. Number of people who drowned by falling into swimming pool is highly corelated with Number of films nicholas cage appeared in.
    ref: http://www.tylervigen.com/ more such spurious co-relations can be found here.
    Analyst should be aware of such funny co-relations.

    Like

    • Hi Davin306, I agree with your points above. I work for an insurance company which has a lot of data that we regularly analyse to determine how we can improve our service to our customers. Over the last couple of years, we are going directly to our customers more and more to engage them in the development of new services.

      Liked by 1 person

  5. Great post, and some very interesting points about the future role of market research vis-a-vis the emergence of Big Data as a source of consumer information.
    The point I found particularly relevant was at the end of your post, where Allen Fromen says ‘…Big Data is no panacea.’ So often the emergence of a new technology, system or process is seen as a ‘silver bullet’ to solve shortcoming in the existing way of doing things. Instead, it is more realistic to view the role of Big Data as another tool in a marketers armoury.
    As William Pink of Milward Brown identifies in his blog post ‘How Big Data Liberates Research’ (http://www.millwardbrown.com/docs/default-source/insight-documents/points-of-view/Millward_Brown_POV_How_Big_Data_Liberates_Research.pdf), the key point is not how much data can be collected and analysed – rather how well business’ and researchers can answer questions.
    Big Data is merely another source of information (albeit a very rich source that can be passively collected). The challenge for many businesses will be how to take this source of data and make it meaningful and relevant for strategic decision making. A cost of collecting massive data sets is the ‘noise’ or junk data that accompanies the valuable metrics. Primary research will continue to have an important role in providing focus and extension to Big Data analysis.
    Big Data may allow us to do away with certain aspects of traditional research methodology, but as you have identified, its own shortcomings for predicting behaviour are likely to see it complement rather than replace the role of market research.

    Like

    • Hi Jasonchuck, could not agree more with what you have said above. I think it is human nature to get excited about the next new big shiny thing, in this case big data, but to suggest that all other forms of research are redundant seems very premature to me.

      Like

  6. i think market research will always have a future as we know data doesn’t speak for itself. data can tell the sales increased last year but why? data cannot answer this question own its own. sales increased last year and we want to repeat it but what should we do to make sure it happens? these things are answered only by market research. i think we should consider data as well as market research.

    Liked by 1 person

  7. Great post! I too think market research will continue to have a role into the future. Data is only as good as the use it is put toward – and it takes some skill to apply the statistical tests, evaluate the responses to ensure there are no spurious correlations, and to understand the limitations of the data and the assumptions taken in the analysis. I also agree with Davin306 that big data may not be as useful for targeting the niche markets – which in themselves tend to be a little more out there on their own.

    The other thing that would be difficult for big data to be able to do is to emulate the human element of marketing. Earlier in the Consumer Behaviour module we heard how marketing can appeal to those human needs for eg. a sense of belonging or safety as in Maslow’s Hierarchy of Needs. I would think this would require a company to be more “in-touch” with their potential market to understand these human needs – I am not so sure you would necessarily get this level of understanding of the market just through access to big data.

    Re your second question – I have caller-ID on my phone and any number I don’t recognise or can’t see I don’t pick up – consequently they end up hanging up 🙂 I’m confident they will get their market research elsewhere – just not during my time.

    Like

    • Thanks Lynette for your thoughts. I think the truth lies in that big data and market research are complimentary and can work well together rather than having to be considered mutually exclusive. Wise move with the caller ID strategy.

      Like

  8. @danmclair Pretty good thinking on working out Big Data and Market Research to be on the same side. Clearly both of them have their own perks. The future would tell what people will chose, the resource effective Big Data, the cost effective market research or rather a combination of both.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s