Is data killing off the art of the marketing prediction?
Opinion At the start of the year, I was approached by several publications and ask to write articles sharing my predictions for the year ahead. I am sure you are familiar with the format; you will see the "top 5 x of y for 2017" appearing online well into mid-March. While it can be fun to make guesses at what might happen, I believe it is becoming a pointless exercise. The pace of change in modern society is increasing, being driven by ever faster technology innovation. If you get a prediction right these days, it was probably a lucky guess rather than a sage like prophecy.
Data is everywhere
Successful businesses no longer rely on guesswork. The days of the HiPPO (highest paid person's opinion) are over. There are so many great examples of how data is delivering efficiency, across all sectors and industries, that it is no wonder our thirst for its use is growing. Statistics on how much data is created each day are mind-blowing and the numbers continue to increase. Why is there so much new data? Mainly because we are getting better at collecting and categorising it. In the past, we had to neatly align data in spreadsheets or databases (structured) to make use of it.
With today's technology, we are able to analyse almost everything from photos and videos, to sound and text (unstructured). As we are more active online, for example sending emails or sharing content on social media, we generate huge data streams. What we often forget is the hidden data, which might be collected when we make a payment transaction (where we were, how much we spent and what we bought) or use any of the smartphone apps we have (that's right, they pretty much all track your location) that is valuable data which can be turned into insights.
Are data predictions always right?
When people criticise data use, they might point to the political elections in 2016. It’s a great example. If we are able to predict what is going to happen so accurately, how come no-one saw Brexit, or the election of Donald Trump, coming?
The simple answer to this is that we didn't actually want to find out. Polling is very different from accurately predicting. There are several reasons for this. Primarily, a poll only takes a small cross-section of the population. It is meant to be a slice of the electorate and it is usually a very broad slice.
If we are able to predict what is going to happen so accurately, how come no-one saw Brexit, or the election of Donald Trump, coming?
If you wanted to predict the results with accuracy, you could do it, but you would have to spend much more money and ask a far larger sample. Polling companies are not interested in this, if they told us at the start of the process what the outcome was going to be and they were right every time, it wouldn't make for a compelling contest.
News organisations want to make an election interesting, they want to have things to talk about. If you know the ending, then the story becomes less interesting. Polling companies also have an agenda; they need to sell their data, they want it to change. Prominent data scientists could take all the fun out of an election build up, but it is not in anyone's interest.
Data for social good
Taking a moment away from the controversial world of politics, it's worth taking a look at how data is improving our lives. Programs of "data for social good" are helping us build more efficient cities and spending resource in more effective ways. A good example of this is how some cities are addressing the problems of bursting water pipes. This is a much bigger problem than you'd typically imagine (unless you have been the unfortunate victim of a burst water main).
An average sized US city will typically have around 300 significant pipe bursts every year, nearly one per day. If you could predict which ones were going to fail, you could proactively repair the and save the damage and water loss. This is what is happening, but not only that, when a predictive failure is identified, city planners are also reviewing which other services can be repaired or installed while the roads are being dug up. For example, can they lay new fibre cable for high-speed internet at the same time? The advantages are clear and the benefits delivered are in terms of time, money and efficiency. Data is playing a big part in our future but often in ways which we aren’t always as visible.
Data analysis underpins our future
Data collection and analysis are not going away. They are helping us advance our society in so many ways that companies, cities and governments are investing heavily in skills, storage and management. In 2016, the Harvard Business Review named data scientist as "the sexiest job of the year”. But what makes it relevant to marketing? Data helps us tell a compelling story, it helps us understand what our customers want, what they feel and ultimately helps us predict how they will react. It is important to remember that data predictions are only as accurate as we want them to be.