The Shelf-Life Of Marketing Data

The MIT Sloan Management Review recently asked Does Data Have a Shelf Life? According to the article:

Creating insights from data is an important, and costly, issue for many companies. They spend time and effort collecting data, cleaning it, and using resources to find something meaningful from it. But what about after the insights have been generated? Do insights have a shelf life? If so, when should knowledge gleaned from old data be refreshed with new data?

[Researchers} suggest that for real-world Knowledge Discovery in Databases (KDD) — applications like customer purchase patterns or public health surveillance — new data is imperative. “It could bring in new knowledge and invalidate part or even all of earlier discovered knowledge. As a result, knowledge discovered using KDD becomes obsolete over time. To support effective decision making, knowledge discovered using KDD needs to be kept current with its dynamic data source.”

What do the researchers suggest companies do? “Model an optimal knowledge refresh policy.”

The author of the article wisely asked the researcher to explain what this is, in lay terms, and was told:

“The model itself aims at deciding when to run KDD to refresh knowledge such that the combined cost of knowledge loss and running KDD is minimized,” wrote Fang in an email. He explained that knowledge loss refers to the phenomenon that knowledge discovered by a previous run of KDD becomes obsolete gradually, as new data are continuously added after the KDD run. Knowledge loss has impacts on several levels: if KDD is run too infrequently, for instance, customers may not respond to promotions that are based on obsolete customer purchase patterns; yet there is a personal cost of managing the KDD process, and there are computation costs of running KDD, regardless of whether it’s run in-house or in the cloud, so running it frequently isn’t the answer.”

My take: I love it when the geniuses at MIT create stuff too complicated for any Harvard MBA to understand.

—————

But seriously…

I can’t speak to data or “knowledge” regarding public health surveillance. I can tell you, though, that the use of the word “surveillance” by the researchers was not the smartest choice of words right now.

I can speak, however, to using data about customer purchase patterns to generate “knowledge.”

—————

Why is “knowledge” in quotes? Because I have no idea what the researchers are talking about when they use the word. Ask marketers what their current “knowledge” regarding customer purchase patterns is, and 999 out of 1000 will say “Huh?” (The other one will cite his firm’s Net Promoter Score).

Using data in marketing doesn’t go through some neat and orderly process (e.g., Data -> Insights -> Knowledge) like some academics would like to think.

Roughly speaking, there are two paths data does go through: 1) Data -> Model -> Action, and 2) Data -> Human Intervention -> Decision.

—————

The first path describes database marketing efforts, where data is input into (and used to develop) a marketing model, and after the model is run, action (contact/no contact) is taken (I could have called this third step “decision”, but it might be worth distinguishing an automated decision from a human decision).

This might sound like a straightforward process, but the number of data elements that go into any model is a messy process, that involves testing, and is subject to a cost/benefit analysis of acquiring the data.

The second path describes the other trillion ways in which marketing decisions get made. It’s messy. Lots of data, some elements more relevant (and/or timely) than others. But lots of human intervention. And lots of iterations.

But nowhere in these paths do marketing decision-makers stop and think about what “knowledge” they’ve gained.

In this context of the first path, the model could be thought of as “knowledge.” Since I don’t know of any marketer who would argue that the relevancy and accuracy of any marketing model is constant over an infinite period of time, you could say that that knowledge has a shelf-life.

Many marketers evaluate the effectiveness of their models at various stages in the life cycle of the model. A well-performing model isn’t likely to get messed with. As a result, a model to predict the shelf-life of the model isn’t something I see too many marketing departments adopting.

In the context of the second path, good luck identifying the “knowledge.”

Marketing practitioners just don’t think in terms of “knowledge.”

—————

What drew my attention to the Sloan article was the title:  Does Data Have a Shelf Life? The article, however. isn’t really about data, it’s about identifying the shelf-life of knowledge.

Too bad, because the shelf-life of data is the more interesting topic.

The question, as stated, however, is a no-brainer. Of course, data has a shelf-life. The challenge isn’t figuring out whether or not data has a shelf-life, it’s figuring out what that shelf-life is. Reducing the problem down to a formula or model just isn’t realistic. Why not? Because of religion and politics.

—————

If you don’t think there’s religion in marketing, you’re a naive fool  There are countless marketers who believe something about marketing that can’t be empirically proved. And if you believe something on faith alone, that’s called religion.

As for the politics of data, assume for a moment that I have data that proves the marketing channel you manage produces superior results compared to other channels. Would you care if that data is three years old? You wouldn’t. But the managers heading up the other channels (looking to increase their budget) would care.

—————

Bottom-line: There’s no question that marketing data has a shelf-life. But determining what that shelf-life is subjective, and I can’t imagine any marketing department relying on a model to figure it out.

As more data sources become available and are used by marketers — and the need to act on that data on a more real-time basis grows — figuring out the shelf-life of marketing data will become a bigger issue for marketers.

It may turn into an advantage for data providers, however. Those that can demonstrate the shelf-life of their data (as well as the shelf-life of competitive data sets) — and successfully defend the determination of that shelf-life — may gain competitive advantages. 

About these ads

5 thoughts on “The Shelf-Life Of Marketing Data

  1. All this talk about shelf-life of data / knowledge goes over my head: Someone from a Top 10 global bank with whom I bank called me c.30 August asking for feedback on the quality of my interaction with the bank’s Phone / Email channel on c. 8 August. I love it when the genius Harvard MBAs at banks devise data gathering programs that are too complicated for someone from IIT like me to understand.

  2. Without wanting to get too existential, it comes down to whether you believe “truth” is digital (binary) or analogue. I tend towards the latter. You can model the world in binary, but it takes a ton of effort and human design to do so. Just look at video games, you get ever closer to something that represents the world we live in, but isn’t the same.

    I’d argue data models are the same. They’re a construct built with a lot of human intervention, now being re-branded as “data science”, which is a bit of a misnomer (science being the search for self evident truths cannot should not be manipulated to sell stuff)…

    Human driven insights are like art that isn’t computer aided (or sometimes is). More like a classic Disney movie, it tells a great story and can change your world view… but it might not actually be *true*.

    So the video game and the cartoon aren’t real, what is?

    I argue there is no such thing as reality, only what works. Data that is no longer useful has expired because it does not work. As soon as I move cities, much of my local purchasing data is now inert.

    Machine learning is the interesting 3rd way to try and model the world, but machine learning in any digital machine is always going to disappoint. Quantum computing is 20 years away from practical application, and even then will require massive human intervention to drive modelling.

    So for the foreseeable future, data modelling or producing human driven insights will be an art, and like all art it takes time to master and produce quality. There is no box you can wheel into your data center that will suddenly drive more transactions. You have to have humans in there, learning by experience either tweaking models or telling stories.

    • ST: I think I completely agree. I say “I think” because I’m not sure I understand the digital/analog distinction, or what machine learning can or can’t do.

      But I think you’re taking the blog post to a whole new level by insinuating that there is no such thing as “knowledge” in the first place. That the concept of “knowledge” is purely subjective. And therefore, that any so-called “shelf-life of knowledge” (let alone the shelf-life of the data that drives the so-called knowledge) is an workable concept because something that doesn’t exist can’t have a shelf-life.

      The undercurrent of this is something I see all the time: An attempt on the part of academics to turn some abstract, highly academic research finding into something that practitioners can or should do. Problem is, most of us practitioners aren’t nearly smart enough to understand what the hell the academics are talking about, let alone to implement it.

      rs

      • Damn, I was kinda hoping the video game, cartoon metaphor made sense. It did in my head (not sure what that says about my head?!)

        Ultimately there’s what works, and what works comes from experience and spotting patterns. Like any art form, it’s a skill that can be learned, whilst others are gifted and just get it. Hence the value put on consultants by businesses.

        It’s my experience that most businesses have such a poor grasp of the facts of what drives their business, and such a lack of people skilled turning that into a story that drives a decision… that the idea a data model you buy in from a 3rd party or academic will change that is crazy.

        Academia is not solving real world issues, it’s pushing boundaries, but most businesses are a billion miles away from that boundary!!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s