Category Archives: Analytics

Maximum Profit Per Customer And The 80/20 Rule

In a recent blog post, Don Peppers of Peppers and Rogers writes:

“If your customer base has an 80-20 skew, then somewhere in [a] stadium full of 50,000 customers there are about 80 customers who do about 2/3 the amount of business done by the other 49,920 customers put together! So rather than paying to put your ad message in front of all 50,000 customers with a series of stadium-wide initiatives, why wouldn’t you invest a little of your marketing budget first just to find out where in the stadium these 80 people are actually sitting? Then you could hire some folks to go up into the stands, sit down next to each one of these 80 customers and buy them a hotdog and drink. During the conversation that follows, be sure to tell them: “If you can tell us what it takes to make you happier with our brand, that’s what we’ll do for you…” By the end of the game you will have sown up 40% of the market, and the entire process was conducted out of your competitors’ view. This is the real power of direct, interactive marketing. One-to-one marketing. Customer-centric marketing.”

My take: This advice ignores two concepts that elude too many marketers — “maximum profit per customer” and “jobs the product accomplishes.”

—————

I don’t dispute the possibility that 20% of your customers account for 80% of the profit. And I’m willing to loosen my grip on reality long enough to allow for the possibility that maybe all it takes is a “little of your marketing budget” to figure out who those 20% are.

But the idea that you can simply ask those customers “what would it take to make you happier with our brand” and then go out and do it and see profits increase is a couple of steps off the deep end.

—————

Let me give you a real-life example of why the advice in the article is wishful thinking.

Thanks to the breakfast-eating habits of myself and of two of my daughters, the Shevlin household purchases 125+ boxes of Quaker Oat Squares (QOS) on an annual basis. I can’t imagine that there are too many other households in the country crazy enough to do the same. As a result, grant me the delusion that I’m among the 20% elite that accounts for 80% of the product’s profits.

If the product manager for QOS found out who I was, sat down for a chat with me, and asked “what could we do to make you happier with the brand?,” I would probably reply: “Nothing. Now give me a free box — or better yet, ten free boxes — and go away.”

The point is, I’ve probably maxed out on the number of boxes of QOS I’m going to purchase on an annual basis. My reason for not buying more has nothing to do with dissatisfaction or unhappiness. In other words, I have maxed out on my individual profit potential to Quaker.

—————

However, if the product manager asked “when and where do you eat the cereal, and who in your household eats this stuff?,” what s/he might learn is that me and 2 of the 3 princesses eat the stuff (one princess is out of the house already, and the Mrs. doesn’t eat QOS, and never will), and that we eat it for breakfast, snacks, and dips.

But by talking to the other 80% of customers (who Peppers implies marketers shouldn’t focus on, or at least place lower priority on), the product manager might discover that many of them only eat the cereal for breakfast, and never thought about using the cereal for snacks and/or dips. 

In other words, there are  other “jobs” for which his/her product can fill.

—————

Here’s a scenario for the profit impact that focusing on the 80% (and not the 20%) and on “jobs to accomplish” (and not “happiness”) might have:

         Before: After:
Customer Profits Profits
1        $1      $15
2        $1      $1
3        $1      $1
4        $4      $15
5        $4      $4
6        $4      $4
7        $8      $8
8        $8      $8
9        $30     $30
10       $30     $30
TOTAL    $91     $116

In the “Before” scenario, 20% of customers — two “elite” customers (#9 and #10) — generate $30 of profit per year, and account for 2/3 of all profits. If these two customers have maxed out on their profit potential, however, no amount of marketing effort will produce incremental revenue or profits (in fact, additional marketing to this segment would end up reducing their profitability).

If, however, by identifying new “jobs to be accomplished” with the non-elite 80%, two customers’ profitability could be raised to half of that of the elite customers, profitability could grow by ~30% (it’s dependent on which customers’ profitability increases).

—————

The big variable, of course, is how much would take in marketing investment to identify the gap (in jobs accomplished) and change behavior.

If we can assume that it would take “little of the marketing budget” to identify the 20% elite, would it really cost that much more to survey a sample of the other 80% and find out how they differ from the elite group in their use of the product?

—————

There are two key takeaways here:

1) Allocating marketing dollars to customers whose profit levels have maxed out isn’t just a waste of effort, it erodes profitability. High profit customers may be in this category because previous marketing efforts have been successful. Maintaining high levels of marketing investment may have diminishing — or no -returns on investment. 

2) Happiness (or satisfaction, or GOD FORBID, likelihood to refer) is a silly thing to focus on if your objective is to grow revenue and profits. Attitudinal measures may help you understand “how you’re doing” but they’re not that good for helping you determine “what to do.”

Data Storyology

It’s conventional wisdom by now that, with all the data we have to analyze, we have to find the “story.” Experts like Tufte have done wonders to improve our capabilities regarding data visualization and presentation — but that’s different from the understanding the story that the data is telling.

A recent HBR blog post titled How to Tell a Story with Data offers the following points of advice: 1) Find the compelling narrative; 2) Think about your audience; 3) Be objective and offer balance; 4) Don’t censor; and 5) Edit, edit, edit.

My take: My points of advice differ. And I think we more rigor (dare I say methodology) regarding data storytelling.

—————

I don’t have an issue with “find the compelling narrative” and “think about your audience” but these points are actually part of a broader process that the article doesn’t define.

Think of data storyology — the art and science of telling stories with data — as having two broad components: 1) Finding the story in the data, and 2) Telling the data story.

If I were to draw a picture, it would look like a yin/yang diagram, not a flow.

Finding the story in the data is an iterative process that involves utilizing data management and statistical tools to cut and analyze data. But in also involves applying human judgment and experience to figure out what the “story” is.

The HBR blog author describes “finding the compelling narrative” as:

“Giving an account of the facts and establishing the connections between them. The narrative has a hook, momentum, or a captivating purpose. Finding the narrative structure will help you decide whether you actually have a story to tell.”

I wish he would have left that last sentence off. If you find a narrative structure, you have a story. Whether or not that story is worth telling is a different issue.

Finding the narrative structure is more than “giving an account of the facts and establishing a connection,” however. In fact, the “account of the facts” is probably the least important part of the story because it’s the part that many people either already know or think that they know.

The interesting part of the narrative is the why, who, and when (more so than the what).  The “what” is the plot, but the “why” is what gives the plot some depth. And just as poor character development in a book diminishes the quality of the book, leaving out the “who” in a data story produces an incomplete (and potentially boring) story.

—————

Finding the story is just the Yin part of the equation. Telling the story is the Yang.

This is where the “think about the audience” part comes in. Good data storyologists (or data artists) often define or uncover multiple stories in the data. Those stories likely have different levels of appeal to different audiences. Telling the story starts with defining who the audience is for the data story, and which of the data stories that were defined is most relevant, or how those stories tie together.

—————

At this point, however, my opinions veer from the blog author’s.

Telling the data story is anything BUT being objective and balanced. Data storyology is about educating, influencing, and motivating people. As a data artist, the last thing you want to do is be objective and balanced. You want to draw upon your insights, opinions, and experience — which are all subjective — to tell the best story. The article says that “a visualization should be devoid of bias.” Perhaps a point for future discussion, but I think that this is simply impossible.

The article also says that “Balance can come from alternative representations (multiple clustering’s; confidence intervals instead of lines; changing timelines; alternative color palettes and assignments; variable scaling) of the data in the same visualization.”

First off, this is a very narrow interpretation of “balance,” in that relates to just visualization. Data storyology is about more than just data visualization. Visualization is not the story.

In addition, I would encourage any budding data storyologist to “censor like hell.” The absence of censorship equals data dump.

—————

With a story and an intended audience, there’s still the art of telling the story.

A number of years ago, the analyst firm I worked for brought someone to train us on the art of storytelling. Still one of the best training sessions I’ve ever had.

The story trainer told us to think about the development of a story in terms of the story’s impact on the audience’s mood, and to strive to achieve the following mood pattern:

To summarize, think of the story development as: 1) Stuff is happening (neutral mood), 2) Things are going to get worse (or the things that are happening will cause problems, doom, despair) 3) Stuff happens or will happen to make it all better.

Story example: Little red riding hood is walking in the woods (#1), she gets captured by the big bad wolf (#2), she gets saved by the Woodsman #3).

Data story example (in financial services): Consumers are fed up with paying the high cost of checking accounts (#1), new providers are coming into the market to steal banks’ customers and drive profitability even lower (#2), banks can deploy new technologies and marketing analytical techniques to provide new forms of value to consumers to retain them and make them more profitable (#3).

—————

All the talk about the rise of data scientists misses the boat, in my book. We need people who can take the data, and not just find the story in the data, but the tell the story in a way that educates, influences, and motivates people. That’s not science — it’s art. It’s data storyology.

Gonna Manage Big Data Like It’s 1999

I started working at Forrester Research in 1997. In retrospect, I think I got hired not because I demonstrated great potential to be an analyst, but because my boss and colleague needed a sucker to join the team and write a report that nobody else wanted to write.

So I joined Forrester and wrote my first report on the hot topic of the day: Knowledge Management (it was a terrible, terrible report).

I interviewed executives from about 50 companies about what they were doing about knowledge management. What I heard was confusing. For the most part, what these companies were doing with IT and data was pretty much what they had been doing for the prior 10 years.

What was different (in 1997), was that now these initiatives were called “knowledge management initiatives.”

There were two key success factors (or barriers) critical to the success of knowledge management initiatives: 1) employees with the right skills in knowledge management, and 2) management support and commitment.

After all, sucking the “knowledge” out of people’s heads and making it available to everyone else in the organization wasn’t easy, and wouldn’t be successful if management didn’t sufficiently invest in it.

—————-

Roll the clock ahead 16 years and you’ll find that nothing has changed. Except the labels.

In a creditunions.com article titled Big Data At A Growing Credit Union, an interview with a credit union executive went like this:

Q: Can you define Big Data?

A: Not really. But in a way, Big Data is what people have been doing all along — looking at and analyzing data. I don’t know the tipping point where a credit union moves from generally looking at data and is suddenly in Big Data.

Q; Can’t data also overwhelm and slow decisions?

A: It can unless you achieve the balance of talent and training. If you put the right data in the wrong hands you can be swimming in that data forever. You’ve got to get people to the point where they understand what’s relevant and what’s not, and that takes time.

Q: What do you feel is critical to success with Big Data?

A: You have to have directors and senior managers who are supportive and understand there are revelations this data can provide.

You’ll pardon me if I can’t help but think that this all sounds vaguely familiar.

—————

If you can’t define Big Data — other than saying it’s what “people have been doing all along” — you are not going to get management support for the efforts.

If you think Big Data represents a different way of managing your business, but you can’t articulate that difference to your employees, you will not get broad employee support for the efforts.

Management is usually willing to fund some initiatives to try something promising. Employees, on the other hand, are generally loathe to change unless the pain of the existing is too much to bear. You might argue that they’re willing to change if the potential upside is appealing enough, but I’m not so sure about that.

Jumping on the management fad bandwagon is a prescription for failure. It trains employees to put everything they want to get funding for under the fad banner, and diminishes whatever potential value really lies in the core of the new concept.

————–

My take: You won’t find anyone more supportive of using data to make business decisions than me. But the path to becoming more data-driven doesn’t mean managing like it’s 1999 and jumping on the fad bandwagon. 

The Wrong Way To Measure The ROI Of Twitter And Vine

In an article titled What Is the Right Way to Measure Your Twitter & Vine Marketing?, HubSpot writes:

“We like to call it closed-loop reporting. Closed-loop reporting on Twitter is the process of tracking the path of a user who clicks on a link in a tweet, visits a page on your website, completes a form on a landing page to become a lead, and, ultimately, converts into a customer — so you can directly attribute customers to your Twitter marketing, and evaluate the effectiveness of Twitter as a marketing channel for your business.”

 My take: Nope. Sorry. Wrong way to measure Twitter/Vine ROI.

Why is this wrong? For one, it ignores how someone came to see the tweet in the first place. Without knowing what other messages/media a prospect has been exposed to, attributing the sale to Twitter is inaccurate.

But there’s something else missing from HubSpot’s methodology. None of the comments on the blog post mentioned this (as of the time I read it), and it’s really too bad that there aren’t more (any!) people calling them out over this. 

The missing element: Cost.

Folks, you cannot — I repeat CANNOT — measure ROI without measuring cost. Cost is the “investment” component of the ROI formula. Sadly, too many social media gurus choose to ignore that.

It’s mind-boggling that the HubSpot makes no mention of capturing the costs involved with using Twitter/Vine as a marketing channel. The cost of sponsored tweets (if used) are easily measured, but allocating the costs of shared (or even dedicated) resources to the channel is no easy matter. 

So what should marketers do? The best answer might be “nothing.”

Accurately measuring the ROI of marketing investments is tricky business. Assume for a moment that you have a $10 million marketing budget, and 40% is in mass media channels, 30% in direct marketing, 25% in various other media/channels, and 4% in social media (excluding Twitter), and 1% in Twitter.

Is your time best spent figuring out the ROI of the 1% or the ROI of the 70% in mass media and direct marketing? Right. 

Back to the ROI drawing board.

The Most Misused Term In Marketing

Compete recently ran article claiming that Mobile Twitter Users Are the Ideal Audience for Advertisers. In it, Compete reports that, compared to other Twitter users,  mobile Twitter users in the U.S. are 86% more likely to be on Twitter several times a day and 57% less likely to use Twitter on a desktop computer.

Compete didn’t stop there. A graphic shows a number of other differences:

My take: The term “more likely” is, in all likelihood (pun intended), inappropriate here. In addition, the term is quite possibly the most misused term in marketing.

It’s possible that my analysis — in THIS case — is wrong, but I know for a fact that it happens in the reporting of many other studies. So, if I’m wrong here, my apologies to Compete. But the explanation will go to show why so many others go wrong.

—————

How did Compete determine that mobile Twitter users are THREE times more likely to use Twitter when commuting?

In all probability (the puns don’t stop, do they), they asked consumers: “Do you use Twitter when commuting?” If 30% of mobile Twitter users and 10% of other Twitter users said “Yes”, then Compete would have concluded that mobile users are “3X more likely to use when commuting.”

If 20% of non-mobile Twitter users use the service at work or school, and and 52% of mobile users do so, then mobile users are “160% more likely to use Twitter at work or school” (32% is 160% of 20%, so added on to 20% equals 52%).

—————

The problem with all this is that none of it actually captures likelihood of doing something.

How much “more likely” are mobile Twitter users to use Twitter on a mobile device than other Twitter users? Since 100% of mobile Twitter users use a mobile device and 0% of other users do, the answer isn’t calculable.

These statistics (and the underlying questions) don’t capture “likelihood.”

If the question had been “How likely are you to check Twitter before going to sleep tonight?” and 100% of one group said “100% chance” and 50% of the second group said “100% chance” then maybe you could say the first group is twice as likely as the second.

But it seems doubtful to me that that’s how the questions were asked.

—————

Regardless of whether or not I’m correct in my interpretation in this instance, there’s no denying that this happens all the  time when marketers report out the results of their studies.

[Hell, I do it myself from time to time, and thankfully I have a colleague who is great at catching it and making me change it.]

Marketers may overuse the terms “disruptive” and “transform” and whatever, but I’m throwing “more likely” into the hat as the most misused term in marketing.

————–

There is another issue with the Compete that bears mentioning, by the way.

It’s bad enough that the term “more likely” is (most likely) being misused.

But the underlying contention that all these “more likely” statistics add to make mobile Twitter users worth focusing on is missing one key element: How big is this segment?

You actually have to click over to the Twitter blog post on this study to find out. There, Twitter says that “60%of our 200 million active users log in via a mobile device at least once every month.”

Hmmm. Those 200 million users span how many different countries? And you, Mr. or Ms. Marketers, are serving consumers in how many of them? 

The key thing to understand here, as a marketer, is what percentage of your customers and prospects are mobile Twitter users, not how many of Twitter’s users are mobile Twitter users. 

It’s conceivable that your customers and prospects that are mobile Twitter users aren’t representative of the total pool of mobile Twitter users. 

————–

So go ahead, Compete, and hate me for tearing apart your research. Join the club.

The Importance Of Disney’s MyMagic+

Disney announced that it plans to introduce something called MyMagic+. The company calls it a “vacation management system,” incorporating rubber bracelets encoded with credit card information. The bracelets would enable alerts to be sent to guests, as well as — you guessed it — offers to buy things, and (very importantly) the ability to pay for things.

My take: Disney is legitimizing the notion that Payments is the new 5th P of marketing. 

—————

To refresh your memory, some time ago, someone came up with the notion that there are 4 Ps of marketing — product, place, price, and promotion. These 4 Ps are the levers that marketers can pull or adjust to influence marketing performance. 

These 4 Ps have survived many changes in the world of marketing. Some folks have tried to introduce new Ps — like “people” or “personality” or “personalization” — but none have stuck (rightfully, so) because they don’t really clear the bar of being a lever that marketers can adjust. People, personality, personalization — and other Ps — are simply not part of the marketing mix.

Last August, I tried to argue that Payments were emerging as the new P in marketing. That, in effect, changing how someone paid for a product or service could influence their choice of product or service in the first place. 

—————

What Disney is doing with its MyMagic+ bracelet is changing the customer experience by changing how the customer pays for something. 

Disney is rumored to be spending $800 million to $1 billion on MyMagic+. 

There are a lot of ways Disney could be spending that money. It could build new rides (Product), it could build a new park somewhere (Place), it could provide $800 million in discounted prices to lure more people to visit (Price), or it could spend that money on advertising (Promotion).

But its not. The company is spending it on changing the way guests pay for things. And by doing so, changing what people buy, and changing the overall customer experience.

—————

What Disney is demonstrating is that payments — as the 5th P of marketing — isn’t an opportunity for just financial services firms and upstarts. 

There’s a saying that politics makes strange bedfellows. The same is true for profits.

The opportunity to increase profits by changing the payment experience is going to create new partnerships for retailers, merchants, travel providers, etc. and financial services firms. 

The problem — from a financial services industry perspective — is threefold. Many financial services marketers:

  1. Lack the marketing sophistication of retailers and merchants.
  2. Are stuck in “interchange fee maximization” mode.
  3. Think their role in payments is “money movement” not “purchase influence.”

The problem — from a retailer/merchant perspective — is twofold. Many retailers/merchants:

  1. Don’t effectively allocate marketing dollars across the existing marketing mix, so adding a new P to the mix causes confusion, not opportunity.
  2. Partnerships with financial services firms have historically delivered little value, and new ventures (e.g. card-linked offers) haven’t proven their mettle yet, so retailers/merchants are disinclined to shift investments to the new P.

—————

Disney’s investment in MyMagic+ shines a spotlight on payments as the 5th P.  You’ll see.

Why Social Media Metrics Blog Posts Are A Waste Of Time

According to an HBR blog post titled Why Your Social Media Metrics Are A Waste Of Time:

“Many companies use the wrong metrics to measure their performance, especially when it comes to social media. If you think pageviews, unique visitors, registered members, conversion rates, email-newsletter open rates, number of Twitter followers, or Facebook likes are important by themselves, you probably have no idea what you’re doing.”

Fair enough. No argument from me. But the article goes on to say:

“Here are four of the most important metrics you can follow — notice how little they have to do with popular social-media metrics: 1) Relevant revenue; 2) Sales volume; 3) Customer retention; and 4) Relevant growth. These metrics are valuable because they measure success at your core business. To measure the value of your social-media activities, you have to look at the results the company is getting overall and track how social media was involved in moving the needle. That’s where you’ll find the only relevant social-media metrics.”

And therein lies the problem with the article (and most blog posts on social media metrics, for that matter). Namely: Preaching without prescription.

Re-read the sentence that says “To measure the value of your social-media activities, you have to look at the results the company is getting overall and track how social media was involved in moving the needle.”

Technically, correct. Practically, useless.

Of the many challenges facing marketers, attribution of results to investments is one of the stickiest. Many marketers design tests to see which offer tests better, or which web page design delivers the highest conversion rate. But tying overall revenue, retention, and growth to social media metrics — in the absence of control groups or other structured testing techniques — is, for the most part, impossible.

Not that that won’t stop marketers from using correlative measures. But it doesn’t prove causation.

Unless you only touch a customer through social media, you can’t claim that social media was the cause of any change in the relationship.

So go ahead, HBR, and publish blog posts about how social media metrics are a waste of time. In the absence of solid prescription and advice on how to tie social media efforts to bottom-line results, those blog posts are a waste of time.

—————

The problem is not that metrics like pageviews, unique visitors, or number of followers are the “wrong metrics” to use. Instead, there are two problems here.

The Metrics Chain

The first is that few (if any) marketers have done the hard work to understand if and how metrics like pageviews, visitors, or followers impact the bottom-line metrics the author of the article wants us to measure.

I know that many of you think the marketing funnel is dead. You’re wrong, and this is not the place or time to tell you why you’re wrong.

We need to think of marketing metrics as following a funnel, from upstream metrics like pageviews and follows to downstream metrics like sales, revenue, and retention.

What about midstream metrics? This is where the concept of customer engagement comes in. What are the things that good customers do — that demonstrate engagement — after they’ve viewed a page, liked your FB page, or visited your site that lead to increased sales and/or retention?

The problem isn’t that upstream metrics are the “wrong” metrics to measure — it’s that they’re an insufficient set of metrics to use to measure performance.

The Cost of Metrics

But that’s just the first of the two issues causing the problem. The second is that it costs money to develop and track a metric.

Some metrics — like pageviews, followers, likes, etc. — are easy and cheap to measure. But developing those midstream metrics, and determining the linkage between upstream, midstream, and downstream metrics are harder to define and measure. And it takes an investment on the part of Marketing to develop and track them.

What’s the ROI on that investment?

There is none. Spending money to develop and track and metric in and of itself will have no impact on the bottom line. You might be able to use that metric to make better decisions that ultimately lead to improved sales or reduced costs, but simply defining and calculating a metric doesn’t produce that result.

So what happens is that Marketing doesn’t invest in a measurement infrastructure. It defaults to tracking the easy-and-cheap-to-measure metrics. The ones the author of the HBR article thinks are the “wrong” metrics.

To Track or Not To Track: That Is The Question

The most important question to address isn’t “what social media metrics should we be tracking?” but “should we even spend time and money developing social media metrics to track?”

Here’s why: Assume that a company’s marketing budget is $100 million, and that 50% of it is spent on TV advertising, 20% on print advertising, 20% on direct mail, 5% on online advertising, 4% on events, and 1% on social media. 

Of the six approaches that marketing invests in, which of the six would you want to have the most accurate marketing ROI metrics?

My top three would be TV, print, and direct mail. Cuz that’s where 90% of the marketing dollars go. 

If the CMO of my fictional company doesn’t have the “right” social media metrics in place, so what? Does it really matter that much? 

—————

Now, I can already hear the SM gurus claiming that social media has a disproportionate impact on customer relationships. 

Maybe that’s true. 

But that just means the CMO of my fictional company needs to know if s/he should be reallocating the investments between the six categories in some other way than it’s being allocated today. 

It doesn’t mean investing more in a social media measurement infrastructure. It means investing more to develop a marketing measurement infrastructure. And that puts many Marketing departments in a chicken-and-egg situation: Can’t prove the ROI of their investments, but can’t afford to invest in a measurement infrastructure that would improve the measurement of marketing ROI. 

—————

Bottom line: Blog posts about social media metrics like the one in HBR tick me off. They miss the important points and issues regarding marketing measurement.

In Defense Of BS Metrics

Mixpanel published a widely covered blog post, titled BS Metrics. According to the post:

“Sadly, we haven’t moved forward over the past decade despite our whole industry becoming smarter about how it measures and analyzes data. Companies still pitch investors with a cumulative user sign up graph, sell advertisers on how many pageviews they get, and bamboozle reporters with the biggest numbers they can find regardless of whether they correlate to success. Companies need to start using a new set of metrics that don’t simply make them feel good. They should use actionable metrics that provide insight, provide guidance, and help businesses make better decisions.”

My take: Good points, can’t argue with that. But BS metrics can still serve a useful purpose. (And if investors, reporters, and advertisers don’t demand better metrics, than don’t blame companies for reporting BS metrics!)

—————

Why do you measure what you measure?

“To know how we did, how we’re doing, and predict how we will do,” you’re likely to say. A great answer. To evaluate and track performance is an excellent objective for a metric.

But “actionable” metrics may have negative side effects.

Revenue per team may be a good, actionable indicator of the performance of the various teams that exist at your company.

But what if one client buys from multiple teams? Let’s say I do such a great job with a client (hey, it could happen) that, as a result of my efforts, the client buys products/services from another team at my company. Their revenue goes up, mine doesn’t, they get credit, and I get screwed.

Eff that metric.

—————

Bottom line:  Different metrics can serve different purposes. Two in particular (and two that serve in defense of BS metrics) are motivation and alignment.

A metric like pageviews may be BS, may not be actionable, and may have no direct connection to bottom line performance, but: 1) If there’s no (or very little) incremental cost to measure it, and 2) It helps get people on the same page (pun intended), then there may be benefits to tracking the metric.

—————

Companies should approach the identification of their metrics as if they were constructing a portfolio.

If you were constructing an investment portfolio, you would likely include investment vehicles with different risk profiles and the such.

Same thing with performance metrics. There is no reason why all your metrics need to be “actionable” metrics.

—————

The author of the Mixpanel blog post writes:

“My experience has shown that companies should start by tracking a single actionable metric that they can literally bet the company on. I call this their One Key Metric (OKM). The best part of OKM is that companies can measure other things related to it, to understand how to improve it. Understanding your OKM often leads to deeper, more valuable questions.”

I’m inclined to agree. The challenge, however, is in defining the right OKM. It’s easier said than done.

But OKM doesn’t mean tracking just one metric. And those other metrics can be BS metrics, as long as you and management know they’re BS.

The Credit Union Cost Per New Member Performance Index

Creditunions.com recently published a couple of blog posts which contained interesting statistics regarding credit unions’ member acquisition costs and membership growth.

I thought it would be interesting to take some of the data and construct a new performance metric for credit unions to use.

—————

The first post ranked states by their aggregate cost per new member, and included the state’s overall membership percentage growth (for the 12 months ending 9/30/2012). Kudos to North Carolina credit unions for only spending $142 to acquire a new member, $300 less than the national average.

The second place state, Alaska, had an acquisition cost per new member $32 larger than NC’s number, but achieved nearly twice the growth, at least in terms of percentage growth.

To see which state really performed best,  I constructed the Credit Union Cost Per New Member Performance Index by dividing the percentage growth by the acquisition cost per new member (a high growth percentage is good, a low cost per member is good, so a really large score is good).

The calculated score produces a number which is kind of meaningless, so the best way to compare results is by indexing the score against the overall US, which grew credit union membership by 2.7% in the time period under question here.

The result is that although NC had the lowest cost per new member ($/NM), three other states (AK, ID, VA) outperformed NC by driving a greater degree of growth out of the money they invested in new member acquisition.

State $/NM  % Chg    Score    Index
NC    $142  3.19%    0.0225    368
AK    $174  6.26%    0.0360    589
ID    $208  5.88%    0.0283    463
VA    $230  6.02%    0.0262    428
MS    $238  3.63%    0.0153    250
TN    $248  4.19%    0.0169    277
UT    $260  4.40%    0.0169    277
OK    $267  4.59%    0.0172    281
WA    $276  5.34%    0.0193    317
NM    $280  4.07%    0.0145    238
US    $442  2.70%    0.0061    100

Source: creditunions.com, Aite Group

—————

The second creditunions.com blog post contained stats on the top 10 states by membership growth. Between this second list and the first, eight states were included on both lists (NC and MS fell off. The data for those states might be available publicly  somewhere, but frankly, I’m way too lazy to try and find it).

By looking at absolute growth, we can calculate what the credit unions in each of these eight states actually invested to acquire new members. Overall in the US, credit unions spent $1.1 billion to acquire 2.5 million new members (#NMs). That works out, with 7,031 credit unions, to be a little more than $157k per credit union.

In Alaska, although the credit unions in that state had the highest Performance Score, they spent the most per credit union to acquire the new members that they did. The 12 CUs in the state invested nearly $7m — roughly $580k per credit union — to acquire the ~40k new members they picked up over the past four quarters.

Virginia, with nearly 180 credit unions, spent almost $100 million to acquire ~434k new members. That’s about $560k per credit union.

Among the eight states, Tennessee spent the least (per credit union) at just $110k per institution.

State  #CUs   #NMs      Avg/CU   $ NM      $NM/CU
AK     12     40,025    3,335    $6.96m    $580.4k
ID     52     32,570    626      $6.77m    $130.3K
VA     178    433,782   2,437    $99.77m   $560.5k
TN     171    75,868    444      $18.82m   $110.0k
UT     82     75,882    925      $19.73m   $240.6k
OK     71     48,306    680      $12.90m   $181.7k
WA     109    149,333   1,370    $41.22m   $378.1k
NM     50     27,841    557      $7.80m    $155.9k
US     7,031  2.5m      356      $1.1b     $157.2k

Source: creditunions.com, Aite Group

—————

What this demonstrates is that, even if your cost per acquisition is towards the low end of the range, you may still under-perform the market if you don’t sufficiently invest in marketing.

—————

Another question that came to mind was: How does all of this compare to banks?

To answer that, I turned to the expert on the topic, Serge Milman from Optirate. Serge pointed me to a number of studies (one of which was my own from a few years back that I had forgotten about) which have estimated banks’ cost of new customer acquisition.

The numbers are generally all over the map. One study puts the number at about $350, but that appears to include both banks and credit unions. Serge also cited a study from Brintech, Cass Bettinger & Associates and Amalfi Consulting from November 2009 which estimated the cost at $143 for online acquisition and $328 for branch acquisition.

—————

So I don’t know what the comparable bank number is for cost per customer acquisition. But if the overall industry is at $350, and credit unions are at $442, then that’s not good.

If it costs 26% more to acquire a new member than what it costs a bank to acquire a new customer now, what’s it going to be when the mistrust and negative sentiment towards big banks fades (and it is going to fade)?

But you should calculate the Cost Per New Member Performance Index for your credit union and see how your CU stacks up.

Are Credit Unions Falling Short Of New Member Expectations?

Consider the two following pieces of news that crossed my desk yesterday:

  1. The December 2012 ACSI Customer Satisfaction scores for financial services revealed that credit unions scored higher than banks (as a whole) for the fifth straight year. However, credit unions’ score dropped from 87 to 82, a 6% drop, from the previous year. This comes was after a seven point gain in 2011 over the 2010 score.
  2. NerdWallet analyzed data published by the NCUA and found that half of all federally insured credit unions experienced an increase in membership from June 2011 to June 2012. According to the NCUA, credit union membership ranks grew by 2.1 million from October 2011 through September 2011.

My take: This raises some interesting questions. What caused the drop in credit unions’ satisfaction ratings in 2012? And, for that matter, what caused the huge jump in 2011?  Why would membership ranks continue to grow in the face of declining satisfaction? Would membership have grown even faster if satisfaction levels had remained at its 2011 level?

I can come up with two competing schools of thought to explain what’s going on:

1. Satisfaction rankings are worthless. Did credit union member service levels fall off a cliff (pun intended) over the past year? Did credit unions look back at their perceived Bank Transfer Day success and say “Great! We’re done. Don’t have to try anymore!”? Doubtful.

The more likely explanation is that the satisfaction scores don’t reflect just members’ satisfaction with their credit unions, but incorporate broader perceptions about banks and the financial services industry as a whole.

There is insufficient statistical history to draw on, but it appears that there may be an inverse relationship between CU and banks’ scores.

In 2009, CUs and banks’ satisfaction scores remained unchanged from their 2008 levels. In 2010, as banks went up a point, CUs went down four points. In 2011, however, banks declined a point, and CUs wen up seven points. This year, banks went up two points, and CUs lost five points.

The drop in credit unions’ scores in 2012 make no sense. As banks were eliminating free checking and imposing fees, credit unions held their ground. Based on the research done by Aite Group and Filene Research, credit unions significantly increased their investments in online and mobile offerings in 2012.

So how do you explain the drop in satisfaction score? Either customer satisfaction scores are useless, or….

2. New member expectations aren’t being met. Tom Glatt likes to argue that tCU membership gains aren’t exactly evenly distributed across CUs, and the NCUA data, which shows that only half of CUs experienced gains, bears that out. But that doesn’t negate the fact that over the past 18 months, a lot of Americans have become new credit union members.

Why? I’d argue that in many cases it was less about the superiority of a particular credit union, and more about the inferiority of the bank they were fleeing.

And what did they find when they started interacting with their new credit union?

They found really really friendly, helpful people — but people who weren’t that good at helping them make financial decisions. They found an online and mobile experience that, although the CUs were investing in them, were inferior to what they had at their big bank. And they found that, despite all the hype, credit unions still charged overdraft fees, inactivity fees, and a host of other fees.

In other words: Their expectations weren’t met. And the result was lower satisfaction scores in the ACSI study.

—————

Bottom line: If you’ve got more theories to explain the drop in satisfaction scores, I’d love to hear them. As for my two competing theories, I strongly lean to one of them as the right explanation. But I’ll play Fox News here — I’ll report, you decide.