KQ Article

Bankers often strive for the perfect metric. This beguiling vision is paradoxically a problem for the adoption of metrics. Rejection of all metrics sometimes follows, as people realize that the metric didn’t solve all of their problems.

We will never develop a perfect metric. So why advocate for the use of metrics? Here are three good reasons:

1. Metrics can certainly be powerful business tools. A metric such as “customer lifetime value” creates an estimate of the profitability of a single customer, and may be central to an effective marketing strategy.
2. Rejection of a metric is often based not upon a sober assessment of the metric’s value, but laziness. Put another way, it is easier to reject a metric than take the time to understand it and apply it appropriately. I recommend cynicism toward all who are convinced that metrics won’t help before any calculations are made.
3. Although no metric is perfect, that doesn’t mean that metrics have no use. This is analogous to saying that because medicine can’t give us immortality, it is worthless. Metrics can be important tools despite gaps in what can currently be achieved.

The arguments I just made may not surprise you. But here’s something that might. I’d like to provide a final and far less intuitive reason for continuing to use metrics: the very imperfectness of their nature can prove invaluable in aiding communication.

Most measurements have flaws. But these imperfections are the very sparks that ignite discussion, expose underlying assumptions and help to align strategy and tactics. 

Imperfection: good
Consider a bank entering a new market. Its marketing department demands a vigorous advertising campaign to stimulate demand. The finance team is unhappy about the idea, because it is not clear that the expensive campaign can be justified. Marketing responds by offering a measurable target for the campaign, promising to deliver 2,000 new customers by a certain date.

Why is this metric imperfect? Because the measurement system to be used to count customers generated by the campaign is ad-hoc, while the strategic aim of the business is to maximize profits, not customers. The marketers have implicitly assumed that more customers necessarily means more profits.

Finance highlights and rejects this assumption, countering that these new customers may actually drain resources, as they will cost more to acquire and serve than they pay in fees. Marketing responds that finance’s customer valuation metrics are inappropriate as they use a one-year horizon, whereas many customers become highly profitable over a period of several years. Finance concedes that the metric is imperfect, but asserts that not all customers stay active. An examination shows those that those who stay active share certain characteristics.

The investigation sparked by these conflicting metrics has created a better, more targeted advertising strategy. Fewer customers are recruited, but those who become new customers are highly profitable. The company’s objective of increased profits has been realized — all through the use and refinement of imperfect metrics.

The common language of metrics facilitates discussion around the flaws of plans. The friction caused by imperfect metrics creates sparks that illuminate any incompatibilities between strategy and tactics. Metrics also rely on assumptions. Rather than seeing this as a weakness, we should realize that metrics allow us to peer into each other’s assumptions, creating a golden communication opportunity. In the scenario above, campaign supporters and opponents alike benefited from seeing what the other was measuring — and assuming.

Make all assumptions known
Imperfect metrics have great value as a communication tool — but only if they’re not kept secret. By “secret,” I am referring to those metrics with owners who hide their data sources and obscure the assumptions behind their calculations.

In this category, I include any metrics that cannot be explained to intelligent observers. If the last decade has taught us anything, it is that if a person can’t explain the assumptions behind a model, he or she probably doesn’t understand it. For metrics to be useful, they must be valuable in their own right. Or, at the very least, they must facilitate communication about what assumptions are being used. In contrast, we cannot ascertain the value of secret metrics. They stifle communication and should be treated with caution.

So, never put your unquestioning faith in the metrics you read. Always identify the underlying assumptions behind them first.

In summary, metrics are indeed imperfect. But so what? They should still be employed widely in all forms of business, including financial services. Some metrics are valuable in their own right, and many contrarians who argue against them are merely being lazy. And remember: an ability to solve all problems does not mean that metrics can’t solve any of them. Even imperfect metrics hold tremendous power to bring assumptions to the surface and facilitate healthy, productive communication.

_{© 2010 Neil Bendle}

What did you think of this article? Share your thoughts below.