In standing up analytics platforms and teams for measuring web activity, I am amazed at the number of companies that still use performance measures that do not connect to the fundamental success measures of the company. I am further shocked at the length of time companies will use an imperfect metric, rather than create the necessary data elements that make corporate metrics accurate, repeatable, and durable. I have even seen companies use an imperfect metric for so long, it becomes so accepted and goes unquestioned. It’s like knowing you have a leak in your walls, knowing there is a problem, and yet being too timid to find out what it will take to fix it completely.
For example, a client of mine had created an Online Sales metric using a monthly batch cycle that matched back all customer logins per month to a master product file for all products closed that month. If the login occurred within an agreed-upon timeframe of the product closing, the product would get attributed to the online channel. Also, product counting rules allowed certain products to be counted immediately while other products could not be counted until they had been used by the customer. The monthly lag created a “driving through the rear-view mirror” approach – and added time to the decision making process.
1. Discrete Channelling
The first problem is the messy matchback methodology. While business rules can help in deciding which channel to attribute a product to, there is a clear benefit to capturing a channel flag at point of purchase and at points of contact and consideration along the way. Not only does this provide clarity in terms of channel performance, it provides the essential markers necessary to build the gold standard of customer analysis: Multichannel analytics.
2. Too Many Rules to Count
The second issue is the product counting rules – sometimes products get counted, and sometimes they do not. Depends on customer eligibility? Sometimes. How about the level of product? Maintaining a minimum balance? Maybe. What about Frequency of service? Perhaps. Product counting rules, by definition, are incomplete views of your product exposures. Managinig product counts by defining the parameters for which they are counted is like having a cupboard of glassware, but only counting the wine glasses. Not only does this create a kind of cryptic language where everyone has to constantly ask “Now are those customer numbers everyone, or just ABC-eligible? Are the products listed those we can count or not?” Product performance is not “managed” by counting rules.
3. Measuring Inertia is Not Managing Performance
When a company measures itself on its own intertia and calls it “performance”, it is simply being lazy, resting on its enviable (though temporary) position of customers coming to them. Having an actionable fact-base means that you move beyond measuring just the outputs. Measuring the inputs is key as well, because you can then form the connections between the inputs and outputs, the levers between cause and effect. I have seen "unsophisticated" companies do this very well, because they understood the relationship between the few but critical "simple" variables to the outcomes of their business. I have also seen "sophisticated" companies over-engineer this, complicating the analysis with multivariate factors and levels so numerous that it was impossible to tell what was driving results.
4. A 3-Second Late Start
Giving a 100-meter competitor a 3-second head start is a bad idea, even if you’re in the junior leagues. You might even still beat them if they are terrible, but with world-class athletes, that kind of lead is an unrecoverable position to be in. But that’s exactly what you are doing when you measure at the monthly level – you’re driving through the rear-view mirror, wondering what happened in month 1 and how you can put into place something in month 2 that will hopefully be in production by month 3, just so you can wait around until month 4 to see if those changes made a difference. Meanwhile your customers knew their key issues in the first week of the year, and have been making course corrections since then. By the time you're out of the blocks, your competition is a third of the way down the field.
That said, it is still possible to create an actionable metric from rear-view reporting. While it may be ideal to have a channel indicator and a different approach to counting products and customers, the reality is you may not be there yet. Driving online performance really requires just two essential components, what I call “Minding the Gap” and “Finding Common Denominators”.
Minding the Gap
Building a true Online Sales metric introduces a new challenge to performance measurement – connecting database actuals with customer behaviors. No matter how precise you measure them, behaviors are directional, and should be seen as the collection of events that lead to an outcome, whether desirable or not. Nevertheless, ratios can be useful to connect the behaviors and actions in meaningful ways that can be useful over time. Trending these ratios is the basis for modeling overall growth and channel penetration. When combined with channel markers, can be an essential asset for planning and managing channel shift over time. Here’s an example:
Online Close Rates: (# of products purchased online) / (# of applications started online)
Finding Common Denominators
Sometimes the concept of Demand and Net can be useful to represent Operational and Financial performance measures, respectively. They are different measures because they are used for different purposes. Demand or Operational numbers are used as indicators of company performance NOW, and point to the business levers that leaders need to pull TODAY. Net or Financial metrics on the other hand are the outcome of operations. They are the result of the cumulative data and decisions that lead to the booked orders and RUM – Revenue, Units, Margin.
That said, it should then be expected that the numerators for both numbers are different. Denominators, on the other hand, should be common to both. The denominator in both cases should be the same number of initial opportunities for a given product sale. And whether that is a measure of Visitors of Sessions, they should be consistent with each other.
Online Conversion: (# of applications completed online) / (# of applications started online)
Online Close Rates: (# of products purchased online) / (# of applications started online)
In the end, there may not be an appetite for changing the legacy metric (no matter how bad), in the near term. You may have to run two parallel metrics for some time, in order to gain buy-in and adoption. Bad thinking—however entrenched—needs to be driven out. When a metric’s credibility is based on its historicity rather than its accuracy, remove everything that prevents you from delivering precision, and focus on driving measurable performance through metrics that are grounded in reality.
Comments