Don’t bank on vanity mobile app metrics

Could mobile app metrics impede your success? You bet, especially if they’re not really metrics.

Last year Mixpanel co-founder and CEO Suhail Doshi called for an end to “bullshit metrics”—often called “vanity metrics”—because they fail to measure actual user engagement or provide meaningful insights that drive a business forward. Mixpanel investor Marc Andreessen vocally backed Doshi, but not everyone was as quick to dismiss the importance of reporting on total registered users, for example.

Our point of view is more in line with that of Greylock principal Josh Elman who said, “There are no bullshit metrics as long as they represent the business in the right way. And any metrics that don’t represent the business are automatically BS.”

An analytics catch 22

Vanity metrics provide little insight into the actual success of your app, but developers would be remiss in writing them off altogether. They’re still cornerstones of press releases, venture capital pitches (although many investors look for additional proof of viability) and annual reports.  Publish a lower number than competitors and your business appears less attractive. This holds true even if you’re not comparing apples-to-apples and your number is based on actual app usage.

Evolving how we measure

The real problem is that vanity metrics aren’t metrics at all. They’re raw statistics: Basic data that’s often meaningless on its own, but absolutely essential for defining metrics that produce meaningful business insights. For example:

  • Total registered users doesn’t tell you if a user is engaged or about who’s actively using your app.
  • Weekly or monthly downloads can keep your marketing team motivated, but don’t reveal if your app is being opened and used. Total counts also leave you in the dark about which factors drive downloads.
  • Average session length is another quagmire. What’s good for one app may be considered terrible for another.  And session length doesn’t tell you what users are doing during that time or indicate where drop-off is happening.
  • App store pageviews can indicate a variety of things. For instance, that you’re using effective keywords, that your marketing is paying off or that one of the tech sites gave your app a killer review. But pageviews are fairly meaningless unless you know what you’re looking for (organic search traffic vs. direct links or information about what users do next).
  • Installs, especially of your most recent version, by day or month are important to know. However, you miss the big picture if you don’t consider your complete data set and all activity being generated across devices and versions. 
  • Uninstalls may be nothing to worry about if the user isn’t your target. On the other hand, a large percentage of users uninstalling when they’re asked to make an in-app purchase to extend gameplay is an extremely telling red flag.
  • Active users are nice to see, but not imperative to every business. You may care if you monetize through ads, but if your app is a value-add for customers designed to increase loyalty, the frequency of log-ins becomes less important. 

So, what’s a metric?

A true metric is made up of three parts. Taken together, these parts reveal trends that affect how you iterate and monetize. What’s important varies from app to app, business to business. You’re combining:

  1. Fact—What you’re measuring. For instance, in-app purchases, gym sessions, check-ins, photo uploads, etc.
  2. Aggregate—How you want to measure (count, sum, min, max, etc).
  3. Dimension—How you slice the data. For instance, by game level, gender, city, recipe ingredient or exercise type. A metric may (and likely will) have multiple dimensions.

Please note: Fact and dimension are specific to your app.

The One Key Metric (OKM) conundrum

Doshi encourages companies to start by tracking “One Key Metric” (OKM). Basically, you hang your hat on a single actionable metric that defines success for your business and then measure things related to it. Using Instagram as an example, he defines the OKM as photo uploads.

Sounds simple—perhaps too simple for some mobile apps. We’re all for using analytics to make smarter decisions about how to improve the user experience and monetization. And it may be fine for a recipe app to use recipes as an OKM. However, we find it’s better for many mobile apps to look at metrics in relation to a specific defined business goal. For example, in a mobile game, you may want to look at factors that influence successful level completion as well as session length and in-app purchases.

What we can all agree on is that it’s most important to measure things that make your active, loyal users happy, moving beyond vanity metrics to the factors that drive your business.

Free thru 12/31/13  Kii Analytics with custom in-app metrics. Try it

- Share -


More posts from

  • Kii IoT platform enables customers to create smart, connected solutions and services
  • Long track record of supporting customers like Docomo, Toshiba, Kyocera and many others
  • Strong ecosystem partners across the globe, consisting of carriers, device/sensor manufacturers, system integrators, network/infrastructure providers, chip vendors and solution developers
  • HQ in Tokyo, US HQ in Silicon Valley & offices in China, Taiwan, Hong Kong and Europe (UK, Spain, Germany)