Greg Yeutter
📈

Framework for Diagnosing Falling Metrics

Description
What to do when the numbers dip
Tags
Product ManagementStartups
Created
Dec 12, 2020 7:29 PM
Updated
Apr 11, 2022 6:57 AM

We’ve all been there. Metrics behind target, under pressure to fix ASAP. Here’s a framework that should help you get to the bottom of it.

Adapted from Linda Zhang’s article, What to do when your metrics dip, with edits from my own experience.

Framework:

  1. Importance: Does this metric even matter?
    • Do a gut check of why we decided to track this metric in the first place
    • Are we just tracking this metric because the boss made us, even though it doesn’t make sense? Did the boss’s boss assign it to the boss? Is the boss willing to have a true conversation?
    • Look back at the business' North Star metric. Does this metric derive from the North Star, or is it irrelevant?
    • Even if it doesn't directly contribute to the North Star metric, does it still contribute to some other goal?
  2. Change: Did we recently ship a feature or change a process, making the old metric irrelevant?
    • What features were launched recently that may have affected one of these metrics?
    • Is the old metric more important than the new feature?
    • Should the metric be updated because the new functionality improves the North Star KPI more directly?
  3. Timeframe & Variance: Over what timeframe did this metric change?
    • This is a great motivator to track the historical trends of a metric.
    • Is this fluctuation normal? Has this ever happened before? Is it linked to seasonality?
  4. Root Cause: Investigate the process and reporting method.
    • First, what is the equation that is leading to this metric? E.g.:
        • # comments = # published articles * # views/article * # comments/view
      1. Dissect the equation to find out which individual component changed.
        • For example, in the formula above, see how published articles, views/article, and comments/view changed in the last month
    • Are all the metrics being reported properly? Is some system down somewhere?
    • Are projects and tasks defined specifically enough to create momentum on the metric?
    • Try asking
      🚘
      The 5 Whys
  5. Growth is Not Unlimited: Is the metric too ambitious?
    • Project goals should be aggressive, but not too aggressive. Does the metric exceed the throughput of our existing processes?
    • Will adding more resources increase the output, even if the increase is marginally lower? Is the marginal increase still cost effective?
    • Can we create more small projects that will directly chip away at the larger metric?