Why isn’t anyone looking at my metrics?
Whenever I coach product teams, one of the first things I’ll hear when we start an engagement is that, although they present information and provide metrics, it seems no one looks at them.
They’ll tell me they show the numbers during meetings. Sometimes, someone in the organization will make decisions without even engaging with the numbers that the product person has spent time and effort retrieving. “If you build it, they will come” or, in this case, “If you show data, they will follow” doesn’t hold much weight, even though common wisdom says good data drives decision-making.
The truth is, numbers on their own aren’t enough. Metrics, at their best, clarify the difference between failure and success in a way that inspires action. Metrics that don’t do this are ignored, truthfully, for good reason. Everyone is busy, and teams want to make decisions as soon as possible.
Metrics that fail to drive decisions are useless. The reason for that uselessness lies in how we present them. When we see a metric that isn’t relevant, actionable and quantifiable (RAQ), it is just a data point. It’s no different from your car’s mileage or today’s temperature.
As product managers, we aim to help teams make better decisions consistently, which requires clear metrics. In this article, we’ll discuss how we can demystify data using the RAQ framework so that we can use the numbers we have to help others make decisions.
If we will demystify the numbers, let’s start by seeing how your numbers mystify those around you.
What Is the RAQ Framework?
In order to be meaningful, a metric must fit into the RAQ framework.
- R: Relevant
- A: Actionable
- Q: Quantifiable
Mystified Numbers
As product people, we have many metrics to choose from. To make them effective, it helps to assume that anyone engaging with the metrics you choose don’t quite know what they represent.
When our metric strategy isn’t clear, we end up with mystified metrics. Here are a few of their characteristics:
- Cognitive Overload: There are simply too many of them. According to the psychological principle known as Miller’s Law, the average human can hold seven, plus or minus two, pieces of information in their working memory at any one time.
- Lack of Clarity: When someone sees a metric, they can’t determine its importance or how to prioritize it among other information.
- Absence of Urgency: When a metric comes across their desks, people wonder, “What am I supposed to do with this, and when?” If a piece of data doesn’t prompt an immediate decision, it’s often shelved or discarded.
- Translation Issues: A metric’s meaning or utility is clear to one department but not another. For instance, the meaning of a metric like daily active users (DAU) might be apparent to a product manager but not to a salesperson, leading to discussions dominated by the highest bidder for influence rather than data-driven decision-making.
In this environment, unless something is obvious and of high urgency, it won’t impact decision-making. But the issue with prominent and high-urgency numbers is that, while they inspire action, such action tends to be reactive. Thus, the metrics don’t mean much after the emergency subsides.
If this sounds familiar, no worries. We will work with the RAQ framework to help turn your metric strategy into something usable.
To help us do that, we’re going to BobCo’s product team to demonstrate how to make this shift.
Relevance: Understanding What Data You Need
First, metrics need to be relevant to those who look at them, and that relevance comes from purpose, goals, and some form of accountability.
- Purpose: Why are we measuring this? It’s not necessary if the metric doesn’t address a specific problem.
- Goals: What are we trying to accomplish? A metric should directly support strategic goals.
- Actionability: Based on this metric, what action can we take? If the metric doesn’t inform an actionable step, eliminate it.
Understanding a metric's relevance can be simplified by telling a story that includes a challenge (push), an objective (pull), and a specific action (verb).
Defining Push, Pull and Verb
- Push: Identify the problem that pushes us into tracking the number.
- Pull: Outline the goal that this number is helping us track.
- Verb: Define the next action that could move this forward.
At BobCo, Jeff, the product manager, had a north star metric that he and his team put on many documents: churn. He realized that he often just said the number without much follow-up, however. The metric seemed important, but without specificity, the relevance of the number wasn’t clear to anyone else. So, the next time he talked about churn, he used the push-pull-verb template.
- Push: The churn of our latest AI product is not matching our predictions.
- Pull: Our goal is to lower churn by 5 percent.
- Verb: To do this, we intend to fix the major bug we’ve highlighted because we think this will lower churn with our current customers.
By telling people the motivation for tracking it, those who engaged with this metric could soon add ideas about how to lower churn and see its relevance.
Actionability: Ensuring Data Drives Decisions
Remember: At their best, metrics help clarify the difference between failure and success in a way that inspires action.
Humans act when we know what the stakes are. Think about the last show or movie you watched. You were initially interested in the movie for one reason or another; maybe you liked the poster or you knew the lead actress.
You stayed interested, however, because you learned the stakes and saw that character figure out how to either beat the odds she faced or succumb to the pressure. Either way, you were engaged because the stakes were presented and you could follow along.
Your metrics are no different, and an actionable metric should:
- Clarify Changes: Indicate what should change from the current state to the desired one.
- Set Deadlines: Specify when the results need to be achieved.
- Link to Strategy: Show how changes will help achieve strategic goals and what might happen if targets are unmet.
Turn those questions into this template:
[METRIC] will change from [CURRENT STATE] to [DESIRED STATE] by [DEADLINE], enabling us to achieve [STRATEGIC GOAL].
- Metric: The metric in question.
- Current state: Where is the metric currently?
- Desired state: Where do we need this metric to be for us to consider the project a success?
- Deadline: By when?
- Strategic goal: If we do, we can do X OR If we don’t, we can’t do Y.
Let’s go back to Jeff and his team at BobCo. Now that the team knew the relevance of churn, they still didn’t know what the stakes were. So, though the team’s questions improved, he knew that they could still be more proactive in terms of action, as they didn’t know what to do first.
So, right after he outlined the push, pull, and verb, he told them about the stakes.
Churn needs to change from 15 to 10 percent by June 1st, enabling us to achieve product-market fit. If not, based on the last board meeting, our ability to fundraise will be hampered.
Jeff let the team know the stakes, which helped them see how important the work is since it is so crucial that those on the team better understand how to act on the metric.
Quantifiability: Measuring What Matters
We don’t work alone. Product is a team sport, so the numbers we use must become intelligible to the teams that don’t work with us every day. Part of the role of a product manager is to work as an effective translator, and quantifying metrics is a part of that job.
Quantifiable metrics should have the following:
- Flexibility: They adapt to different contexts and needs.
- Strategic Connection: They link directly back to overarching goals.
- Alignment: They harmonize with leading and lagging indicators to provide a complete picture of performance.
Let’s talk about leading and lagging indicators here, as they are critical to a team that is looking to build a map that other teams can also use.
- Leading Indicators: These predict future performance and allow for proactive adjustments. Use these with your product team to inform tactical moves.
- Lagging Indicators: These reflect past performance and outcomes to inform strategic adjustments. Coordinate these across different teams to ensure a common strategic language.
One way to learn what the leading and lagging indicators are for a particular metric is to track the “how” of the number. For example, what are the reasons churn happens? There are many: unresolved customer support tickets, targeting the wrong customer, a lack of usage. As it happens, leading indicators tend to have a many-to-one relationship to a lagging indicator.
In that answer, you’ll find not just the leading indicator for yourself, but a language to talk to other teams as you’re creating a metric map.
At BobCo, Jeff built a metric map around churn.
He identified feature utilization as the key metric the product team needs to focus on to reduce churn. He also noticed a few metrics that affected Sales and Customer Success like customer health score or win rate and knew to bring them up as a bridge to talk to those teams about churn. As a result, each of the teams felt much more on track with the strategy.
Get Your Metrics Off the RAQ
In product management, metrics are vital for decision-making, yet their effectiveness depends on their presentation and utility. The RAQ framework and our case study at BobCo highlight the need to make metrics relevant, actionable, and quantifiable to maximize their impact.
Transforming numbers into clear insights requires understanding what drives our teams and business. By using strategies like push-pull-verb, as Jeff did at BobCo, we clarify the purpose and urgency of metrics.
Effective metrics should compel teams to act decisively. They must be woven into our strategic conversations, driving not just individual goals but the collective success of the organization. Metrics should become catalysts for growth and innovation.
