Focus on the Metrics that Matter: Identifying Your Product's Key Metrics and KPIs

Let's talk about product management KPIs and metrics. We live and work today in a world that is increasingly data-driven, but we cannot successfully adopt a data-driven approach to decision making without first identifying the metrics that matter most.In business, when faced with decisions, there’s far less “Because I think or feel like…” and far more “Because I crunched the numbers and they told me to.” Product management intuition is still a great thing to develop, but statistics and analytics will help you hone your product sense and justify your decisions to stakeholders.Plus, there’s no shortage of tools to help us quantify seemingly everything. As product managers, data is especially important at work where key metrics and KPIs are vital signs to our organizations and products. The right product metrics give us a read on the health of our product, help us identify strengths and weaknesses, track improvement over time, diagnose problems, etc.In order to make data-informed decisions or quantitatively validate ideas, we must first have numbers to work with. Collecting data is no longer a issue - there’s no shortage of tools out there to grab every metric you want. The real issue is lack of focus. With big data comes big responsibility; before we can (responsibly) use data, we must understand and identify the metrics that matter most to our product vision, customer needs, and company goals.Michael Rutledge a Product Manager at American Express describes the data predicament well, “So the challenge becomes, with all this data how can we use it all effectively? I’ve seen companies that run in circles over and over again because they can’t get clarity and agreement on what metrics matter. It can become a sickness when there are too many metrics flying around and some are going up, others going down, and actions are paralyzed because there’s too much discussion about whether some test has illustrated an improvement or not. This is sometimes called analysis paralysis.”So how do we know what data we should be paying attention to? What are the metrics that actually matter?

The One Metric That Matters (OMTM) Approach

You could set out to measure everything related to your product if you really wanted to, and you’d likely have no trouble collecting the data with the seemingly infinite analytics and tracking tools available right now. However, the fewer the metrics you chase, the more focus you (and your organization) will have. In fact, some businesses have taken metric minimalism to the extreme by adopting the idea of the “One Metric That Matters.”

“This isn’t an excuse to ignore other parts of your business, or to cease efforts to collect all of the important data your business generates. Instead, the one metric is a line you draw in the sand, a commitment you make, for one day, one month, one year, however long you need to optimize your company’s performance based on that metric.” Henry Hund, Director of Marketing and Growth at RJMetrics explains.

When you’re trying to focus on a single metric to drive your entire business, you obviously want to make sure you’ve chosen wisely. Oftentimes looking at a rate or ratio is a better bet than an absolute number because it gives you something to compare it to over time and ask yourself, “are we doing better, worse of the same?”Ben Yoskovitz, VP of Product at Golnstant says cohort analysis is one thing to consider, “‘Increased conversion from last week’ is better than ‘2% conversion.’ The key here is cohort analysis, where you track a metric over different groups of people, typically over different periods of time. For example, you drive traffic to your site through Google AdWords and measure conversion for a week. But you also measure engagement and churn down the road for those users.

Then you make a change (to your website, product, or something else) and track those numbers for another batch of people. Each group of people visiting and signing up is a cohort.”Of course, focusing on a single metric will probably be a little too broad to create anything truly actionable (though it can provide an excellent focus point to rally the entire company around.) In some cases, adopting a less extreme variation of the “One Metric That Matters” approach is a viable option; instead of picking a single metric to rule them all, you could identify a single metric for each facet of your product (i.e. adoption, retention, usage, revenue, service).

What are the Metrics that Matter to Stakeholders?

While some metrics might be fascinating and useful to you or to your engineering team, you’ll need to be reporting up and out in your organization as well. You should know your audience and what matters to them because it’s critical that you measure and communicate the metrics that stakeholders and other decision makers within your organization need to see.

‍Which metrics do stakeholders care about? According to Tim Platt, a Product Manager at American Express, it can get a little personal at times, “It’s a metric on which you will be scrutinized and against which you will need to show how you are moving the needle or influencing it in a manner that is desirable. They’re the things you need to be moving the needle on to ensure you stay funded or to show your leadership that the world you’re doing is delivering value both to the business and also the end users of your product, the two are intertwined. Satisfying, acquiring and engaging users benefits the business and a successful business benefits your product.” (Source.)Sure, when you think about what’s important to your executive team and other functions within your company resource consumption, customer satisfaction, Net Promoter Scores, and other topics may come up. But ultimately, stakeholders care about cash.

‍Whether it’s an indicator of how much you’re making today or how much you’re going to make in the future, monetary metrics matter.In fact, financial metrics such as revenue will always be the most important KPIs of any product. “Some people may object to money being the central purpose of a product… but since money pays for salaries, electricity, office space, health benefits, taxes etc, the success of the product must somehow be traceable to this,” says Anders Linsdorf, CEO of Sensor Six, noting that revenue, margins, and equity are particularly important.Your product’s business model will also influence key metrics. If your product has a subscription model, its key metrics (or more accurately, vital signs) are retention rate and churn rate. These are the yin and yang of any subscription business and will tell you how many customers are still happy to keep paying up and how many customers are voting with their feet. Your retention rate is the percentage of customers who stick around at the end of a subscription cycle or term by renewing their contract, while churn refers to the percentage of customers who do not stick around; the percentage of customers who cancel or simply don’t renew.“Retention rate and churn rate should again be based on the typical term length and billing period, so if you are dealing in enterprise SaaS, retention rate is number of customers who extended their contract in the period / number of customers whose contracts where up for extension. Churn rate is 1 – retention rate.” advises Lars Trieloff, Director of Product Management at Blue Yonder.Stakeholders are also going to care about Average Revenue Per User (RPU), Customer Lifetime Value (LTV), gross margins and a host of usage data (total users, monthly/daily users, monthly/daily active users and average session time). These are the kind of metrics you need to be reporting up and the ones that most closely align what you’re doing as a product manager and the overall direction of the business.

Key Metrics for Features

While the previous set of metrics we discussed are key to the business, there’s another set of measurables that are far closer to the PM's heart - metrics that help you understand how customers are interacting with your product and identify opportunities to improve.

• How often is each feature being used?
• What feature sets tend to be used by the same people?
• Which features are your engaged users using most?
• Where do users get stuck and abandon the product?
• How long are users spending on each feature?
• Who abandons it and who keeps using it?
• Are there seasonal trends that can be used for predicting future revenue and usage?

The list can get lengthy, and can be really specific to the nature of your business and products. When looking at feature and usage metrics, you need to not only identify what you WANT to know but what COULD you know. Having more data to play with in this arena is usually a good thing because you can test different hypothesis for correlations, but you can reserve reporting for only the data that pertains to the goals you’re currently pursuing.“It’s most important to track the metrics related to your top goals. Avoid the temptation to add “interesting stats” to your list. Will you actually use these numbers to help you make a decision? Do you really need to track them over time, or is a current snapshot sufficient? Stay focused on the metrics that are closely related to your goals to avoid unnecessary implementation effort and dashboard clutter.” Kerry Rodden, a User Experience Researcher at Google says in a blog post on Google Ventures’ blog.

Engagement Metrics Help you Solve for “Why?”

“A metric will tell you that something is happening, while an analysis will tell you why something is happening.” - Vince Law, General Assembly

An understanding of what your users are doing and how they are using your product is an invaluable tool throughout the product development process, and should be one of the cornerstones of your measurement strategy.User engagement metrics are particularly helpful when it comes to understanding how users interact with your product, writes Evgeny Lazarenko, a Data Analyst at Paktor, “A good starting point for user engagement and behavior analysis is to trace how often your users log in to their accounts. What also helps is to conduct the same analysis for a cohort of users over time.” (Via Quora.) But, these metrics must tell a story in order to be useful to you and your team.Let’s say you decide to measure average sessions per user per month and noticed it's increasing. That’s not a stat you can exactly drop in a PowerPoint and send to the management team by itself; instead you should be accompanying that data point with the answer to the “why” question that you know will follow the pending celebration. What changed and how can you leverage that elsewhere?“The whole point is to learn as you go, then reinvest this information to make better product decisions down the road. If all you know is whether your investment was successful, but you don't know why, then you won't know what your next investment should be, and eventually your luck will run out” says Ben Foster on PM Rant.

Build to Measure (and Don't Ship Before You Have Metrics)

Waiting to define your key metrics until after your product has shipped and someone gets curious is not exactly a recipe for success. If you wait this long, you will have already missed your opportunity to start measuring from a known baseline (of zero) and it may not be possible (or practical) to tack on measuring capabilities to something once it’s already in the wild.Don’t ship anything until you’ve identified the metrics that matter, advises Tim Platt, a product manager at American Express, “Make metrics part of your definition of ready i.e. the criteria that you assess your user stories against in order to determine whether they can be considered defined and ready to be consumed by your scrum team. It’s a great way of forcing yourself as a product owner to ensure everything you do is measurable where possible and your product team will truly value it.”Pro Tip: build metrics testing into your QA process to make sure the data you’re collecting is accurate in a lab setting before you start trying to quantify the unknown.

You Are Not in Sales, So Stop Reporting Metrics Like You Are

Obviously, the metrics most in demand by stakeholders relate to sales; conversions, funnels, performance against targets and forecasts, customer acquisition costs…you get the point. These are all great stats that product managers should be intimately aware of, however these metrics have nothing (or very little) to do with your product itself.Even if you had to get involved in making the reports for this data happen, let sales report their stats and stick to talking about your product. It’s a matter of focus; don’t fall into the trap of reporting on the successes and failures of sales and marketing unless you are actually responsible for some of those areas. Keep your focus on what you do control: your product.Don’t know what to talk about? Mike Smart of Pragmatic Marketing suggests highlighting your product’s speed to market and the competitive landscape: “As an example, speed-to-market is a metric that can be tracked in several ways. How fast did we reach the market with new product releases compared to last year? How many times did we beat the competition to market with comparable product releases? Where do we rank within our industry segment? Over time, we learn what effect improving this measurement has on increasing revenue. Every team member should understand these relationships and how their projects and key activities affect the outcome of the operational drivers.”

Measure Your Product's Performance, Not Your Product Management

Finally, it can be really easy to point the barrel of your metric making at your internal processes and performance vs. how your actual products are doing in meeting your business’s goals. If HR or your manager is making you track how long it takes you write a spec or how many user stories you create a month then you might need to include a bit of navel-gazing into your analytical approach to work, but don’t confuse what a great (or lousy) product manager you are with whether or not your product is kicking butt or falling behind.Quantifying the outcome of all your hard work--your product and its performance--is more important than attempting to quantify “how the sausage is made.” As Ken Allred, Founder & CEO of Primary Intelligence, puts it:

“While I do think that measuring the internal aspects of product management is important, I would propose that measuring the actual results of product management is much more vital.” (Source.)

Just like developers shouldn’t be congratulating themselves on how many lines of code they wrote or bugs they fixed, the true measure of your worth is what’s happening outside of your office.

The TL;DR

Focus is critical when it comes to your product, which is why you need to identify what KPIs (Key Performance Indicators) and product metrics are most important in order to chase the right ones. Your key metrics should act as a set of product vital signs that provide insight that’s not only useful to you as a product manager but also to stakeholders and other folks within your organization. The specific metrics you monitor within your team (and/or with your engineers) will likely vary from those your stakeholders care about individually, though as a whole should act as levers to what you’re tracking across your organization.