3 Tips for Better Product Reporting

As part of Mainsail’s 2019 webinar series, Eric Boduch, co-founder and Chief Evangelist of Pendo, led a discussion around data driven product reporting. Pendo is a collection of product analytics tools that enables product teams to drive product adoption, customer loyalty, and team innovation. As a member of the founding team at Pendo, Eric discussed how thoughtful reporting on product metrics can lead to faster growth, better product teams, and easier communication. Here are a few tips from his talk.

Strive for consistency in product reporting

For board or stakeholder meetings, Eric stresses the importance of showing the same metrics trending from one check-in to the next, “one of the most frustrating things that I see are different slides with different metrics”.  What are those top metrics to track?  Eric advises product leaders to sit down with their executive teams and discuss exactly what core product metrics stakeholders want to see every quarter, but in general, he recommends four categories:

• Adoption – Are customers using the product/feature? Having a baseline understanding of the volume and frequency users are interacting with your product is critical. If you release a new product or feature, understanding if your customers are actually using it is an additional barometer for success and a potential way to predict future adoption.
• Critical event metrics – Are customers using the product in the way you intended? How are customers using your product 1-month post onboarding vs. 3 months? Understanding what workflows and paths users have within a product can help unearth issues and help your team address them. For example, if a customer is using 10 clicks to generate a report when there is a feature that completes it in 2, a team member can add in-app guides or messaging to help nudge that user in the right direction.
• Customer happiness – Are they satisfied with the new product/feature? Tracking NPS or CSAT scores helps give a clear picture on how customers feel about your product. Try tracking a customer’s NPS over time and at different critical events (post onboarding, after a support request is fulfilled, etc.) to have a nuanced understanding of when your product is functioning at its strongest and when customers are having issues.
• Retention – Are customers consistently using your product? Monitoring customer usage over time can help your teams fill gaps in the product as well as reduce churn. If certain cohorts, customer segments, or user types haven’t used a product or feature after a long period of time, it can be a precursor to address issues with the product or proactively engage with frustrated customers.

Set goals early and publicize them widely

Adoption, critical events, customer happiness, and retention are great metrics to track but one of the most common questions is “How do I set the right goals?” For the launch of a feature, should success be defined if 50% of users are using it or should it be 70%? 100%? The easy answer is that it depends, but what’s important is that your team is clear as to why a goal is set and that it’s backed by data. If a company is releasing a feature that is only relevant for 25% of its user base, having that background when developing goals for feature adoption is key. If a company has historically churned 2% customers for the past few months, using that historical data as an input while setting cohort retention goals for the next quarter is critical.

When product comes up at your next board or executive team meeting, Eric recommends taking the time out to set expectations on what success will look like for product or feature launches down the line. If it takes too much time to develop an understanding of what success looks like, it could be a signal that the product or feature shouldn’t be launched. It’s important to have a definition of a product or feature that can be distilled for your broader team to understand. Eric recommends creating simple, data driven product goals and publicizing those product goals to the executive team early.

Align key metrics to your strategic goals

Apart from repeatable metrics to consistently track, Eric also recommends having ad hoc product reporting metrics that are applicable to a company’s strategic goals. A key mistake made by product teams is getting complacent with product reporting and using the same metrics to track goals across every scenario. For example, if a company wanted to drive adoption of an accounting feature for end-of-month reporting, tracking Daily Active Users could misrepresent the feature’s success. When deciding on what ad hoc metrics to track, Eric stresses that companies should strive to pick product metrics that align with strategic goals; for example:

• If your goal is to improve the onboarding experience for customers
  • Measure cohort: time to value – After a user first logs in, are they logging in consistently over the next 90 days and using the product’s core features?
  • Measure NPS (after first 90 days) – Are users happy after they onboard?
• If your goal is to start selling to an underserved customer segment
  • Measure adoption in a segment – Are users within a specific segment using the product?
  • Measure NPS in a segment – What do current users within the segment think of the product?
• If your goal is to increase engagement with the product
  • Measure Daily Active Users/ Monthly Active Users (change formula depending on usage frequency)– How many users are engaging with your product at a high frequency?
  • Measure high and low usage edge cases – Who are your power users? Who hasn’t logged in for 30 days?

Mainsail believes strongly in aligning product planning with the company’s strategic goals and using tools like Pendo to measure progress along the way.  For more outcome-based product management, check out our previous posts on better product roadmapping and objectives and key results.