3 Tips for Better Product ReportingBy: John Mathew
As part of Mainsail’s 2019 webinar series, Eric Boduch, co-founder and Chief Evangelist of Pendo, led a discussion around data driven product reporting. Pendo is a collection of product analytics tools that enables product teams to drive product adoption, customer loyalty, and team innovation. As a member of the founding team at Pendo, Eric discussed how thoughtful reporting on product metrics can lead to faster growth, better product teams, and easier communication. Here are a few tips from his talk.
Strive for consistency in product reporting
For board or stakeholder meetings, Eric stresses the importance of showing the same metrics trending from one check-in to the next, “one of the most frustrating things that I see are different slides with different metrics”. What are those top metrics to track? Eric advises product leaders to sit down with their executive teams and discuss exactly what core product metrics stakeholders want to see every quarter, but in general, he recommends four categories:
- Adoption – Are customers using the product/feature? Having a baseline understanding of the volume and frequency users are interacting with your product is critical. If you release a new product or feature, understanding if your customers are actually using it is an additional barometer for success and a potential way to predict future adoption.
- Critical event metrics – Are customers using the product in the way you intended? How are customers using your product 1-month post onboarding vs. 3 months? Understanding what workflows and paths users have within a product can help unearth issues and help your team address them. For example, if a customer is using 10 clicks to generate a report when there is a feature that completes it in 2, a team member can add in-app guides or messaging to help nudge that user in the right direction.
- Customer happiness – Are they satisfied with the new product/feature? Tracking NPS or CSAT scores helps give a clear picture on how customers feel about your product. Try tracking a customer’s NPS over time and at different critical events (post onboarding, after a support request is fulfilled, etc.) to have a nuanced understanding of when your product is functioning at its strongest and when customers are having issues.
- Retention – Are customers consistently using your product? Monitoring customer usage over time can help your teams fill gaps in the product as well as reduce churn. If certain cohorts, customer segments, or user types haven’t used a product or feature after a long period of time, it can be a precursor to address issues with the product or proactively engage with frustrated customers.
Set goals early and publicize them widely
Adoption, critical events, customer happiness, and retention are great metrics to track but one of the most common questions is “How do I set the right goals?” For the launch of a feature, should success be defined if 50% of users are using it or should it be 70%? 100%? The easy answer is that it depends, but what’s important is that your team is clear as to why a goal is set and that it’s backed by data. If a company is releasing a feature that is only relevant for 25% of its user base, having that background when developing goals for feature adoption is key. If a company has historically churned 2% customers for the past few months, using that historical data as an input while setting cohort retention goals for the next quarter is critical.
When product comes up at your next board or executive team meeting, Eric recommends taking the time out to set expectations on what success will look like for product or feature launches down the line. If it takes too much time to develop an understanding of what success looks like, it could be a signal that the product or feature shouldn’t be launched. It’s important to have a definition of a product or feature that can be distilled for your broader team to understand. Eric recommends creating simple, data driven product goals and publicizing those product goals to the executive team early.
Align key metrics to your strategic goals
Apart from repeatable metrics to consistently track, Eric also recommends having ad hoc product reporting metrics that are applicable to a company’s strategic goals. A key mistake made by product teams is getting complacent with product reporting and using the same metrics to track goals across every scenario. For example, if a company wanted to drive adoption of an accounting feature for end-of-month reporting, tracking Daily Active Users could misrepresent the feature’s success. When deciding on what ad hoc metrics to track, Eric stresses that companies should strive to pick product metrics that align with strategic goals; for example:
- If your goal is to improve the onboarding experience for customers
- Measure cohort: time to value – After a user first logs in, are they logging in consistently over the next 90 days and using the product’s core features?
- Measure NPS (after first 90 days) – Are users happy after they onboard?
- If your goal is to start selling to an underserved customer segment
- Measure adoption in a segment – Are users within a specific segment using the product?
- Measure NPS in a segment – What do current users within the segment think of the product?
- If your goal is to increase engagement with the product
- Measure Daily Active Users/ Monthly Active Users (change formula depending on usage frequency)– How many users are engaging with your product at a high frequency?
- Measure high and low usage edge cases – Who are your power users? Who hasn’t logged in for 30 days?
Mainsail believes strongly in aligning product planning with the company’s strategic goals and using tools like Pendo to measure progress along the way. For more outcome-based product management, check out our previous posts on better product roadmapping and objectives and key results.
The information herein is based on the author’s takeaways on Eric Boduch’s webinar. Opinions and beliefs on product reporting and related topics are not necessarily representative of those of Mainsail. There can be no assurance other third-party analyses would reach the same conclusions as those provided herein. The information herein is not and may not be relied on in any manner as, legal, tax, business or investment advice.
Certain information contained in this content piece has been obtained from published and non‐published sources prepared by other parties, which in certain cases have not been updated through the date hereof. While such information is believed to be reliable for the purposes of this content piece, neither Mainsail nor the author assume any responsibility for the accuracy or completeness of such information and such information has not been independently verified by either of them. The content piece will not be updated or otherwise revised to reflect information that subsequently becomes available, or circumstances existing or changes occurring after the date hereof, or for any other reason.
Third-party images and logos included herein are provided for illustrative purposes only. Inclusion of such images and logos does not imply affiliation with or endorsement by such firms or businesses.
Certain information contained herein constitutes “forward-looking statements,” which can be identified by the use of terms such as “may,” “will,” “should,” “could,” “would,” “predicts,” “potential,” “continue,” “expects,” “anticipates,” “projects,” “future,” “targets,” “intends,” “plans,” “believes,” “estimates” (or the negatives thereof) or other variations thereon or comparable terminology. Forward looking statements are subject to a number of risks and uncertainties, which are beyond the control of Mainsail. Actual results, performance, prospects or opportunities could differ materially from those expressed in or implied by the forward-looking statements. Additional risks of which Mainsail is not currently aware also could cause actual results to differ. In light of these risks, uncertainties and assumptions, you should not place undue reliance on any forward-looking statements. The forward-looking events discussed in this content piece may not occur. Mainsail undertakes no obligation to update or revise any forward-looking statements, whether as a result of new information, future events or otherwise.
No representation, warranty or undertaking, express or implied, is given as to the accuracy or completeness of the information or opinions contained in the enclosed materials by Mainsail and no liability is accepted by such persons for the accuracy or completeness of any such information or opinions. For additional important disclosures, please click here.