Gathering user feedback is crucial after the initial launch of any new software, product, or service. It allows companies to understand how real people are actually using and experiencing their offering, identify issues or opportunities for improvement, and make informed decisions on what to prioritize for future development.
For our initial launch, we had a multi-pronged approach to feedback collection that involved both quantitative and qualitative methods. On the quantitative side, we implemented tracking of key metrics within the product itself such as active user counts, time spent on different features, error/crash rates, completion of onboarding flows, and conversion rates for core tasks. This data was automatically collected in our analytics platform and provided insights into what parts of the experience were working well and where users may be dropping off.
We also implemented optional in-product surveys that would pop up after significant user milestones like completing onboarding, making a purchase, or using a new feature for the first time. These surveys asked users to rate their satisfaction on various aspects of the experience on a 1-5 star scale as well as leaving open comments. Automatic trigger-based surveys allowed us to collect statistically meaningful sample sizes of feedback on specific parts of the experience.
In addition to in-product feedback mechanisms, we initiated several email campaigns targeting both active users as well as people who had started but not completed the onboarding process. These emails simply asked users to fill out an online survey sharing their thoughts on the product in more depth. We saw response rates of around 15-20% for these surveys which provided a valuable source of qualitative feedback.
To gather perspectives from customers who did not complete the onboarding process or become active users, we also conducted interviews with 10 individuals who had started but not finished signing up. These interviews dug into the specific reasons for drop-off and pain points encountered during onboarding. Insights from these interviews were especially helpful for identifying major flaws to prioritize fixing in early updates.
For active customers, we hosted two virtual focus groups with 5 participants each to get an even deeper qualitative understanding of how they used different features and what aspect of the experience could be improved. Focus groups allowed participants to build off each other’s responses in a dynamic discussion format which uncovered nuanced feedback.
In addition to directly surveying and interviewing users ourselves, we closely monitored forums both on our website as well as general discussion sites online for unprompted feedback. Searching for mentions of our product and service on sites like Reddit and Twitter provided a window into conversations we were not directly a part of. We also had a dedicated email for user support tickets that generated a wealth of feedback as customers reached out about issues or requested new features.
Throughout the process, all feedback received both quantitative and qualitative was systematically logged, tagged, and prioritized by our product and design teams. The in-product usage metrics were the biggest driver of prioritization, but qualitative feedback helped validate hypotheses and shed new light on problems detected in analytics. After distilling learnings from all sources into actionable insights, we then made several iterative updates within the first 3 months post-launch focused on improving core tasks, simplifying onboarding flows, and addressing common pain points.
Following these initial rounds of updates, we repeated the full feedback collection process to gauge how well changes addressed issues and to continue evolving the product based on a continuous feedback loop. User research became embedded in our core product development cycle, and we now have dedicated staff focused on ongoing feedback mechanisms and usability testing for all new features and experiments. While collecting feedback requires dedicated resources, it has proven invaluable for understanding user needs, identifying problems, building trust with customers, and delivering the best possible experience as our service continues to evolve.