Speech by Pavel Aksyonov, former CPO of Samolet Plus, at the All-Russian Forum of Product Management Professionals PRODUCT MANAGEMENT 2024, held in Moscow on May 30-31.
How to choose priority hypotheses not based on internal feelings, but on the basis of impartial formulas?
Selecting priority hypotheses in product management is the most important stage that can determine the success or failure of the entire project. However, decision-making is often based not on impartial formulas, but on internal feelings and assumptions. To eliminate errors, it is necessary to use special models and tools at the hypothesis prioritization stage. For example: WSJF, ICE, DHM, RICE, Rean, MoSCoW.
The fundamental point in choosing the best hypothesis is the ability to predict the income from its successful implementation. According to Pavel Aksyonov, the algorithm for such a calculation is quite simple. To do this, you need to:
Identify the key change metric.
Determine the required target audience.
Make a selection based on the database.
Build a forecast of the change in the metric for the sample.
Obtain a monetary metric by comparing AS IS and TO BE models.
This algorithm is applicable in almost any situation. For example, in the case of a mobile application for ordering a taxi.
An active user of a taxi service decided to call a car for a ride. He entered the application and ordered a taxi. However, the company, realizing that users often forget to specify the destination, decided to implement a new feature. The system automatically detects the user's last ride and offers him the same destination and fare that he chose last time. This simplifies the ordering process, prevents misunderstandings and is highly likely to motivate the client to travel in a higher class, since the company noticed that users who ordered a taxi in a higher class have a higher frequency of repeat trips in the same class.
To evaluate the effectiveness of the new feature and the hypothesis built on it, the company conducted a data analysis.
To do this:
I found out the key metric of change. Users who will book a higher class more often influence the fact that the average check for trips will increase.
I defined the necessary target audience and made a sample based on it. I took those passengers who had an upgraded class in at least one order over the last 3 months. I downloaded them.
We built a forecast of the metric change for the sample. To do this, we assumed that each economy class after an upgrade would convert into another trip in an upgraded class. For example, with a probability of 10%.
We calculated the number of such trips at a higher rate, multiplied it by the delta of the average bill between economy and higher class, and predicted the income based on this hypothesis.
As a result, the new feature has improved the on page seo service user experience (customers can now order a taxi faster and more conveniently, even if they forget to specify their destination) and increased the average bill for the company.
Using data on user behavior and preferences, the company was able to calculate the potential profit from implementing the new feature. When analyzing the hypothesis, various scenarios and audience segments were taken into account - this allowed us to present a realistic forecast.
How can we more or less accurately predict what the revenue will be from changes in the product and what specific factors influence this?
First, the key indicator of change is determined. For example, this could be conversion to purchase. Then the target audience is selected and the impact hypothesis is calculated. For example, a “repeat order discount” button on an online store page can increase conversion to purchase by 20%. Adding a “repeat order” button has a direct impact on conversion and revenue.
To calculate the income from changes in a product, you need to take into account factors such as:
the number of users who will use the new feature,
average purchase receipt,
conversion to purchase before and after the implementation of changes,
the cost of developing and implementing a new product feature.
Calculating the potential profit from changes in the product can be based on user behavior data. User reactions are analyzed - reviews, questions, likes, comments. This gives an understanding of what changes in the product can lead to an increase in income. It is necessary to take into account various scenarios and factors affecting conversion and income.
Analyzing metrics such as Net Promotion Score (NPS) and Customer Satisfaction Index (CSI) can help predict how changes to a product will impact user satisfaction and their willingness to recommend the product to others.
The impact of factors such as wait time and user satisfaction on product metrics can be measured and converted into monetary values.
To more accurately forecast changes, it is necessary to take into account experience and past fluctuations in metrics. Including experience from past mistakes. Cooperation with competitors and exchange of experience also helps to more accurately forecast income from changes in the product.
Our website contains programs for upcoming forums, lists of speakers, and other information about our projects.