Read at Medium
Product ratings can be set up to measure a superficial view for gaining stakeholder trust or user acquisition instead of being an accurate reflection of the quality of the product.
Product teams can game the system by either pushing only good ratings to the app stores or capturing positive feedback strategically at certain parts of the product, for example, at the completion of a transaction.
The Net Promoter Score (NPS) tries to circumvent these issues by scoring the overall user experience with a contributing reason. However, relational NPS is susceptible to negativity bias while transactional NPS has the same effect previously mentioned.
Moreover, product ratings in any numerical representation form suffer from the issues of subjectivity, anchor bias, and noisy ratings.
A fair approach to measuring user satisfaction involves measuring factors such as time savings, task efficiency, and current quality. Users care about what they are getting now more than what others got in the past. Averaged ratings muddies decision-making.
Measure how fast the job can be done (time savings for the user). Measure the frequency of task completion (efficiency of time). Measure recency with a time window to reflect the current quality of the product.