It’s summer in the northern hemisphere. And as holidaymakers travel to unfamiliar places, that means demand for online customer reviews. Want to find a restaurant that won’t give everyone food poisoning, or the perfect accommodation for a city break, or a mosquito repellent that actually works? Whether you are looking on Tripadvisor, Airbnb or Amazon, you will almost certainly be guided by reviews from other people. Should you be?
The short answer is yes: better to have some information than none. But the flaws of online reviews are evident. For products with some objective measures of quality, there is a big gap between the views of punters and experts. A study in 2016 by Bart de Langhe of Vlerick Business School, in Belgium, and his co-authors found that user ratings for 1272 items listed on Amazon.com bore little relation to either the verdict of Consumer Reports, a US product-testing organisation, or to their resale value.
Should you trust that five-star rating on Airbnb? The short answer is yes but you need the long answer to win the game of online reviews.Credit: iStock
That might be because consumers place greater value on more subjective things like a product’s brand. But if ratings are based on subjective criteria, then another problem arises: what if your tastes differ from other people’s? The best book ever, according to members of GoodReads, an online community of bibliophiles, is The Hunger Games by Suzanne Collins. You may agree, but plenty of people do not.
Another problem is that the people who bother to leave reviews and ratings may not be representative of consumers as a whole.
In a study published in 2020, Verena Schoenmueller of Esade, a business school in Spain, and her co-authors examined the distribution of ratings left in around 280 million reviews of more than two million products and services on 25 different platforms. They broadly confirm a familiar pattern: a polar distribution of ratings. More of them were at the extremes of the scale than in the middle, and there was a skew towards more positive ratings.
Loading
There are lots of theories as to why online reviews follow this pattern. People who have chosen to buy something are already more likely to be satisfied with it. Extreme experiences, good and bad, are more likely to prompt reviews. Some write-ups are not real: estimates of the prevalence of fake reviews vary but they are certainly a problem, and one that generative AI may make worse.
The type of platform matters, too. Sharing-economy markets have a different feel. You could leave a four-star review for your Airbnb stay, but now that you have established a relationship with the hosts, and since they are also rating you, it’s much easier to just award five. A paper by Georgios Zervas of Boston University and his co-authors, last updated in 2020, found that average ratings for Airbnb properties are consistently higher than those for hotels on Tripadvisor.
In theory, businesses have an interest in soliciting as representative a sample of reviews as possible. Honest customer feedback is the best way to spot and fix problems, after all. In practice, the importance of good ratings, particularly for firms that are struggling for visibility, is an incentive for jiggery-pokery. A study from 2013 by Dina Mayzlin of the University of Southern California and her co-authors suggested, for example, that small, independently owned hotels generated more positive fake reviews on Tripadvisor than branded hotel chains.
