Yelp is suggesting that gaming (fake reviews) is eliminated by their filter based on a working paper by Michael Luca of Harvard Business School. I think that is going too far.
Yelp Ratings and Statistics
Yes, when there are enough reviews, statistics allows filters to find abnormalities in the ratings. With enough data, ratings will have a certain distribution, unless we have a full-on statistically-managed assault on a business. Most restaurant owners and people who sell fake reviews are not capable of that type of statistical manipulation, especially if they have not scientifically studied normal Yelp distributions. Let’s though see how gaming can be successful in influencing people on the fence.
Text of Yelp Review and the Difficulty of Language
Now the text of the reviews is not easily filtered, unless Yelp claims some breakthrough in artificial intelligence. We all know there is a 1-star review and then there is a 1-star repulsive review. Same with 5-star reviews. A good writer knows how to avoid certain words and phrases and can still write a very damaging review of a restaurant. They can reverse engineer the little triggers of Yelp’s system and modify their style. Anyone interested in gaming the system will also write unbiased reviews to win the trust of the filter. In reality, I don’t believe a huge number of fake reviews get through the filter (relatively speaking), but I see no way to make it airtight with current technology. Therefore, the most sophisticated attempts will probably be a step ahead of Yelp for the foreseeable future.
Here is Yelp’s simplistic description of their filter (the narrator is talking very fast):
In contrast to truly horrible reviews, I have read many 1-star that leave me open to seeing what others say. These 1-star reviews tend to focus on one complaint too much. The repulsive ones can really disturb you especially if you are choosing a restaurant for someone else.
Limited by Technology
As Yelp has not claimed an accurate algorithmic understanding of language or proven it (to the best of my knowledge), Yelp cannot tell the difference between an authentic 1-star review and a fake 1-star review. It only knows to get rid of one of them. So the text of the fake 1-star review may make it by Yelp’s filters. The difficulty of understanding language made the IBM supercomputer victory on Jeopardy so amazing. IBM, however, is normally 5 to 10 years in front of everyone else. And Yelp would have to go through millions of review with a much bigger and more powerful supercomputer that has accurate software.
Now, Yelp, of course, must manage how other customer react to the review by clicking helpful, funny or cool, but most readers have no idea what to expect if they go and how can they say if the review is right. Most ignore and barely anyone returns to click a review as helpful making the statistical usage of this meaningless. Yelp does assess the trustworthiness of the reviewer. That normally means someone who wants to game the system has built a substantial and largely accurate profile of reviews. It takes time to outsmart the filter, but it can be done and people do get paid to do it.
Customers on the Fence
I do think the general ratings hold a disproportionate influence. For those who pay attention to them the most, they don’t even visit the specific Yelp page but scan the ratings in the search. But for people on the fence, they tend to mine the text for opinions.
There is another phenomenon that I have noticed and is very hard to detect but can be deadly. This is when someone gives a negative rating with a positive review or the opposite. This is, if done with a rough statistical understanding, can be method to counter statistical distributions. The conflicting review will be suspicious, but that’s not the one that will serve the faker.
As a writer, I have noticed that it is incredibly easier to lie in writing than while talking. We have evolved to sense lying when hearing voices, looking at faces and bodies. Lying in text is so much more difficult, as variation is very individual. We have unconscious giveaways we use while lying in person that don’t have equivalents in writing. Successful face to face liars are made, while most English majors can replicate fake opinions. Yelp’s vague blanket statement doesn’t hold true for me, unless they are way in front of the rest of the computer and artificial intelligence world. If they were, I’d imagine they could make tens (if not hundreds) of billions of dollars in much more profitable fields.