In a TechCrunch post this morning, Klint Finley took a look at a new startup, Review Signal, that is working to clean up the “cesspool” of web hosting reviews with a system that uses software to track discussion about a given hosting company on Twitter, and share the aggregate of that data as a percentage rating on a positive-to-negative scale.
According to TechCrunch, the service was started by Kevin Ohashi, a former moderator at the online forum Web Hosting Talk, who had first-hand experience with the disorganized, questionable, even unreliable state of online reviews concerning web hosting services.
The reliability of web hosting reviews has been in question as long as the reviews themselves have existed online, with honest reviews (most of the reviews posted on Web Hosting Talk are a good example) generally representing first-hand accounts and very frequently motivated by bad experiences, and many formal web hosting reviews sites seemingly motivated by affiliate links or other profit motivations.
The overall sketchiness of web hosting reviews is an issue we’ve faced directly at the WHIR, and it’s the reason we’ve chosen not to offer anything like reviews on our site, or to recognize positive “reviews” in our reporting. A similar problem seems to exist in the world of web hosting “awards.”
Ohashi seeks to pull back the curtain on the methodology around the reviews with Review Signal, a project that spawned from his masters thesis on sentiment analysis. The service scans Twitter for posts about a given web host in order to create a picture of how satisfied its customers are.
Because sentiment analysis is an imprecise science with a pretty significant margin for error, Ohashi tells TechCrunch, the tool throws out those tweets that it can’t definitively identify as either positive or negative. The project also requires a lot of effort to fight against the ability of spammers to game the results, meaning Ohashi spends time looking over the tweets that are analyzed and updating the system to spot bad results.
While the results may be “honest,” in the sense that they’re not being created or massaged by the site, they may not be correlated directly to the quality of a particular hosting service so much as a measure of the effectiveness of that company’s social media efforts. Judging by the overall ratings on the site (pictured above) the ratings in general seem to be low, with hosting companies just outside the top 10 barely breaking the 50 percent positive mark. This is probably owing to Twitter being used by most people as a vehicle for complaining more than applauding the services they consume.
The Review Signal website includes a “how it works” section, that explains in pretty good detail how positive tweets are distinguished from negative, and how tweets about a business’s services are distinguished from other, “irrelevant” tweets about a business.
The site is monetized by posting affiliate links to the companies listed in its rankings, which would seem to raise the reliability issues and conflict of interest that plagues other web hosting reviews sites, but Finley writes that Ohashi’s plan for proving its reliability has to do with transparency – Review Signal lets users view the tweets that are being evaluated by its systems. And the fact that Linode, which doesn’t have an affiliate program, is near the top of the site’s rankings, would seem to support its unbiased claims.
According to Finley’s Tech Crunch post, Ohashi isn’t interested in developing the site into a brand monitoring service, but does plan to expand into hosting-related services such as domains and email.
Talk back: Do you think there’s a need for an unbiased web hosting reviews site out there? Do you think the formula for Review Signal meets that need? Let us know in the comments.