1

helpfulness of reviews

I wish that users can give feedback about whether a staff review is helpful or unhelpful.  If many users find a staff review is unhelpful, then those (e.g. blurbs) that should either be deleted or rewritten can be identified.  Staff reviewers will also improve their own writing based on feedback.

2 replies

(Just as I'm praising Allmusic editors about bios, I come across an example that can benefit from this idea.)  The O'Kanes was moderately rated 3 stars with a review blurb that quipped, "Introspective lyrics and occasional guitar/mandolin jams make an interesting concept".  In contrast, the average user rating is much higher at 4.5 stars.

Since I'm unfamiliar with that album, I'm puzzled by the ratings contrast.  My puzzlement becomes clear upon reading the bio though, which embedded a full review of that album.  The full review praised:

The self-titled, first, and arguably strongest effort contains everything that is best about the O'Kanes' sound. It is rich in country music's finest traditions, yet it is by no means a nostalgia album. It is sparse in instrumentation, yet richly textured. Most of all, the album contains direct, honest music whose emotional intensity stays with the listener long after the sound waves have stopped vibrating. The O'Kanes' vocals recall the best of country harmony. Some critics liken them to the Louvin Brothers. Others, because of the more driving sound of their backing, compare them to the Everlys. The instrumental sound ranges from bluegrass (prominent mandolin) to the tense drive of Sun rockabilly (their hit "O Darlin'" is evidence of this). The addition of an accordion adds both Tex-Mex and unmistakably bluesy feels to the proceedings. This is truly hybrid music.

By implementing this idea, users who have rated that album could have also voted its review blurb as "unhelpful".  If Allmusic flags editor reviews that receive at least 10 unhelpful votes, then it can decide whether it concurs with the unhelpful votes.  If it concurs regarding this review blurb, then the review blurb can be easily replaced by the biography's full review.

I understand the logic here, but there are two likely issues.

One is that we already get a lot of feedback from users who are upset when a reviewer criticizes their favorite artist and I could see this being a tool that would generate a lot of noise in that area.

Two is that the team of editors is already trying to keep up with the current onslaught of new albums, digital releases and creating new biographies (and biography updates) for newer artists and albums, they would be unlikely to put that aside to rewrite a 1999 review of a John Herrmann / Tim O'Brien / Dirk Powell budget-line recording.

We do have the ability for users to contribute their own reviews, so if a fan feels strongly that an album has been underserved, they can contribute their own take on the record, and we do occasionally pull a list of the albums that have wildly different editorial ratings and average user ratings to see if a high-profile older review should be revisited.

Zac, I don't believe fanboys will misuse a helpful/unhelpful rating on the staff review because they can more passionately criticize by contributing a user review.  Such a rating can only benefit staff reviewers.  If such a rating proves not beneficial, what's the loss?

(I'm not expecting reviews to be rewritten for old albums.  I'd actually prefer to delete an unhelpful review.)  My primary interest in this idea is as quality control that can deter lazy reviewers and identify problematic reviews (e.g. reviewers who shouldn't be qualified to review a genre).

LT

I have the same concerns as rootsmusic as well.  There are a couple users that have accumulated thousands of reviews that are very terse and lacking of detail.  Definitely an issue of quantity over quality with legitimate concerns over whether the reviewer actually took the time to listen to the whole album.  One person that I have in mind (that I will not identify in a public forum) has so many unhelpful review ratings from other users in a particular currently non-mainstream genre that random individuals on outside forums have asked about what his deal is.

Do you all take email feedback over some of these user reviewers that are leaving so many reviews in bad faith?