ARE WE TOO GENEROUS WITH GAME REVIEWS?

Posts

Mirak
Stand back. Artist at work. I paint with enthusiasm if not with talent.
9300
My reviews are mostly walkthroughs with my impressions on what i found enjoyable or shitty. If people read my reviews before playing the game they'd be very very spoiled. I usually try to warn readers beforehand, but yeah, i don't recommend people reading reviews on this site before playing the games, i do recommend considering the star rating.
Sooz
They told me I was mad when I said I was going to create a spidertable. Who’s laughing now!!!
5354
author=Ratty524
@Sooz: You're completely misrepresenting what I said. I recommended abolishing the star-rating system, not reviews themselves. A scathing review of a game has the same kind of impact as getting a low score, only with the review by itself, it could potentially get people to pay more attention to the reviewer's opinions on a game before trying it rather than going "DURR it has less than X stars imma not play it."

Your argument is "we should abolish this thing because it discourages people who can't deal with criticism."

Again, if someone is posting their work for public consumption, they NEED to be able to deal with the possibility that their work is not, in fact, ready for prime time.

Also holy jeez if you have to resort to making people look stupid when they make their decisions based on a score, you suck at arguing. I mean yeah I tend to feel a score is less reliable than looking at the review, but I don't feel like I need to denigrate someone who uses just the score as slow in the mind or some shit.

ETA: I think I've thoroughly covered my opinion on this, and it's not especially conducive to humor, so I'm out.
author=Sated
author=Solitayre
author=Sated
We're too generous with the standard applied to reviews that are allowed onto the site.
Why should someone have to aspire to professional review standards to be allowed to write a review?
Why would I want professional review standards? That would mean *my* reviews wouldn't get accepted.

In fact, there probably wouldn't be any reviews...

nobody could AFFORD them, amirite?

(geddit? because all professional reviews are paid for)
I'm in the "keep the star-rating" camp. If someone is looking for a game they don't want to spend more than an hour reading reviews for games they are not gonna play anyway before finally finding something that might be good. Not wanting to waste gamers time beats hurting some snowflakes feelings every day in my book. The games aren't here for the developers, they are here for the players. If you don't want your feelings hurt don't put the game up for public consumption.

It's also good for the site if players can find games quickly, because they won't get irritated by the shitty system in place and might come back for more. People are lazy and want things fast, and that's not gonna change anytime soon. And the star system is a good quick indicator on playability.
ESBY
extreme disappointment
1238
i think the answer is to be able to review reviews
You mean like comments on reviews?


Well have I got news for you! :DDD
ESBY
extreme disappointment
1238
no, i want to be able to leave reviews on reviews and to give them a star rating
Kloe
I lost my arms in a tragic chibi accident
2236
author=ESBY
no, i want to be able to leave reviews on reviews and to give them a star rating

I second this!!
Ratty524
The 524 is for 524 Stone Crabs
12986
author=Sooz
author=Ratty524
@Sooz: You're completely misrepresenting what I said. I recommended abolishing the star-rating system, not reviews themselves. A scathing review of a game has the same kind of impact as getting a low score, only with the review by itself, it could potentially get people to pay more attention to the reviewer's opinions on a game before trying it rather than going "DURR it has less than X stars imma not play it."
Your argument is "we should abolish this thing because it discourages people who can't deal with criticism."

Again, if someone is posting their work for public consumption, they NEED to be able to deal with the possibility that their work is not, in fact, ready for prime time.

Also holy jeez if you have to resort to making people look stupid when they make their decisions based on a score, you suck at arguing. I mean yeah I tend to feel a score is less reliable than looking at the review, but I don't feel like I need to denigrate someone who uses just the score as slow in the mind or some shit.

ETA: I think I've thoroughly covered my opinion on this, and it's not especially conducive to humor, so I'm out.

Nice try. I never said that we should abolish star ratings because they hurt feelings. I'm fully aware of the value of criticism and how thick-skinned you need to be to survive in this shtick and especially the industry. Do you need a stupid icon stuck to the corner of the page to tell you that?

My concern about the star-rating system doesn't even go towards protecting feelings, more than it is about getting people to try games. Most of the time, when a game gets a low total star rating, people don't even comment or give a second look to games, and receiving no feedback at all is a hell of a lot worse than negative feedback.

Also, you should consider the chances of a complete dependency between score and the content of reviews themselves. I've seen, and even written an early review where the tone you are describing the game with doesn't match the score given. I'm considering this as an option to alleviate abuse.

Finally, I'd like to mention that you really aren't in a better boat than I am in holding an argument if you can't even be bothered to actually read and pay attention to what the other side is saying, let alone just completely cop out of the discussion as you just did, but I rest my case for now.
in a study I read about they mentioned that any rating at all encourages people to try things than having no rating system. The conclusion was, having a rating (even a shitty rating) lead to more clicks overall in the system than having no ratings. It had something to do with human biases kicking in subconciously, in that seeing that someone else had taken the time to check it out implies that it might be worth my time to check it out.

Basically "someone took the time to assign value to this. I should dig deeper". I can't remember the study or the name of the effect.

Personally, this is how I behaved when I encounter an undifferentiated morass of free games - I just left the site. Picking games at random wasn't worth the effort.
Ratty524
The 524 is for 524 Stone Crabs
12986
author=kentona
in a study I read about they mentioned that any rating at all encourages people to try things than having no rating system. The conclusion was, having a rating (even a shitty rating) lead to more clicks overall in the system than having no ratings. It had something to do with human biases kicking in subconciously, in that seeing that someone else had taken the time to check it out implies that it might be worth my time to check it out.

Basically "someone took the time to assign value to this. I should dig deeper". I can't remember the study or the name of the effect.

Personally, this is how I behaved when I encounter an undifferentiated morass of free games - I just left the site. Picking games at random wasn't worth the effort.

That's an interesting case. I mean, my suggestion on no ratings was an initial thought more than anything to get consumers to pay more attention and alleviate abuse tactics, but I can see how that could backfire based on the study you mention.

Part of me feels like the best way to revise our review system is to have a "like it";"meh"; or "dislike" type of singular rating system similar to what GameXplain has with their recent reviews. It would bring a lot more clarity with reviews as a system like that wouldn't be left up to interpretation as much as a numerical star rating.

Yes, Steam and Youtube has a similar rating system, but the core difference is that Steam has absolutely no regulations/quality filter with its reviews while we do. I think this change is doable.
author=Ratty524
That's an interesting case. I mean, my suggestion on no ratings was an initial thought more than anything to get consumers to pay more attention and alleviate abuse tactics, but I can see how that could backfire based on the study you mention.

Part of me feels like the best way to revise our review system is to have a "like it";"meh"; or "dislike" type of singular rating system similar to what GameXplain has with their recent reviews. It would bring a lot more clarity with reviews as a system like that wouldn't be left up to interpretation as much as a numerical star rating.

Yes, Steam and Youtube has a similar rating system, but the core difference is that Steam has absolutely no regulations/quality filter with its reviews while we do. I think this change is doable.

The problem with that is that basically any game would be rated "all good" "all bad" or "meh" since most games here don't get more than one or two reviews.

That system would work if games typically got a lot of reviews, however because of the size of this particular community, it wouldn't work as well. If a game only gets one review, that'd be the deciding factor between if the review score was all good or all bad. There'd be no way to distinguish kind of good games from really good games, and vise versa.

You could argue that people could just read the reviews, but as other users have already stated, a lot of people don't like to read reviews until after they've played the game to avoid spoilers.
I read most of this study I found: Quantifying Social Influence in an Online Cultural Market
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3348939/#pone.0033785-Salganik1

while I don't think this is the study I read years ago, I think it references it. Might have been Experimental study of inequality and unpredictability in an artificial cultural market. It's behind a paywall, though.

The studies focused on Download Counts as the social indicator, but I would bet similar behaviours would arise for ratings. The gist I remember is, visible social indicators (download counts, ratings, views, etc) increase inequality but also increase absolute numbers of samples/downloads overall in the system.

The takeaway I got from that was, while higher rated and downloaded games would get increasingly more downloads proportionately while less popular games would get fewer downloads proportionately, both the popular and unpopular games would get more absolute number of downloads than a system where such social indicators are absent.

The other takeaway is that such social indicators are NOT the first or even second consideration when someone decides to download something. Availability is (ie- featured, its position in a list or page, visibility). Then its peer-to-peer influence ('Solitayre liked it!') and marketing forces (hype!).



tl;dr - ratings aren't as important as we're collectively assuming they are, and the mere existence of ratings probably increases the absolute number of downloads on RMN
Ratty524
The 524 is for 524 Stone Crabs
12986
author=Pancaek
author=Ratty524
That's an interesting case. I mean, my suggestion on no ratings was an initial thought more than anything to get consumers to pay more attention and alleviate abuse tactics, but I can see how that could backfire based on the study you mention.

Part of me feels like the best way to revise our review system is to have a "like it";"meh"; or "dislike" type of singular rating system similar to what GameXplain has with their recent reviews. It would bring a lot more clarity with reviews as a system like that wouldn't be left up to interpretation as much as a numerical star rating.

Yes, Steam and Youtube has a similar rating system, but the core difference is that Steam has absolutely no regulations/quality filter with its reviews while we do. I think this change is doable.
The problem with that is that basically any game would be rated "all good" "all bad" or "meh" since most games here don't get more than one or two reviews.

That system would work if games typically got a lot of reviews, however because of the size of this particular community, it wouldn't work as well. If a game only gets one review, that'd be the deciding factor between if the review score was all good or all bad. There'd be no way to distinguish kind of good games from really good games, and vise versa.

You could argue that people could just read the reviews, but as other users have already stated, a lot of people don't like to read reviews until after they've played the game to avoid spoilers.

The star rating system doesn't really do better, since the way it's currently set up is that it pretty much averages based on the total reviews. If a game only gets one review, the star rating it receives reflects just that one review.

I just feel like the alternative I proposed would be more concrete. I'll take a look at the study kentona linked later, though.
Mirak
Stand back. Artist at work. I paint with enthusiasm if not with talent.
9300
How about two star systems, a faster one where people can rate the game without reviewing, and a more prominent one that averages based on total reviews?

After downloading a game, maybe a notice of some sort could remind the player to rate and review the game. When you favorite artwork in deviantart a little speech bubble pops up above the author icon that says "don't forget to leave a comment" so maybe something like that.
Marrend
Guardian of the Description Thread
21781
Rate without needing to review? I think an idea like that was kicked around before under the label of "Impressions", or similar term? I can't remember. I've seen myriads of conversations about how the site should handle reviews, and sometimes, I'm not quite sure what changes people are asking for.
To answer the OP: Yes. People here tend to be too 'generous', and this is something you'll notice everywhere not just in reviews. It ties to the "Hobbyist" mentality that permeates the site. People are more interested in exchanging pleasantries "This rocks! You're awesome!" than to be critical and really analyze things "What does this aims to be? How can it be improved?" etc.

From time to time this community will send you mixed signals. They'll wonder why we're not taken seriously, what can we do to uplift games and developers. But the moment you suggest something, they'll quickly revert to their "But we're amateurs. We just want to have fuuuuun" mindset, and shot down any ideas you bring up no matter how easy they're to implement.

Give up trying to change the review system. It's not going to happen. Not in any meaningful way. Not any time soon at least...
author=ESBY
no, i want to be able to leave reviews on reviews and to give them a star rating


Yeah, Steam actually sort of does this (except for the star rating: you can rate whether a review is helpful or not).
Yeah, and it's abused to fuck and back in order to get rid of reviews the game devs don't like. There's a lot of things wrong with steam's system - namely that there's just so many users that they don't bother enforcing any kind of rules at all, so it's very very easy to abuse.


Honestly, before we had numbers added to a game page, how many reviews/downloads/etc were we getting - vs the amounts we get now? I'm gonna go out on a limb and say we get a hell of a lot more now than before. Why? Because ease of knowledge is encouraging. Sure, you get a few games (and don't kid yourself, it's a few) that slip through the cracks when it comes to getting a fair(er) score, but for the most part the people here do a decent job of rating games.

Personally, as a developer who was around before there were star ratings, I much prefer them. I've a few reviews that have no score attached to them at all and whilst it's nice to know the thoughts behind the review and how they liked the game, having a hard-coded 'score' - especially an average - is a lot better to me than not having one. Because that encourages players to check the game out.

If you want another review to 'be fairer', all you have to do is ask - there's a lot of reviewers on the site and doing a quick look through the review section gives you a good idea of who to ask for one.

Besides, not getting a good score pushes people to actually, idk, improve their shit? Yeah.