ARE WE TOO GENEROUS WITH GAME REVIEWS?

Posts

ESBY
extreme disappointment
1238
i think we need a new scale of rate games, let's rate them from esby to ESBY
I give this game an ESby out of ESBY
Solitayre
Circumstance penalty for being the bard.
18257
Reviews are based on individual preferences. There's no real 'we' here. I try not to arbitrate what scores people are 'allowed' to give. That isn't my job.
author=Solitayre
Reviews are based on individual preferences. There's no real 'we' here. I try not to arbitrate what scores people are 'allowed' to give. That isn't my job.

The post is asking if you personally feel if the community as a whole, generally is too generous with reviews. Meaning that you feel people give a higher review than what they actually feel the game is worth, or not. Whether or not it's your job has nothing to do with anything.
author=ESBY
i think we need a new scale of rate games, let's rate them from esby to ESBY
I second that.
Sooz
They told me I was mad when I said I was going to create a spidertable. Who’s laughing now!!!
5354
author=PentagonBuddy
I went through the list of full games, sorted them by score (from 5 to 0.5), and did some counting. I counted 1572 reviews scored 5 to 0.5 and here were the results:



This was very informal so I would not be surprised if my numbers are a little off.

Only 28.3% are scored 4 stars or higher, and 48.16% lie between 2.5 and 3.5 (so a little below average, average, and a little above average)

That being said, the amount of 3, 3.5, and 4 star reviews are very similar, so I could see why it might seem that 4 star reviews are especially common.

Most games have 1 review or less, which means that the opinion of 1 particular reviewer can determine...well, the entirety of a game's score, if it even gets one. I think people who have strong opinions are more likely to bother contributing a review, too. (These are numbers I found out previously but am too lazy to get the spreadsheet atm...)

Personally, I tend to leave unrated reviews, but this is something I'm trying to change when I review a game with no previous reviews/I actually have feelings about some kind of score. It's kinder to the developer's makerscore (though probably not their feelings if it's a low rating).


God bless you, Penta, and your extreme number nerdery. <3
Corfaisus
"It's frustrating because - as much as Corf is otherwise an irredeemable person - his 2k/3 mapping is on point." ~ psy_wombats
7874
author=Pancaek
author=Solitayre
Reviews are based on individual preferences. There's no real 'we' here. I try not to arbitrate what scores people are 'allowed' to give. That isn't my job.
The post is asking if you personally feel if the community as a whole, generally is too generous with reviews. Meaning that you feel people give a higher review than what they actually feel the game is worth, or not. Whether or not it's your job has nothing to do with anything.

It comes with experience and a certain "I've got my opinions, I've averaged them out and I don't care what others feel about my opinion" attitude. I won't be scared away from giving a game I've played through to the end a 1-2 star review if it has little to no redeeming qualities. Of course, I've been convinced to remove my rating because the game in question was a demo when I reviewed it, but that's out of fairness and not from community backlash.

I give only what I think a game is worth, and I'm very meticulous in deciding on the score I feel like giving.

And the review for your game that you brought up averaged out to 3.25, so I'm not sure why the reviewer went with 4 instead of 3.5, thereby rounding up twice.
Another thing to consider is experience when reviewing. How many games has that person played before? How many Good/Bad games have they played before? How fresh are the ideas that that particular game has presented to that reviewer?

If someone's played a lot of games they're going to have a bit more of a defined curve than someone who has played very few games, and it also depends on what they expect out of a game made in the engine. I've seen reviews on Steam about RM games that praised the 'unique' graphics and awesome gameplay - games which wouldn't even get past our standards. I've also seen reviews condemning games for certain aspects that can't be helped if you're using RPG Maker to create a game (for example, slower battle speeds in 2k3 games) - things that someone who knows the engines would allow to slide because they understand that the creator of the game doesn't have as much control over that aspect.

There is, of course, also the aspect of people writing about games they enjoy, as stated before.

Honestly, I'm not afraid to give harsher scores but I'm a bit of a softy so if I see that someone has tried hard, I'll give them a decent enough score as long as nothing is too bad with the game. Then again, I've played some truly terrible games in my time, so my experience is skewed towards .5 being almost unplayable and littered with bugs, 3 to being decent enough and 5 being impossible to reach unless it's highly polished and very enjoyable (I don't give 5s often).

There's also the fact that we do ask people to make a case for why they give the scores they give - which is a lot more than other sites do. This makes the writer think about the different aspects of the game whilst writing and they might go into the review thinking of giving one score, but reminiscing has helped them get a clearer image and change the score. There's been a few times I've went into a review going to give a 3 and changed it to 3.5, or vice-versa.



Also, for future reference, please don't double post. Instead, use the handy edit button below your previous post and add more to it. It clutters the topic up and is against site rules. Thanks! >.<)b
ESBY
extreme disappointment
1238
I think the admission that reviews, and therefore scores, are entirely dependent on individual preferences lends credence to the idea of there being some sort of guideline on what each score actually means or removing the star ratings in some form, perhaps just from the front page of a game. I don't see the point in having star values if the values themselves don't have any merit.
We do have a sort-of reference to what scores mean. When you pick a score it has a descriptor next to it...



We've talked about getting rid of stars so many times that it's stupid. We end up deciding to keep them each and every time because there's no good reason to get rid of them. People bring up the same arguments that keep getting shot down because they're not good arguments to do so.
Mirak
Stand back. Artist at work. I paint with enthusiasm if not with talent.
9300
Speaking as a normal user, I think the stars are a good visual cue. It's easy for me to quickly know if a product is worth my time with just a glance. If i'm curious enough (and i'm almost always are), i'll check out the reviews if the score is low, since i want to know why they think the game deserves that score.
Vaccaria
You'd think MZ would use a dictionary for switches/variables by now?
4936
There's no such thing as being 'too generous'. Also, I got whiplashed by reviewing a game despite it being my own opinion (and as a reviewer, the critical analysis). Still, you can call it a review if it combines critical analysis alongside fairness and your opinion. If it is not, it's called a mood whiplash towards a game that you hated or unfair praise towards one.
author=KatanaHiroshi
There's no such thing as being 'too generous'. Also, I got whiplashed by reviewing a game despite it being my own opinion (and as a reviewer, the critical analysis). Still, you can call it a review if it combines critical analysis alongside fairness and your opinion. If it is not, it's called a mood whiplash towards a game that you hated or unfair praise towards one.


Perhaps generous wasn't the right word to use. I more or less just meant: "Do some people give better reviews than what they actually think the game is worth just to be nice?"

Although that'd be one hell of a title.
Problem for me is that if a game is so bad that I won't even give it 3 stars, then I'll just not play it. And it feels unfair giving a bad review without having played the game.
author=RyaReisender
Problem for me is that if a game is so bad that I won't even give it 3 stars, then I'll just not play it. And it feels unfair giving a bad review without having played the game.


...How would you know a game is that bad if you haven't played it?

Granted it is pretty easy to spot a low quality game, but at the same time it's hard to really accurately judge a game if you haven't played it. Maybe the art is really bad but the writing and game-play is outstanding.
Most people don't feel it proper to review a game they've only played a bit of and then stopped because it was so damn shite.
Ratty524
The 524 is for 524 Stone Crabs
12986
author=Shinan
Most people don't feel it proper to review a game they've only played a bit of and then stopped because it was so damn shite.

I don't blame 'em since it seems to be the #1 means to stir up drama in this community.
author=Pancaek
author=KatanaHiroshi
There's no such thing as being 'too generous'. Also, I got whiplashed by reviewing a game despite it being my own opinion (and as a reviewer, the critical analysis). Still, you can call it a review if it combines critical analysis alongside fairness and your opinion. If it is not, it's called a mood whiplash towards a game that you hated or unfair praise towards one.
Perhaps generous wasn't the right word to use. I more or less just meant: "Do some people give better reviews than what they actually think the game is worth just to be nice?"

Although that'd be one hell of a title.


Giving extra points on a review just to be nice isn't much of an issue in this community, and those who go out of their way to give, say, 5 stars to a game that is clearly 2 stars get called out pretty fast.
Isrieri
"My father told me this would happen."
6155
These threads that keep popping up on the sanctity of reviews make me never want to review anything.
(EDIT: I said that to make a point, but it makes me sound like I'm hating on the thread itself. Not my intention)



Its like, "Okay, do you guys want a Zero Punctuation style 5 minute video breakdown of your game? Do you want an animated mobius strip? Shall I throw in a couplet? I'm guessing just a few pictures and italicized captions is all most folks are looking for. But then, are the stars really that important? Would you like your review served with a side of caviar?"

There's games I've seen get good scores that I totally disagree with, but if I really thought that was a problem I'd just make my own review. The whole business with reviews on this website is that they're not done by reviewers, just the people who've played it. Its a populist thing we've got going on, and I think most of the site understands that.

Here's an idea: Have our biggest reviewers (like, the ones who make it onto the monthly stats) be given a "RMN Seal of Quality" or whatever. Problem solved forever.
author=Pancaek
author=RyaReisender
Problem for me is that if a game is so bad that I won't even give it 3 stars, then I'll just not play it. And it feels unfair giving a bad review without having played the game.
...How would you know a game is that bad if you haven't played it?

Granted it is pretty easy to spot a low quality game, but at the same time it's hard to really accurately judge a game if you haven't played it. Maybe the art is really bad but the writing and game-play is outstanding.

Some games just look so bad or have such a horrible font / interface, that no matter how good their writing is, you just know you won't like it.

You can also often judge a game by its feature list and reviews already written. If a game praised itself or is praised because of one feature of which I know that I'd absolutely hate it, I also don't bother downloading it.

Finally there are many games you try and after 5 minutes just realize how bad there are. Even here you don't feel like writing a review about it, because you couldn't give much insight in the first place.

These are the only games I'd rate 1-star and 2-star.

Games that fascinate me so much that I actually finish them are worth at least 4 stars.