REVIEW SCORING: STANDARDIZATION, PROFESSIONALISM, ETC.

Alphanumerical. HELLO? WHY ARE YOU ALL IGNORING THIS?
lol.
I would keep the stars. Just make different reviewing "rules."

No, it's combining them.
Like grading a paper.
98% A+
or
42% F
You are wrong :P
Maybe I mean like a grading system, unless you people in Canada do something else.
(Probably 1-5)

author=kentona

author=Sailerius
@kentona: I'm still in favor of removing stars and replacing them wholesale with thumbs up/thumbs down, representing whether or not you would recommend the game. There was a lot of momentum and support for this idea.

There has been a lot of support against this idea as well!

For ranking purposes, an average star score is a much fairer system than a Thumbs up/down, as an average score is less influenced by popularity. (as Max has pointed out, we have plenty of metrics measuring popularity already).

Also, I just don't see how extreme loss of granularity is a good thing? The distinction between great and good is really important, as is the distinction between bad and terrible. Most important, though, is the idea that a game can be average. Why reduce things to a boolean?

From most granular to least granular, the possible metrics I can think of are:

* Qualitative (i.e. textual review only)
* "Alphanumeric", i.e. a grade from F to A+. I think this does actually have some merits, because the connotation of assigning a game a FAILING grade is extremely obvious, and might be easier to get people to agree to, unlike what a '1 Star' review means. Then again, maybe not.
* Rating out of 5 Stars. What we have now.
* Boolean. Thumbs Up/Thumbs Down. By far the least granularity....someone HATING a game or LOVING it has no more weight than someone 'liking' or 'disliking' it.

I haven't included things we haven't considered because they'd be mostly arbitraty cosmetic changes, like having a score out of 100.

author=Max McGee

author=kentona

author=Sailerius
@kentona: I'm still in favor of removing stars and replacing them wholesale with thumbs up/thumbs down, representing whether or not you would recommend the game. There was a lot of momentum and support for this idea.

There has been a lot of support against this idea as well!

For ranking purposes, an average star score is a much fairer system than a Thumbs up/down, as an average score is less influenced by popularity. (as Max has pointed out, we have plenty of metrics measuring popularity already).

Also, I just don't see how extreme loss of granularity is a good thing? The distinction between great and good is really important, as is the distinction between bad and terrible. Most important, though, is the idea that a game can be average. Why reduce things to a boolean?

From most granular to least granular, the possible metrics I can think of are:

* Qualitative (i.e. textual review only)
* "Alphanumeric", i.e. a grade from F to A+. I think this does actually have some merits, because the connotation of assigning a game a FAILING grade is extremely obvious, and might be easier to get people to agree to, unlike what a '1 Star' review means. Then again, maybe not.
* Rating out of 5 Stars. What we have now.
* Boolean. Thumbs Up/Thumbs Down. By far the least granularity....someone HATING a game or LOVING it has no more weight than someone 'liking' or 'disliking' it.

I haven't included things we haven't considered because they'd be mostly arbitraty cosmetic changes, like having a score out of 100.

I would personally support a purely qualitative system so that readers are forced to read the reviews instead of just scanning the bottom line.

EDIT: The point of thumbs up/down isn't to say whether you love or hate the game, it's to say whether or not you would recommend it. Ciel said it best:

author=Ciel
mediocrity is not really a problem because like i said it all comes down to whether you are willing to recommend that others play the game or not. they will either play it or not play it. your reputation as a reviewer is at stake! the point of a review is ultimately to tell people if they should invest (time, money) in the game or not. you can't really say 'well i kind of think you should play it but then again you shouldn't 5/10', a reviewer who does that has rendered himself completely useless to everyone. you will eventually have to bite the bullet and tell people if you can recommend it or not. caveats can be expressed in the written portion (which people should be reading) and were never effectively communicated by any form of numerical system anyway.

author=Space_Monkey
boo to thumbs up/down, pass fail, 1-10 with decimals, etc. x/5 stars works fine.

x/5 stars is not going anywhere.

Also, we can't force anyone to do anything. If you think removing a score "forces" anyone to read the review, you are wrong. It's just as likely for them to ignore it completely.

I don't understand what you are trying to say there. We nullify people who just look at the star rating...?

You guys have to understand the differences between reviews here and the purposes they serve compared to those on IGN or in a magazine.

Those reviews do have an ultimate aim of recommending (or not) a game, however people often go there with a game in mind and would be going there to get their opinion before purchasing that game (or maybe to validate their own opinions of the game).

We don't have the benefit of marketing promoting our games outside our sphere of influence. People don't come here with "Hey, I'm interested in Final Fantasy: Awakening of the Warrior, I wonder what the reviews on RMN have to say about it".

The granularity, fairness, and ranking possible with a rated score (say, x/5 stars) fits with what we offer, what we are trying to achieve and minimizes the negative effects of our small sample size.

We offer free unknown RPG games. A granular rating system can aid in searching for games. While x/5 stars is rather arbitrary, it is what was implemented here at RMN. So I can't say specifically why x/5 stars is good, but a rated score of any kind is good.

Since we do have a small sample size for reviews and scores, an average score is a more accurate representation of a game's quality, especially relative to other games on RMN. Whereas a thumbs up/down system would heavily favour popular games, regardless of their quality. A single review with a rating has more merit and worth than 2 thumbs up, 1 thumb down (in my opinion).

Thumbs up/down is more a reflection on popularity (and longevity) than quality, though. And this is more apparent on small sample sizes. I mean, lets say Hero's Realm has 432 thumbs up, and 215 thumbs down. But Generica has 8 thumbs up and 4 thumbs down. What does that tell an end user? To me it just tells me that HR is more popular than Generica. But I can already tell that by looking at the download count.

I fear that a thumbs up/down system will end up pushing more games into obscurity.

If you just had the thumbs up/thumbs down thing there'd be less bullshit.

I feel like there'd be exponentially more bullshit. Because at that point it's LITERALLY JUST a popularity contest.

I don't actually think the review system needs to be changed, I just think that the agreement between review text and review score need to be more carefully monitored, with an agreed upon understanding of what every score means, and with every score accurately reflecting the text of the review it is associated with.

That is my only concern here.

author=Max McGee

If you just had the thumbs up/thumbs down thing there'd be less bullshit.

I feel like there'd be exponentially more bullshit. Because at that point it's LITERALLY JUST a popularity contest.

How so? You still have to write a review to explain why you recommend or don't recommend the game and people can still comment on your reviews. I assume reviews that are just shamelessly upvoting a game wouldn't be approved.