REVIEW SCORING: STANDARDIZATION, PROFESSIONALISM, ETC.

I think at the end of the day, the current rating system itself is fine, and no matter what rules you lay down, people will try their hardest to do what they want, how they want, anyway. If you're looking for more reviews, I think it might be worth considering allowing for some kind of 'short' review, or even just simply the ability to rate it and do nothing else. Whether this is directly incorporated with the existing reviews, or as a separate but similar system, that's a bit open. And I must say, seeing games have 1000+ downloads and not even a dozen reviews is rather jarring to me, as I'm used to reviewing any game/demo you complete as being common courtesy.

From the reviews I've read, a lot of them have several 'me too' comments. Which by their own rights are fine, but they don't end up reviewing or ranking the game themselves. If there was a quicker, simpler alternative, I think pretty much all the people who just make those quick comments would put back a little more feedback into the game, even if it's a little minimalistic. It's at least some way to gauge how many people could at least be bothered to play it through. Of course, then there might be less incentive to do proper reviews. I don't see any reason not to hand out a couple of makerscore for reviews if it'll shore up interest.

EDIT: I think this is more or less what kentona just suggested. Only worded slightly differently. So, um, I guess I'm in favour of it.

As for the human aspect of reviewing, my current philosophy these days is to try and be nice. No matter how poorly it may have been, someone still poured hundreds or thousands of hours into making the game, and hearing it be called a failure isn't exactly heartwarming. Nobody wants to hear that all their effort was basically wasted (or worse), especially when it comes to a hobby-like thing like this, where they're giving up their own free time just to be ridiculed.

Do I have to go to RPG RPG revolution now? The much stricter and serious RPG maker place? The place that topics that go nowhere or have all of these negative responses are immediately closed? (EEW RPG RPG REVOLUTION)
Please do not make me have to! lol
Because a lot of people leave this place for that reason...
Though I plan on staying here for a while.(4ever)
lol Like that matters. lol
In RMN4 can we change the Review Rating style to Alphabetical? Or Alphanumeric?

author=Marrend
"The more things change, the more things stay the same."

How appropriate this is to the review system of RMN? Only time will tell.

Words of a wiseman(or wisewoman?)

I think the things you all are doing for the site are great. The people saying that probably just wonder if you will state what you intend to do in the future with reviews. You probably have not told us because you do not know.

I already made my opinion (not necessarily the staff's) known and was told to settle down. :)

On the other hand, I also detest old reviews present for games that have undergone dramatic changes. Maybe we will come up with some way to change them to "legacy" reviews and not affect the overall score -- problem then is who decides what kinds of changes are major enough to invoke such a status.

I don't have time to read 12 pages of discussion, someone summarize what the problem is and I'll see if I can code a solution, or I'll make kentona code it. (if we had emoticons a sarcasm emoticon would go here)

So, do we write reviews to provide feedback to the game developers? Help players decide which games to play? Or both?

Personally, I don't think they can or should do both. I think reviews should let people on the site know, "is this game worth playing?". We should find a separate outlet to provide feedback to the developers.

thanks for your post homie great job!!!!

somedude: "Lighten up guys!"
RMN: "GTFO!"



Anywho, I have read and re-read this topic, and posted in it and here are my summaries:

1) The review system we have now is serving 2 purposes: a) critiquing the game for the benefit of the creator, and b) reviewing the game for the benefit of potential players. Most reviews tend to fall into one or the other category, and a few fall into both.

2) Even with this ambiguous use, people only get their panties in a knot when someone rates a game that is not in line with the content of the review. For example, Sailerius's review for FE which was rated 0.5 stars (ie- "an unplayable pile of steaming shit with no redeemable qualities") even though the content of the review would suggest about a 1.5 or 2 star rating ("subpar but with some glaring issues"). This disconnect between the review and the score seems to cause the biggest grievance.

3) Review scores are dominant, and this is TERRIBLE! :o



so, here's what I plan on doing:

A) Enforcing a greater standard on review scores. Consistency is an important element of trust in the review's score and average score. To that end, I already have implemented a word associated dropdown for the score when submitting a review.
1 - Poor
2 - Subpar
3 - Average
4 - Excellent
5 - Outstanding
I will also make a new article or page outlining this standard, and make it fairly visible and accessible.

Now, of course what is "subpar" for someone may be "excellent" for another, so the rating is still open to interpretation, but the current rating system we have now works pretty well most of the time and so I have only implemented a minor tweak.

However, hopefully, this will help reduce some confusion or uncertainty for both the reviewer and potential players, and help instill a greater trust in the review score metric.

B) Looking at adding a Rated Comment to gameprofiles. A rated comment would be like a regular comment on a gameprofile, but with the ability to give a Thumbs Up or Thumbs Down. This would be prominently displayed in the comment, and a running total of recommendations (both up & down) would be displayed below the game's average star rating. Also, I would probably implement some sort of filter so that people can quickly see a list of all of the rated comments. Users would be limited to one rated comment per game.

Common complaints/requests about reviewing games on this site:
i) There is too much of an overhead for writing a full review
ii) I want a thumbs up/thumbs down!
iii) There needs to be a quick way to provide feedback that matters!
iv) Too much weight is given to the game's average score

With a rated comment I hope to address these concerns by giving a quick way to give meaningful feedback to a game, and which will complement the review score and gets to the heart of the matter of whether or not you want to recommend a game.

Rated comments do not go through the submission queue, so we would be relying on the memberbase to report flagrantly abusive rated comments (so that they can be removed).

C) Adding a Facebook LIKE to games. This is already implemented in RMN4, actually. You can now Facebook Like, Tweet, Google+1 and whatever-else-kids-are-doing-these-days a gameprofile blahblahblah Web 2.0

I added that in to RMN4 last Friday, and I am considering extending this to Reviews and Articles. The total number of these kinds of "Likes/Recommendations" are shown under the game's star rating.


An alternative solution to all of this was also proposed:

author=cho
novel solution to problem: stop being whiny titty babies when people don't like the thing you like

aka

author=rcholbert
You can change the review mechanisms and scoring displays all you want, but that won't prevent our own inabilities to be fair, helpful, or respectful. None of the proposed changes would be immune to such drama or teach basic manners, such as how to disagree without being disagreeable.

Both solutions have merit. Thoughts?

Personally, I don't think there is anything really wrong with the review system we have now and the 5 star rating system behind it, but for a while I've wanted implement something to supplement or complement it and in the end garner more official feedback for use of both makers and potential players.

Of course, a good compromise leaves everyone unhappy.

EDIT:
Also, we already have a Critique system implemented at RMN. It's called a PM, and you can send one to a maker any time you like. Reviews are for reviewing a game.

Y'know...after giving the topic in question a moment's thought, I looked at what, exactly, is on this site and I've noticed something you all might want to take into consideration: 90% of the games here are made on pirated game engines that are full of copyrighted material, cliches, and poor spelling and grammar.

I do not think that you are doing your statistics right.

A) Enforcing a greater standard on review scores. Consistency is an important element of trust in the review's score and average score. To that end, I already have implemented a word associated dropdown for the score when submitting a review.
1 - Poor
2 - Subpar
3 - Average
4 - Excellent
5 - Outstanding
I will also make a new article or page outlining this standard, and make it fairly visible and accessible.

Now, of course what is "subpar" for someone may be "excellent" for another, so the rating is still open to interpretation, but the current rating system we have now works pretty well most of the time and so I have only implemented a minor tweak.

However, hopefully, this will help reduce some confusion or uncertainty for both the reviewer and potential players, and help instill a greater trust in the review score metric.

I want to read and respond more carefully to the whole big kentonacracy post above (I am still against like/dislike and thumbs up/thumbs down), because it seems very important, but here is a suggestion pursuant to the text I've bolded.

What about the idea of finding five or ten reviews and canonizing them? Maybe two 1 star reviews, two 2 star reviews, etcetera. Selecting and highlighting reviews where the text perfectly agrees with the score? Good examples are good, right? We could also select reviews that are exemplary in terms of writing quality, content, and fairness. That way, we'd be able to say "textually, this is what a 3 star review should be close to".