REVIEW SCORING: STANDARDIZATION, PROFESSIONALISM, ETC.

author=Fallen-Griever

I expect to be judged according to the standard I attempt to meet.

This is so ridiculously important it's not even funny. You should always, always, always write a review based on the aims and intent of the developer.

Technically, this is part of the important difference between "criticism" and "reviewing". "Criticism" is designed around doing this; "reviewing" is not. Reviewing ignores artists' intent, and is all about making a recommendation to the consumer about whether or not something will be entertaining.

This is pretty out there, but what if we allowed both critiques and for games? Critiques would have NO NUMERICAL SCORE at all (not even one disguised as a string) while Reviews would be all about awarding a numerical score via a standardized rubric.

I don't see the value of a lot of suggestions I've seen thrown a round, for that matter. It seems to me like we're just spitballing here.

Someone suggested banning scores entirely. I can get behind that...except for the fact that it will make another relatively arbitrary factor (page presentation) the deciding one in which games get played.

My point is, there's no difference between a star system and a word system when the star system explains it's own terminology to the person using it.

This is also true.

Great graphics are only a good thing if the player values great graphics. A player that doesn't value great graphics will not be influenced by that when rating something.

All reviews are subjective.

They would if we had mandatory subscores like some sites. If we added a graphics subscore, that would force them to think critically about the graphics. (Of course, there are many advantages to the 'wide open, choose your own style' approach that RMN takes.)

I personally don't think they should be compared to professional games. There is a huge difference between a game one guy made that is FREE for everyone to download and a 60$ epic Xbox 360 game made by some huge company. If someone is charging for their RPG Maker Game it's one thing to compare them to professional games since it costs money, but other than that its pointless to compare in my opinion. If you do that, literally every game on this site (mine included) would be a one star/five because no matter how biased you might be to a game, it's not gonna hold up when compared to Mass Effect or something like that.

I think the pro/con tags are a pretty neat idea. Especially if there were a broad set of common tags that could be searched on. Hasvers is right about the Misaos, but we could at least limit the number of pro tags and limit the number of con tags. It bears further exploration.

(I'm not sure I would require writing a full review to attach them to - I've contended before that one of the major problems with using any of the numbers from reviews around here is low sample size, so I'm always kind of on the lookout for more streamlined ways for people to leave information for players...)

Uncle Pennybags is right.

People who write negative reviews of RM games mean to do one of the 3 things.

1. Discourage the creator.
2. Discourage users from playing the game.
3. Give suggestions and criticisms to help the creator to improve the game and their game making skills.

This is the most interesting post anyone has made, I think. It's also somewhat flawed! Reviews can also INTEND to genuinely advise consumers (site visitors) on whether or not they should play a given game. As a matter of fact, AT MOST SITES where ANYTHING is reviewed, this is the primary purpose of reviews. They can also actually accomplish this. I imagine that all but the most douchebaggy reviewers INTEND to provide site users with a play/don't play recommendation and HOPEFULLY provide some suggestions to the creator. However, regardless of reviewer intent, it IS accurate to say that a negative review can these three possible impacts.

Of these impacts, the first and second are the most important. And I primarily include the second because of its impact on the first.

If any one of you would tell me, "I am not the slightest bit discouraged by negative reviews, and they don't negatively effect my willingness to continue making games" I would NOT call you a liar. Some of you are extraordinarily humble and thick-skinned &c &c, a very desirable quality.

However, if EVERY one of you were to tell me "I am not the slightest bit discouraged by negative reviews etc" I would make the statement that at least half of you are liars.

The truth is, game design is HARD WORK and we all don't do this purely for fun or in pursuit of some artistic zenith. Praise, recognition, and respect are all important motivating factors--factors that motivate the creators of games to create games.

A system that is biased in favor of positive reviews is weighted to encourage the creation of games.

A system that is biased in favor of negative reviews is weighted to discourage the creation of bad games.

(Surprisingly, if the current site stats indicate the average review score is 3.15, then RMN is actually the former.)

Believe it or not, people do learn from the very act of making games, not only from the feedback and criticism they receive. The games I made in my twelfth year of game making are five times better than the ones I made in my second year, not just because people explained to me all the things that were wrongtarded with my earlier games, but because I have ten more years of experience. Therefore, for me personally, it is more important to encourage the creation of games than to discourage the creation of bad ones. I want games to be made more than I want bad games not to be made.

Several of you disagree with me very much, which is fine.

This is my final thought for right now. In my opinion, the advantage of reviews that err on the side of generous is that they encourage the hobby, and they encourage those who participate in it. In the absence of bribes from corrupt game publishers like in the commercial arena, this is the force that encourages the average review score being 3, rather than 2.

i thought everyone went by the 2.5 = average rule. are we really getting so many unjustifiably high/low scores that we need to change the system? granted, sai's reviews are apparently very harsh (2.5 being such a well-deserved score 'very few games would make the cut'...:|), but i thought most reviewers had a good standing on what an average RM game is, to not only them but the rest of the community.

if it seems like a review/score is too extreme why not allow people to say 'hey i disagree, and here's why...' we're not babies, we can take it. i should think the reviewer has enough of a basis for his reasons to back up the review without having a breakdown - if not, then you can blame bias and blame the reviewer, don't blame the system(wish that didn't sound so corny haha). if someone wants to tear a game apart they will do it rating or not.

in regards to having words instead of star rating - i thought this:
Terrible -> Very Poor -> Poor -> Below Average -> Average -> Above Average -> Good -> Great -> Outstanding -> Perfect (granted i would change outstanding to excellent and perfect to exceptional/must-play or something)

was basically represented in the stars anyway so there is pretty much no point in substituting one for another if that's what people are suggesting. the same goes for a grading system to be honest, it's all basically the same thing. can't we just follow kentona's guide to being a good reviewer haha.

author=Max McGee

author=Fallen-Griever
If we want to be taken seriously then we should definitely compare our games to professional/indie games when applicable

F-G I love you baby but the inverse of this is that maybe if we want to be taken seriously we shouldn't SELF-RATE in such a way that the average game rating is 1.5 or 1 or some shit. Self-deprecation only goes so far before it becomes infuriating instead of ingratiating. If the message from our reviews is 'we suck, we suck, we really really suck' then some people can't be faulted for believing us. It is insane to have the '5 Star' rating mean 'an excellent professional game made by hundreds of people over several years by a professional studio' when almost every game on here is a one-man show.

I personally see absolutely no value in harsh reviews.

Of course, the use of the word professional in the topic title is itself highly arguable. This post will expand over time.

A 2.5/5 should be an average game regardless of who made it or what engine it was made in.

The man speaks sense.

Personally I feel that if we don't hold ourselves to the same rating as a standard game we are more or less lying to ourselves which hurts more than helps our creativity as game developers. If you make games for fun, good for you, but don't expect to be given special treatment because your game isn't supposed to be a blockbuster in the indie gaming world. The more highly you think of yourself, the less likely you are to learn and vice-versa

LR, I'd have to disagree with that, but I'll counterattack your point with Max's, since I don't feel like typing out why.

Me saving space:
we've retreaded the same 3 arguments like 10 times in 5 pages, each time coming to a standstill. If you're looking for comments that are wasting time and space, look no further than your own.

errr no it doesn't work that way - you think nicoB didn't put any effort into his game? saying 'people should make better games,' isn't going to help or change anything...

Question is, how can you communicate to a person the things they need to improve on while giving them a 5/5? Same as rating a game a 0.5/5.

There's been talk about what the system does just fine when not marred by the harsh or lackadaisical reviewing standards of a few. But what does the system do for the game creator?

Well, most of the time, the game creator cares only about the star score. They're happy with every 5-star review, and angry with every 1-star or less. This is natural, they want the game to do well because they worked hard on it. But sending a message to a game creator that they need to improve a lot of things in a 1-star package doesn't inspire, it angers.

I personally don't care for scored reviews. Any metric system that doesn't require a full review, like thumbs up, Like, what have you, is just going to be abused to affect visibility. We could never have a review system like professional magazines either. We're not professional reviewers, mostly because there is 0 accountability for the things we write. And imagine how well it would go if we left overall scoring to a staff panel.

But how do we deliver the best content to site visitors? If we wanted to implement a system that promotes impartial feedback to game creators in an attempt to truly make their games better, we could probably do that. Game creators would probably be more responsive in that field as well, when harsh criticisms are not affecting the visibility of their game. But the problem remains with directing visitors to the best games.