Full Games List
Featured Games
New Games
Reviews
Media
Images
Blogs
Game Updates
Images Feedback Summary
Game Design Highlights
Tutorials
Articles
Scripts
Utilities / Plugins
Resources
Engines
Engine Bulletins
What is Makerscore?
The Simple Times has started!
Past Events
Achievements
Misaos
Forums
Post History
Chat
New to RMN?
Full Store List
RMN Music Pack (free)
REVIEW SCORING: STANDARDIZATION, PROFESSIONALISM, ETC.
Posts
Versalia
View playlists
Closeauthor=Irili
Reviewing the reviewer? Not a bad idea.
people already do that by viciously ripping apart any review that is of the opposing viewpoint in the comments section
http://www.amazon.co.uk/gp/help/customer/display.html/ref=help_search__1-2?ie=UTF8&nodeId=15891471&qid=1307396704&sr=1-2
Also, an objective review isn't a review, it's a list of observations.
Great graphics are only a good thing if the player values great graphics. A player that doesn't value great graphics will not be influenced by that when rating something.
All reviews are subjective.
Also, an objective review isn't a review, it's a list of observations.
Great graphics are only a good thing if the player values great graphics. A player that doesn't value great graphics will not be influenced by that when rating something.
All reviews are subjective.
why don't we look at this from the other point of view?
When attempting to choose something (be it a game here, or an item on amazon.com or a movie or whatever) what do you look for? Do you read reviews? What kind of reviews? Why that review? Do you care if other people find that review helpful? or do you look at average scores? Or are others more important, like the products description, or some official reward/recognition?
We probably don't have to try to reinvent review systems
When attempting to choose something (be it a game here, or an item on amazon.com or a movie or whatever) what do you look for? Do you read reviews? What kind of reviews? Why that review? Do you care if other people find that review helpful? or do you look at average scores? Or are others more important, like the products description, or some official reward/recognition?
We probably don't have to try to reinvent review systems
I guess my point was that Sal's long half-star review is still, no matter how technical you want to get.. Just a statement of:
"I did not like this"
"I did not like this"
author=NewBlack
http://www.amazon.co.uk/gp/help/customer/display.html/ref=help_search__1-2?ie=UTF8&nodeId=15891471&qid=1307396704&sr=1-2Also, an objective review isn't a review, it's a list of observations.
Great graphics are only a good thing if the player values great graphics. A player that doesn't value great graphics will not be influenced by that when rating something.
All reviews are subjective.
1 star: I hate it
2 stars: I don't like it
3 stars: It's OK
4 stars: I like it
5 stars: I love it
The other thing to keep in mind here, reviews at this site tend to serve 2 purposes. Elsewhere they are rating the final product (be it a game or a pair of shoes). Here reviews also serve as a critique that are meant to help polish or even guide the development of the final product.
Maybe they should be differentiated or split here....?
Blitzen
3253
I don't think there is a problem with the review mechanism we have in place. Reviewers simply need to make clear on what principles are they basing their review, which is something they should be doing anyways.
I would be happy with some kind of auto-averaging system.
See, everyone wants to review as if they're an expert, which is different to reviewing as a player.
I suggest this system:
The reviewer is required to enter an out-of-five-stars rating for (possibly with the inclusion of an N/A choice as to not dilute games like Zero Base with the need to be held to narrative standards and shit):
Production Quality (for the elitist crowd)
Technical Prowess (for the event and script masters)
Writing & Originality (for the pretentious writers)
Aesthetics (For artists and musicians)
Enjoyability (experience as a player and NOT a maker - this is where you take your RM hat OFF)
Then through the amazing power of maths, the rmn system will magically give a score based on individual ratings of those categories. If the person reviewing doesn't think they can comment on a certain aspect because they don't know everything then they only need to give a rating based on "ENJOYABILITY" and the average will just be based on that one input. - In other words the reviewer can decide to go "no comment" on a certain aspect if they don't know about that aspect much without it brining down the review score.
I'd be up for better suggestions for the sub-rating categories... I'm just waxing lyrical here.
This way "I know better than you about xyz - so your opinion based on your own experience of something is WRONG!" will be eliminated.
I don't think it's very good that we have a climate in which this can happen:
"Oh, I thought the writing was pretty good, but that dude there is acting like he's the expert on writing, so I must be wrong.. I can't have enjoyed the writing"
See, everyone wants to review as if they're an expert, which is different to reviewing as a player.
I suggest this system:
The reviewer is required to enter an out-of-five-stars rating for (possibly with the inclusion of an N/A choice as to not dilute games like Zero Base with the need to be held to narrative standards and shit):
Production Quality (for the elitist crowd)
Technical Prowess (for the event and script masters)
Writing & Originality (for the pretentious writers)
Aesthetics (For artists and musicians)
Enjoyability (experience as a player and NOT a maker - this is where you take your RM hat OFF)
Then through the amazing power of maths, the rmn system will magically give a score based on individual ratings of those categories. If the person reviewing doesn't think they can comment on a certain aspect because they don't know everything then they only need to give a rating based on "ENJOYABILITY" and the average will just be based on that one input. - In other words the reviewer can decide to go "no comment" on a certain aspect if they don't know about that aspect much without it brining down the review score.
I'd be up for better suggestions for the sub-rating categories... I'm just waxing lyrical here.
This way "I know better than you about xyz - so your opinion based on your own experience of something is WRONG!" will be eliminated.
I don't think it's very good that we have a climate in which this can happen:
"Oh, I thought the writing was pretty good, but that dude there is acting like he's the expert on writing, so I must be wrong.. I can't have enjoyed the writing"
NicoB
3056
author=Feldschlacht IV
I get the feeling that this topic will ultimately change nothing. Hopefully I'm wrong!
Hasvers
1298
First time taking part in one of these threads (I'm going downhill), nevertheless I like writing reviews on this site so I thought I might comment.
Now this seems sensible. The only reasonable way to rate in my opinion is : if I've had a perfectly good time, I'm willing to give it a 5/5, and I don't care if the game is one hour long or the graphics are awful or it's just a comedy game or there's almost no gameplay or whatever. It must be good at what it tried to do since I'm not bored, and that's the whole point, isn't it ?
If you need to talk about professional games, people love Starcraft and Xenogears for entirely different reasons, the only thing that counts is whether a given game achieves what the devs were striving for, and you know that when you enjoy the game, that's all. There's no such thing as a "universal" or "official" scale, not even according to "professional" game reviewers.
Anyway there's no perfect system and the current one isn't bad at all ; if it causes some people to give a 0/5 or 5/5 just to change the average score, then the same thing will happen with a word system or anything else, and it happens all the time on "serious" sites and in magazines. Any kind of reviewing gives rise to ridiculous results and fads and whatever (have you ever looked at the top movies and games on IMDb or Metacritic ?). And the concept of "good" or "average" is just as relative and hazy as that of 2/5 : "average" with respect to what ? It will always be different due to the expectations and experiences of each gamer.
And frankly, it isn't even that important for the visitor. I have a rather standard procedure, which I think is shared by a lot of people, for determining which game I want to try on a given site. Of course, I go for featured games or those which I've heard about first. For other games,
- first, I do look at the score. I have a threshold : 80% and upward retains my interest instantly, and if there are too few games in this class, when I'm finished browsing them I extend to 60-80%, so the actual mean on a given site is not that important, the quantity of games at the top is what is meaningful.
- then, the titles and scores of the individual reviews. These often convey the additional information that is lacking in a simple score : a context, to know relative to what the game has been rated (for instance, if it's a simple minigame, a 50h epic, usw.)
- then, if I'm still interested, I read one review or a few (depending on whether the scores are very scattered or not)
And I'm quite sure that this, combined to simple word of mouth for the rare exceptions, is highly sufficient. Everything else averages out in the end, so there's no point in trying to change it.
author=Craze
I score by amount of fun and the ease of accessing fun
games are fun
hopefully
Now this seems sensible. The only reasonable way to rate in my opinion is : if I've had a perfectly good time, I'm willing to give it a 5/5, and I don't care if the game is one hour long or the graphics are awful or it's just a comedy game or there's almost no gameplay or whatever. It must be good at what it tried to do since I'm not bored, and that's the whole point, isn't it ?
If you need to talk about professional games, people love Starcraft and Xenogears for entirely different reasons, the only thing that counts is whether a given game achieves what the devs were striving for, and you know that when you enjoy the game, that's all. There's no such thing as a "universal" or "official" scale, not even according to "professional" game reviewers.
Anyway there's no perfect system and the current one isn't bad at all ; if it causes some people to give a 0/5 or 5/5 just to change the average score, then the same thing will happen with a word system or anything else, and it happens all the time on "serious" sites and in magazines. Any kind of reviewing gives rise to ridiculous results and fads and whatever (have you ever looked at the top movies and games on IMDb or Metacritic ?). And the concept of "good" or "average" is just as relative and hazy as that of 2/5 : "average" with respect to what ? It will always be different due to the expectations and experiences of each gamer.
And frankly, it isn't even that important for the visitor. I have a rather standard procedure, which I think is shared by a lot of people, for determining which game I want to try on a given site. Of course, I go for featured games or those which I've heard about first. For other games,
- first, I do look at the score. I have a threshold : 80% and upward retains my interest instantly, and if there are too few games in this class, when I'm finished browsing them I extend to 60-80%, so the actual mean on a given site is not that important, the quantity of games at the top is what is meaningful.
- then, the titles and scores of the individual reviews. These often convey the additional information that is lacking in a simple score : a context, to know relative to what the game has been rated (for instance, if it's a simple minigame, a 50h epic, usw.)
- then, if I'm still interested, I read one review or a few (depending on whether the scores are very scattered or not)
And I'm quite sure that this, combined to simple word of mouth for the rare exceptions, is highly sufficient. Everything else averages out in the end, so there's no point in trying to change it.
author=Hasvers
If you need to talk about professional games, people love Starcraft and Xenogears for entirely different reasons...
This is something I feel like no one ever considers around here. There's an entire continuum of ways a game can be good, not just a single set of circumstances that always produce the best games.
mellytan
995
I believe RMN should abolish quantitative scores completely.
(Neither should it have "word scores" of "Good, Great, Perfect" etc since such a system would have essentially the same inherent flaws.)
Of course, I am pretty biased in my opinion. My favorite game magazine was Play Magazine (RIP) after they got rid of scores; regardless of how subjective and "gushy" they got, reviews seemed "purer"--if that makes any sense--without some floating-point number looming at the bottom of the page in bold (8.7 this is a must-buy, it's GOTY, Editor's Choice, stop reading and play this, reading is for chumps, just believe us and play what we tell you to). In addition, as kentona mentioned, the presentation (and "marketing") of a game is what captures attention anyway.
I would, however, like to hear what supporters of keeping quantitative scores say are the advantages to keeping such a system.
Brevity in expression is of course one thing that would be missing without scores. So maybe be adding a mandatory Pros/Cons section to reviews would be a good substitute. It might perhaps implement a system of customizable pro and con "tags" that reviewers of a game can create and that other reviewers can also use if they agree--for example, the first reviewer lists "Great custom sprites" as a pro, and later reviewers can agree and add that as a pro in their review, which adds to that pro's tally. These tags and their tallies can be made visible on the game's page for quick reference by potential players.
Otherwise, if we are to keep scores, let's not take them seriously. Let's make the scores be out of 11 and pick some game to compare all games to. "Is this game better or worse than Forever's End?" will be the question and O Worse O Better will be the radio button answer. The results will weighted by the author's maker score and compiled in a 4D pie chart for display on the game's front page.
(jk jk)
In any case, if the scoring system is to stay, let's hear some justification (besides the fact that the site was coded with this scoring system in mind, which of course is a perfectly valid reason to keep it considering the staff is all-volunteer and not making any money). Let us not copy "professional" game review systems for the sake of keeping up appearances or acting like we're something we're not. Be the change, RMN...
Great points in this post, NewBlack. I absolutely agree. But some games inspire "emotions" in us, which often seep into our writing. Some of the best game reviews I have ever read have been intimately subjective. But this is getting into the quality of writing and narrative of a review, which is an entirely different subject...
(Neither should it have "word scores" of "Good, Great, Perfect" etc since such a system would have essentially the same inherent flaws.)
Of course, I am pretty biased in my opinion. My favorite game magazine was Play Magazine (RIP) after they got rid of scores; regardless of how subjective and "gushy" they got, reviews seemed "purer"--if that makes any sense--without some floating-point number looming at the bottom of the page in bold (8.7 this is a must-buy, it's GOTY, Editor's Choice, stop reading and play this, reading is for chumps, just believe us and play what we tell you to). In addition, as kentona mentioned, the presentation (and "marketing") of a game is what captures attention anyway.
I would, however, like to hear what supporters of keeping quantitative scores say are the advantages to keeping such a system.
Brevity in expression is of course one thing that would be missing without scores. So maybe be adding a mandatory Pros/Cons section to reviews would be a good substitute. It might perhaps implement a system of customizable pro and con "tags" that reviewers of a game can create and that other reviewers can also use if they agree--for example, the first reviewer lists "Great custom sprites" as a pro, and later reviewers can agree and add that as a pro in their review, which adds to that pro's tally. These tags and their tallies can be made visible on the game's page for quick reference by potential players.
Otherwise, if we are to keep scores, let's not take them seriously. Let's make the scores be out of 11 and pick some game to compare all games to. "Is this game better or worse than Forever's End?" will be the question and O Worse O Better will be the radio button answer. The results will weighted by the author's maker score and compiled in a 4D pie chart for display on the game's front page.
(jk jk)
In any case, if the scoring system is to stay, let's hear some justification (besides the fact that the site was coded with this scoring system in mind, which of course is a perfectly valid reason to keep it considering the staff is all-volunteer and not making any money). Let us not copy "professional" game review systems for the sake of keeping up appearances or acting like we're something we're not. Be the change, RMN...
author=NewBlack
Also, an objective review isn't a review, it's a list of observations.
Great points in this post, NewBlack. I absolutely agree. But some games inspire "emotions" in us, which often seep into our writing. Some of the best game reviews I have ever read have been intimately subjective. But this is getting into the quality of writing and narrative of a review, which is an entirely different subject...
author=Feldschlacht IVWe are having a hard time trying to even pin down the root issue here, let alone determine where we want to go!
I get the feeling that this topic will ultimately change nothing. Hopefully I'm wrong!
You have to take into account that the simplicity of the current review system is a kind of strength. Most suggestions here are increasing the complexity of the review system but without any guaranteed improvements (and we haven't decided in which way we want to go, so we can't really measure whether one idea or another moves us in the direction we want to go (and is thus an improvement)).
Right now I am reading:
-numbers are too abstract
-replace numbers with words!
and
words are equally abstract.
and then some other suggestions like removing average scores, increasing the complexity by introducing thumbups for reviews (but to what end?)
People who write negative reviews of RM games mean to do one of the 3 things.
1. Discourage the creator.
2. Discourage users from playing the game.
3. Give suggestions and criticisms to help the creator to improve the game and their game making skills.
27 year old unemployed virgins who live in mommy's basement write with the intent of 1 and 2. We should let this filter through our minds each and every time we even consider writing negative reviews, and ask ourselves, "do I meet the qualifications to write this review? No? Perhaps I should let someone else do it then."
1. Discourage the creator.
2. Discourage users from playing the game.
3. Give suggestions and criticisms to help the creator to improve the game and their game making skills.
27 year old unemployed virgins who live in mommy's basement write with the intent of 1 and 2. We should let this filter through our minds each and every time we even consider writing negative reviews, and ask ourselves, "do I meet the qualifications to write this review? No? Perhaps I should let someone else do it then."
Hasvers
1298
author=mellytan
I would, however, like to hear what supporters of keeping quantitative scores say are the advantages to keeping such a system.
That's pretty simple : when you arrive on a site and you want to know what games are worth playing, you need a way to determine at least which are the most appreciated. The main german RM community site has no ratings or top list or anything in its game list, and it's a real pain to find any good game. You're guaranteed to browse through hundreds of really awful aborted 5 minute projects. I don't consider this a brilliant idea.
Actually your tag system would be a decent alternative, if we may filter games using them. But then the Misaos have proved time and again that the most popular games would end up with all qualificatives at once.
The good thing with reviews is that they demand some investment on the part of the writer, and are thus slightly more reliable than thumbs up, votes or tags, but you need something to summarize them when browsing through a list. Thus, the ratings. Which are not that bad, even when they're misused (which happens mostly for popular games which have a lot of other reviews, so they are averaged out anyway)..
cho
0
people literally can't figure out how a 5-star rating scale works
yeah, i guess counting your fingers would be pretty tough when you're sucking your goddamn thumb all the time
novel solution to problem: stop being whiny titty babies when people don't like the thing you like
insert your favorite dealwithit gif here (my favorite's the johnny cage one)
yeah, i guess counting your fingers would be pretty tough when you're sucking your goddamn thumb all the time
novel solution to problem: stop being whiny titty babies when people don't like the thing you like
insert your favorite dealwithit gif here (my favorite's the johnny cage one)
I don't know about anybody else, but I've been absolutely terrified of making reviews of late. This fiasco was certainly a factor, I admit, but the fiasco with Legendary Legend, and now Forever's End are also deterrents.
I'm not sure where I'm going with this thought, though. I'd certainly like to see more insightful/helpful commentary as far as review comments are concerned, but being insightful/helpful is not required, nor can it be reasonably enforced.
I'm not sure where I'm going with this thought, though. I'd certainly like to see more insightful/helpful commentary as far as review comments are concerned, but being insightful/helpful is not required, nor can it be reasonably enforced.
author=Fallen-Griever
If we want to be taken seriously then we should definitely compare our games to professional/indie games when applicable
F-G I love you baby but the inverse of this is that maybe if we want to be taken seriously we shouldn't SELF-RATE in such a way that the average game rating is 1.5 or 1 or some shit. Self-deprecation only goes so far before it becomes infuriating instead of ingratiating. If the message from our reviews is 'we suck, we suck, we really really suck' then some people can't be faulted for believing us. It is insane to have the '5 Star' rating mean 'an excellent professional game made by hundreds of people over several years by a professional studio' when almost every game on here is a one-man show.
I personally see absolutely no value in harsh reviews.
Of course, the use of the word professional in the topic title is itself highly arguable. This post will expand over time.
A 2.5/5 should be an average game regardless of who made it or what engine it was made in.
The man speaks sense.
