Full Games List
Featured Games
New Games
Reviews
Media
Images
Blogs
Game Updates
Images Feedback Summary
Game Design Highlights
Tutorials
Articles
Scripts
Utilities / Plugins
Resources
Engines
Engine Bulletins
What is Makerscore?
The Simple Times has started!
Past Events
Achievements
Misaos
Forums
Post History
Chat
New to RMN?
Full Store List
RMN Music Pack (free)
REVIEW SCORING: STANDARDIZATION, PROFESSIONALISM, ETC.
Posts
halibabica
View playlists
CloseOne thing I'd like to mention that I see popping up a lot in this discussion so far is the general idea of comparison. We keep saying whether we should change the scale based on comparison to professional games vs. comparison to our own homebrew stuff.
I would argue we shouldn't be comparing any games to any other games. It's only natural to want to gauge something's quality by holding it up to something similar, but this is not an accurate method because the objects being compared both need to be on a more universal scale of comparison! In other words, there's an ambiguous set of standards by which everything under them is defined. Unfortunately, trying to make those standards less ambiguous is nearly impossible. As such is the case, we're best off leaving reviewers to their own designs and giving them a rating system that is most intuitively conducive to accurately reflecting their opinion.
We can't solve this by adhering to a 'professional' rating system like magazines use, because as was said before, these scores are altered for commercial purposes. In a community like this, we can call a shitty game shit for being so without any business repercussions, so why anyone would support such a system is beyond me.
I personally think the proposed word scale is the best idea. It's much easier for a reviewer to pick out of a set of words how they felt about the game they played, and if we kept it as a scale of ten, then the numbers would still work like they do now (just behind the scenes). I'm sure you could rig up the average star rating to display the average word instead. Even have it say "Overall, this game is reviewed as:" above the word displayed so those seeing it know where it came from.
And yes, there will still be trolls and abusers and people who have friends write reviews to hype them up; those are the sort of things that are supposed to be caught in the submission queue, so let's not even worry about that. The current system works well; it could just use something more universally understood.
You can't tell me words and numbers are equally ambiguous.
I would argue we shouldn't be comparing any games to any other games. It's only natural to want to gauge something's quality by holding it up to something similar, but this is not an accurate method because the objects being compared both need to be on a more universal scale of comparison! In other words, there's an ambiguous set of standards by which everything under them is defined. Unfortunately, trying to make those standards less ambiguous is nearly impossible. As such is the case, we're best off leaving reviewers to their own designs and giving them a rating system that is most intuitively conducive to accurately reflecting their opinion.
We can't solve this by adhering to a 'professional' rating system like magazines use, because as was said before, these scores are altered for commercial purposes. In a community like this, we can call a shitty game shit for being so without any business repercussions, so why anyone would support such a system is beyond me.
I personally think the proposed word scale is the best idea. It's much easier for a reviewer to pick out of a set of words how they felt about the game they played, and if we kept it as a scale of ten, then the numbers would still work like they do now (just behind the scenes). I'm sure you could rig up the average star rating to display the average word instead. Even have it say "Overall, this game is reviewed as:" above the word displayed so those seeing it know where it came from.
And yes, there will still be trolls and abusers and people who have friends write reviews to hype them up; those are the sort of things that are supposed to be caught in the submission queue, so let's not even worry about that. The current system works well; it could just use something more universally understood.
You can't tell me words and numbers are equally ambiguous.
deboo
85
but having your game rated as 1 star and having it rated as poor - isn't that an equally bad score for the designer? in some cases if i was the developer i would less like to have 'this game is rated: terrible' than 0.5 stars. i just don't see what difference it would make; if a reviewer wants to bash a game so much he would give it 0.5 stars then he will also give it 'terrible.' if you want to go with words over numerical values you might as well totally abolish the rating scale (which i don't think is really necessary - it isn't like people don't use words to back up their score anyway).
standardise what the star scores are first (which kentona has already basically done, and something i think most people automatically do correctly anyway), and if someone goes off that scale, say, decides 2 should be an average, then we can call them out on it and say 'i don't think this is fair in comparison to the other reviews.' but honestly, star system or word system, the results and opinions in the review will be the same if the reviewer decides he wants to be too harsh/too nice, so i don't see what difference it's going to make. we can't change a reviewer's opinion on whether he wants to compare a game to a professional game or whatever, but only hope they have the decency and understanding not to.
standardise what the star scores are first (which kentona has already basically done, and something i think most people automatically do correctly anyway), and if someone goes off that scale, say, decides 2 should be an average, then we can call them out on it and say 'i don't think this is fair in comparison to the other reviews.' but honestly, star system or word system, the results and opinions in the review will be the same if the reviewer decides he wants to be too harsh/too nice, so i don't see what difference it's going to make. we can't change a reviewer's opinion on whether he wants to compare a game to a professional game or whatever, but only hope they have the decency and understanding not to.
But that's the thing; the end result should be the same. It's just that value words are easier to understand than numbers, and with a word system, you won't get people who think 2 is average while others think 3 is. Average will be average, and there won't be any confusion (and we won't have to call people on it).
For the rating system to be effective, reviewers need to understand how to use it. Words are easier to understand than raw numbers.
For the rating system to be effective, reviewers need to understand how to use it. Words are easier to understand than raw numbers.
deboo
85
yeah i understand what you're saying...i guess it just seems like an unnecessary change from something people should know how to work with in the first place. seriously, i'm surprised that people believe 3 or 2 or whatever could considered to be the average when it's just plain not.
honestly i haven't very much come across a review that believes 1.5 = average when they get to the conclusion. not even 2 or 3. in the end they've always backed up their opinion that it is below/above average. the problem is, other's believe, in the standing of average, it should have been rated better/worse. unfortunately, we can't put a set rule on what 'average,' should be in relation to other games but put faith in the reviewer that his average is at least similar to ours. even using words people can still say 'this is average,' while others think it is actually exceptional, because the view of average is different; there would still be just as much confusion.
for example, in sai's review, many might see forever's end as at least 'average,' and might rate it as that. sai however made it clear he would probably rate it as 'terrible,' so having a word/star rating would have changed nothing. please correct me if i'm wrong sai but i hope not because then there is a serious problem of what you believe is symbolised on the star rating scale.
honestly i haven't very much come across a review that believes 1.5 = average when they get to the conclusion. not even 2 or 3. in the end they've always backed up their opinion that it is below/above average. the problem is, other's believe, in the standing of average, it should have been rated better/worse. unfortunately, we can't put a set rule on what 'average,' should be in relation to other games but put faith in the reviewer that his average is at least similar to ours. even using words people can still say 'this is average,' while others think it is actually exceptional, because the view of average is different; there would still be just as much confusion.
for example, in sai's review, many might see forever's end as at least 'average,' and might rate it as that. sai however made it clear he would probably rate it as 'terrible,' so having a word/star rating would have changed nothing. please correct me if i'm wrong sai but i hope not because then there is a serious problem of what you believe is symbolised on the star rating scale.
Pokemaniac
3559
I think, in addition to a scoring or word-based system, we need some thumbs up/thumbs down component to reviews. Something to say "I recommend this game" or "I don't recommend this game". You know when you read a review and there's a huge disparity between the tone and the scoring? This sort of system works around that, I think. Similarly, a "breakdown" of what type of gamers will like the game would be helpful too.
Is the issue that we cant read review scores? I swear, I saw a thread somewhere that gave guidelines on what each star value represented. If I recall, 1 star was "terrible", 2 was "bad but playable", 3 was "average", 4 was "good", while 5 was "excellent". Or something along those lines.
Is the issue that people enjoy flaming reviews that they personally believe should not have been written in the first place? There's absolutely nothing that can be done of that. While I would personally prefer a level-headed conversation, some people are just not like that.
*Edit: People do enjoy their drama-fests!
Is the issue that people enjoy flaming reviews that they personally believe should not have been written in the first place? There's absolutely nothing that can be done of that. While I would personally prefer a level-headed conversation, some people are just not like that.
*Edit: People do enjoy their drama-fests!
from boobledeebooCommon sense is amazingly hard to come by.
it just seems like an unnecessary change from something people should know how to work with in the first place.
from boobledeebooWell, of course what some call average others call exceptional. That's because we're dealing with opinions here! If anything, it creates less confusion, because the game's review score isn't about what everyone thinks. It's about what the reviewer thinks. A person giving a review to a game could assign any amount of value to a number based on their own personal thoughts. But there's a lot less debate between what's poor, average, and good versus what's 2, 3, and 4. See what I mean? This is about 2 things: giving the reviewer a more clear choice, and having that choice reflect on the game in a way others understand correctly.
even using words people can still say 'this is average,' while others think it is actually exceptional, because the view of average is different; there would still be just as much confusion.
Also, I don't think thumbs up/down would help much at all. Like others have said, we likely wouldn't have enough people to use it for it to really mean anything, and it still wouldn't tell much even if we did. To me, it's just an easy way for lazy people to not have to express their thoughts in words.
deboo
85
i do see what you mean but we could still just have the stars as representations of the words so long as everyone agreed to it. i suppose if people find it so hard to represent 2.5 as average, 4 as good or whatever then yeah words would be better, but it's just like spelling things out that we shouldn't need to.
the scores are backed by information explaining why the score is the score anyway, so...ahh just seems a bit pointless haha.
the scores are backed by information explaining why the score is the score anyway, so...ahh just seems a bit pointless haha.
Here are things I can do, or look into doing:
1) I can add the associated words (poor, average, exceptional etc...) next to the numbered score in the review submission form (ie- the droplist that the review uses to select a score) while leaving everything else the way it is. So if there is confusion with respect to the reviewer and the scoring system, then that should help alleviate that.
2) I can implement something like "X many people found this review helpful" ala Amazon.com (but I am not entirely sure what this will accomplish!)
3) I can look into adding some sort of Favourites list to members here, so that they can publicly identify games they really liked.
4) I can look at adding some sort of Recommended counter for gameprofiles, something like "X many people recommend this game". (I guess this is kind of like a Thumbs up/down system, but without the thumbs down part).
5) We have spitballed an idea to implement some sort of easy-to-use scoring system where we can just leave a brief comment and a score (or scores for various categories). So basically a Rated Comment on a gameprofile. I can look into that and how to incorporate that concept into RMN if you guys want.
Lastly, Max and I both brought up the point that the review system is servicing to things: Critiques and Reviews. Though similar, they serve different purposes (in a nutshell, reviews are for potential players and critiques are for the developer). We also allow reviews for demos or unfinished games (which has sometimes caused some grief). I can probably look into implementing some sort of critique system for that purpose, and leave reviews as is.
I like how the review system here works right now, scores and stars and all. It is a very simple and flexible system, and works well 95% of the time. We can tweak it a little but I don't see a need to overhaul or replace it. However, I do see a need for some sort of supplemental system (hence the bevy of proposals above).
Thoughts?
1) I can add the associated words (poor, average, exceptional etc...) next to the numbered score in the review submission form (ie- the droplist that the review uses to select a score) while leaving everything else the way it is. So if there is confusion with respect to the reviewer and the scoring system, then that should help alleviate that.
2) I can implement something like "X many people found this review helpful" ala Amazon.com (but I am not entirely sure what this will accomplish!)
3) I can look into adding some sort of Favourites list to members here, so that they can publicly identify games they really liked.
4) I can look at adding some sort of Recommended counter for gameprofiles, something like "X many people recommend this game". (I guess this is kind of like a Thumbs up/down system, but without the thumbs down part).
5) We have spitballed an idea to implement some sort of easy-to-use scoring system where we can just leave a brief comment and a score (or scores for various categories). So basically a Rated Comment on a gameprofile. I can look into that and how to incorporate that concept into RMN if you guys want.
Lastly, Max and I both brought up the point that the review system is servicing to things: Critiques and Reviews. Though similar, they serve different purposes (in a nutshell, reviews are for potential players and critiques are for the developer). We also allow reviews for demos or unfinished games (which has sometimes caused some grief). I can probably look into implementing some sort of critique system for that purpose, and leave reviews as is.
I like how the review system here works right now, scores and stars and all. It is a very simple and flexible system, and works well 95% of the time. We can tweak it a little but I don't see a need to overhaul or replace it. However, I do see a need for some sort of supplemental system (hence the bevy of proposals above).
Thoughts?
I really like the idea of a Critique system. It's a great idea. I wonder if the critiques can be set to private so that only the developers can see them. Afterall, they're more like a private review, right? I may be thinking of it differently than what you've got in mind, though.
I'm also all for the Recommended games (you know Heroes Realm will be on every profile, right? :P) and the Favourites list.
I'm also all for the Recommended games (you know Heroes Realm will be on every profile, right? :P) and the Favourites list.
Private critiques would get written 10% as often as public ones. Part of the fun of reviews is hearing yourself talk :)
author=Illustrious
People getting their panties in a bunch when lots of people post because a reviewer's scoring standards were most disagreeable part of it. Sometimes, it's what there is to talk about. If people disagree with the content being said, they talk about that. Then some prick like person jumps in and essentially trolls the people he considers trolling, which is more derailing than what's currently being discussed. Really, you want to talk about standards in the actual context of a review you get shunned. You want to make your own topic about reviewing standards and you get shunned for making a topic thats been repeated tons of times. Fuck, let people discuss. So a topic is following the rules, and if you don't like where its going, the topic is not for you at that moment. Just stay away from it. Being the guy stepping in and saying "I don't like what you are saying in this topic/review.. all of you sux" is not any more amicable than what you do not like reading in your topics.
Perhaps you are right to an extent, though I think your response is also highly exaggerated. The reality is we are discussing what are in some cases sweeping changes to a review system in an attempt to standardize review scores. Many of the changes requiring using a standardized format, mandating subscores, assigning explicit meanings to the ratings, or replacing the star system with words and/or grades. These issues when considered by themselves may be fine and/or needed but none of them do anything to address the actual reason for wanting to change the system to begin with.
When looking at the review that generated this discussion, almost none of the proposed changes would have changed the final result at all. Reviews are subjective even when the scoring system attempts to be objective. If we switched to a word score we end up with a "hated it" instead of a 0.5. If we switched to a grade we end up with a F- instead of a 0.5. Neither case fixes the problem. We could mandate grading the games played by individual subcategories as has been suggested, but again in the case of the review that has caused the controversy it would not have changed anything -- it even had subscores already.
The comparison to professional/commercial reviews don't really work well since a) it's well known that games journalism is mostly a joke with ultra inflated scores and b) these are not professional games. You can't hold a RPG Maker game to the same standards as you would a professional game (and if you say you do you are probably lying).
The bottom line is no matter what changes are possible, reviews will always be subjective. Some reviewers rate games as standalone products, others rate games compared to other games or experiences, and some factor in non-game related data into the equation. What is an "average" RPG Maker game to some people is a "just barely playable game" to someone else. The outlier scores can only be corrected via having larger sample sizes to counteract them and give a better overall view of how people are feeling about the game. And that requires... a lot more reviews.
Volunteers?
If you don't want a game to be rated less than a two then put effort into your work. It's honestly that simple. If your game has effort put into it then it won't receive a low score.
I call bullshit.
You have no idea what you're talking about. It's idiotic statements like this that necessitate this conversation in the first place. I have had games I've made given reviews of less than 2 stars; you can say what you want about the quality of those games, but if you're going to say that they haven't had effort put into them, you need to STFU. One of the games I'm talking about took literally YEARS of work to complete. The others have had hundreds upon hundreds of man hours logged working on them. SHOULD a review of less than two stars indicate that a game did not have effort put into it? Yes. DOES it indicate that? Not remotely.
TLDR if you think that 'less than two stars' does/should mean 'has no effort put into it' then you ARE, in fact, advocating for change in the review standards, not defending them.
but having your game rated as 1 star and having it rated as poor - isn't that an equally bad score for the designer? in some cases if i was the developer i would less like to have 'this game is rated: terrible' than 0.5 stars. i just don't see what difference it would make; if a reviewer wants to bash a game so much he would give it 0.5 stars then he will also give it 'terrible.' if you want to go with words over numerical values you might as well totally abolish the rating scale (which i don't think is really necessary - it isn't like people don't use words to back up their score anyway).
standardise what the star scores are first (which kentona has already basically done, and something i think most people automatically do correctly anyway), and if someone goes off that scale, say, decides 2 should be an average, then we can call them out on it and say 'i don't think this is fair in comparison to the other reviews.' but honestly, star system or word system, the results and opinions in the review will be the same if the reviewer decides he wants to be too harsh/too nice, so i don't see what difference it's going to make. we can't change a reviewer's opinion on whether he wants to compare a game to a professional game or whatever, but only hope they have the decency and understanding not to.
I agree with all of this except the part in bold (the formatting is mine). To the part I put in bold:
Sure we can! It's called a rubric, and there's no reason we shouldn't have one. Keying each possible numeric score to specific, descriptive words...or even better, a few concise full sentences, brings us closer to having a rubric.
If my 2, F-G's 2, Sali's 2, and Soli's 2 don't all mean roughly the same thing, all reviews are less useful. This can even exert pressure on Reviewers, like F-G who at one point felt compelled to go back and lower all of his scores to comply with the overall harsher standards in the community at large. We need to standardize these numbers. Reviews which are judged to grossly disregard the rubric can probably be safely deleted, after a reasonable warning period.
The truth is, my 3 is equivalent to Salierus's 2 or someone else's 4, because we don't have a standardized rubric. Ideally, I'd like a standardized rubric that matches MY standards (and who wouldn't!) but it's better to have any than none.
2) I can implement something like "X many people found this review helpful" ala Amazon.com (but I am not entirely sure what this will accomplish!)
3) I can look into adding some sort of Favourites list to members here, so that they can publicly identify games they really liked.
4) I can look at adding some sort of Recommended counter for gameprofiles, something like "X many people recommend this game". (I guess this is kind of like a Thumbs up/down system, but without the thumbs down part).
I'm against all of this, especially 2 and 4. We already have enough metrics in place to measure the ongoing popularity contest--download count, page views, make score, etcetera, subscribers, etcetera. We don't need any more channels to numerically illustrate to aspiring developers how their work isn't reaching anyone compared to the really popular games and they should just give up.
author=Max McGeeIf you don't want a game to be rated less than a two then put effort into your work. It's honestly that simple. If your game has effort put into it then it won't receive a low score.I call bullshit.
You have no idea what you're talking about. It's idiotic statements like this that necessitate this conversation in the first place. I have had games I've made given reviews of less than 2 stars; you can say what you want about the quality of those games, but if you're going to say that they haven't had effort put into them, you need to STFU. One of the games I'm talking about took literally YEARS of work to complete. The others have had hundreds upon hundreds of man hours logged working on them. SHOULD a review of less than two stars indicate that a game did not have effort put into it? Yes. DOES it indicate that? Not remotely.
TLDR if you think that 'less than two stars' does/should mean 'has no effort put into it' then you ARE, in fact, advocating for change in the review standards, not defending them.
NicoB is burnt out. He poured his heart into Forever's End. Sai rips it to shreds.
It happens a lot more than one would think.
Of course, since this is the internet, the most popular and socially acceptable response is to say BFD, NicoB needs to get the fuck over it, and stop being a whiny, butthurt pansy.
Don't get me wrong: negative reviews, even strongly worded ones, have their place.
I actually broke down my own rubric like so (blast from the past early 2010):
ANYWAY, HERE'S A RADICAL SUGGESTION:
* CRITIQUES don't have scores, and the first place they go is directly to the developer, via a private channel. The developer, not the reviewer, has the choice whether or not to make the critique public. Critiques are fully textual, and don't have scores. The target audience is the developer(s). Critiques don't have to be gentle and in fact probably shouldn't be. For the reasons mentioned by harmonic (people like to hear themselves talk), critiques should be worth slightly more makerscore than reviews, but we definitely want to encourage BOTH.
* REVIEWS are always public, and their target audience is potential players. That's who it written to. Reviews should have a standardized rubric, a score (numerical or starred) and maybe even established subscores. Reviews should treat an average game as being three stars, for the reason someone mentioned above. A 5/10 is a FAILING GRADE. The truth is, a 2 or 2.5 star review is much more discouraging than a 3 star review encourages. Reviews of less than 2 stars should be reserved for games that are really, offensively, disgustingly awful, lazy, stupid, unplayable, etcetera.
I know this would be a big change, but the fact of the matter is, there is a fundamental dichotomy in the two things that reviews are trying to do: advise the creator, and advise the audience. As of right now, only rare and masterful reviews manage to successfully do both.
Don't get me wrong: negative reviews, even strongly worded ones, have their place.
I actually broke down my own rubric like so (blast from the past early 2010):
author=Max McGee
Like Darken said this is not monumentally different from {Kentona's} and it is not well-worded (because I spent five minutes on it, rather than the five hours it probably deserves) but hopefully it will help elucidate my thoughts on the matter:
0.5: This game not only does nothing RIGHT but also has some things that are so wrong as to be offensive to the tastes of a potential player. Simply a game that is "bad" at everything is not enough to earn this score. A game must be bad at everything have at least one aspect so awful it causes the reviewer nearly physical pain. This should by far be the rarest score and should never be given based largely on personal tastes.
1: This game does everything wrong and has no strong points but shows enough effort and/or thought that it is not a 0.5. "Bad" but "inoffensive" would be the distinction and using expletives to describe the game is probably not appropriate at this point.
1.5: This game fails at most things but has perhaps one category which partially redeems it or shows promise. Alternatively, the game is mediocre or below average (but not bad) in every category.
2-3: The vast majority of games should fall into this rage. These games are "average" and are worth playing; whether or not they are "good" or you will enjoy them is largely up to personal tastes and preferences. These games have good points and bad-points. A 3 has more Pros than Cons; a 2.5 is exactly average, and a 2 is well executed or conceived in at least a few areas even if it has several weaknesses. All of these games are worth your time especially if their specific features interest you.
3.5-4.5: These games do most things right and have few flaws. Out of the categories measured in your review, most are good or excellent.
5: The second rarest rating, given only to games that are EXEMPLARY in every area and have no significant weak points. This seal of "perfection" reflects your personal opinion. Do not expect everyone (anyone) else to agree.
.
Also I think it is probably ludicrous for anyone to review a complete game with the same standards as a demo. When you are harsh in reviewing a demo, the author can take your feedback into account and alter the game in production. In fact the point of reviewing a demo is to give the author feedback. Reviewing a complete game, on the other hand, the point seems to be to evaluate the author's accomplishment and act as a "reviewer" giving the community-as-audience an idea of the game's entertainment value. I think that perhaps if complete games- which generally can't be improved based on feedback anyway- were given a little bit more delicacy and respect from reviewers, we would have more of them.
One thing I really like on this site is how anyone can write a review in whatever style they want using whatever categories they want. But of course, when it comes to standardizing responses, that too is problematic...
ANYWAY, HERE'S A RADICAL SUGGESTION:
* CRITIQUES don't have scores, and the first place they go is directly to the developer, via a private channel. The developer, not the reviewer, has the choice whether or not to make the critique public. Critiques are fully textual, and don't have scores. The target audience is the developer(s). Critiques don't have to be gentle and in fact probably shouldn't be. For the reasons mentioned by harmonic (people like to hear themselves talk), critiques should be worth slightly more makerscore than reviews, but we definitely want to encourage BOTH.
* REVIEWS are always public, and their target audience is potential players. That's who it written to. Reviews should have a standardized rubric, a score (numerical or starred) and maybe even established subscores. Reviews should treat an average game as being three stars, for the reason someone mentioned above. A 5/10 is a FAILING GRADE. The truth is, a 2 or 2.5 star review is much more discouraging than a 3 star review encourages. Reviews of less than 2 stars should be reserved for games that are really, offensively, disgustingly awful, lazy, stupid, unplayable, etcetera.
I know this would be a big change, but the fact of the matter is, there is a fundamental dichotomy in the two things that reviews are trying to do: advise the creator, and advise the audience. As of right now, only rare and masterful reviews manage to successfully do both.
deboo
85
author=Max McGeetbf i thought he dealt with it pretty well
NicoB needs to get the fuck over it, and stop being a whiny, butthurt pansy.
critiques separate from reviews seems like a good idea - reviews tend to be a lot more on the side of 'i'm doing this to help you as a developer,' and i think that can skew them to being overly critical.
Illustrious
520
The issue for that review doesn't even really seem to boil down to what Sai's normal standards are. Of course I am not in his head, but this is how its is seen - A corrective review. "This game has all the hype and prestige but I don't like it. Wtf are these people thinking? Now lets overcompensate and attempt to bring it down to size." No matter how true it is, it reads like that to most people and they are arguing over what they feel it is wrong. This is what I consider is the rogue reviewer that Holbert mentioned. And this is that type of reviewer he feels theres no hope of changing. They will do what they will do.
This problem is not solved by word association, but at least is made better. Example:
Three star rating - Average game. Are you sure this is what you are trying to communicate to your readers?
People will still have different opinions of what average and good is. What is different is you know this reviewer isn't giving a game one star and telling you
"Lol, do not feel bad. 1 star is what I call average so its not a bad score." By giving it one star, he has now confirmed he is telling you it is inferior crap and this is what he means to say. A difference in what is liked and not liked is healthy and feuls the review system.
For all intents and purposes, lets expect our reviewers to judge comparable to other efforts on the site. It's clear that that is the most accessible value to the majority of users on the site. We know no user is expecting professional calibur games by going to rmn. It's more along the lines of expecting something rough using one of the toolkit game makers. May those game makers who transcend such an expectation receive a 5 and the accolades that go with it. Theres something in it still for the game makers who take their projects seriously and make them well. To think that being tougher on reviews will raise the bar for content on this site is a fallacy. I expect people are making just what they can with time, resources and creative ability. People are at different stages of the craft. If they have the desire, they will become better with how they have learned to improve. No person should feel so motivated by star ratings that they totally change their game.
Last rant. You all are just too afraid to give out that 5 to a game. Yes, its special, but you don't need to wait for that perfect game that will never come.
I am convinced enough if your 5 star rating does nothing more than describe your ENJOYMENT level of the game. Someone said it's hard to give someone a 5 and be critical. http://rpgmaker.net/games/2138/reviews/970/
I pointed out a great many things I felt could be better. I am sure theres many other critiques I forgot to put in. Far from perfect here. It can have my 5 because I enjoyed it as much as I did.
This problem is not solved by word association, but at least is made better. Example:
Three star rating - Average game. Are you sure this is what you are trying to communicate to your readers?
People will still have different opinions of what average and good is. What is different is you know this reviewer isn't giving a game one star and telling you
"Lol, do not feel bad. 1 star is what I call average so its not a bad score." By giving it one star, he has now confirmed he is telling you it is inferior crap and this is what he means to say. A difference in what is liked and not liked is healthy and feuls the review system.
For all intents and purposes, lets expect our reviewers to judge comparable to other efforts on the site. It's clear that that is the most accessible value to the majority of users on the site. We know no user is expecting professional calibur games by going to rmn. It's more along the lines of expecting something rough using one of the toolkit game makers. May those game makers who transcend such an expectation receive a 5 and the accolades that go with it. Theres something in it still for the game makers who take their projects seriously and make them well. To think that being tougher on reviews will raise the bar for content on this site is a fallacy. I expect people are making just what they can with time, resources and creative ability. People are at different stages of the craft. If they have the desire, they will become better with how they have learned to improve. No person should feel so motivated by star ratings that they totally change their game.
Last rant. You all are just too afraid to give out that 5 to a game. Yes, its special, but you don't need to wait for that perfect game that will never come.
I am convinced enough if your 5 star rating does nothing more than describe your ENJOYMENT level of the game. Someone said it's hard to give someone a 5 and be critical. http://rpgmaker.net/games/2138/reviews/970/
I pointed out a great many things I felt could be better. I am sure theres many other critiques I forgot to put in. Far from perfect here. It can have my 5 because I enjoyed it as much as I did.
S. F. LaValle
1069
author=boobledeebooauthor=Max McGeetbf i thought he dealt with it pretty well
NicoB needs to get the fuck over it, and stop being a whiny, butthurt pansy.
Max doesn't actually think that, he was making a point.
As others have all pointed out, the intent behind the review dictates not only how the review and score is written, but how strong our reaction is towards it.
For example, Sai's review. The score was very low, to the point where I personally do feel it fabricated. But the intent wasn't necessarily, "Nico's game is too successful, time to take it down a notch." It was clear in reading the review, that Sai was saying, in so many words, "HEY! Everyone is gushing about your game, but there are VERY SERIOUS PROBLEMS with it." The score was to get people's attention. It was meant to ask people, "why does no one else have a problem with x, y, and z? This is bad design."
It's critique, very necessary critique, and as illustrated above by others' points, it's in a controversial venue. Nico is far less likely to appreciate the criticism when he sees the review as slander. While everything Sai said in his review is important to consider, does it belong in the visibility equation?
I'd say no; not just to protect people and their games from negative reviews affecting their overall score, but for that other reason (game creators eschewing the critical advice given), and probably others as well.
Setting reviewing standard is also not enough, I believe. Because the intention behind the review would still be there. People are talking a game up, ignoring serious playability or other issues, and the need to write an attention-getting review will still remain, as will the ensuing drama.
We can ask people to be responsible, but that's basically the honor system we've been on since reviewing started. And these reviews will continue to come up, for better or for worse. A lot of people would just ask us to remove them as troll reviews. We wouldn't want to remove them because they actually provide important feedback. We'd just have to accept the way things are and let the heated discussion and trolling continue.
So, that's why I back my opinion on giving criticism a proper outlet, and leaving reviews to be something for the site visitor to digest. Many of us here are aware of the flaws of the star system already, and thus we ignore that metric when browsing games on the site. A critique system has the potential to help not only the game creator, but also other developers on the site as well. If critiques were posted similar to how blogs are, for example, we could read about people's opinions to certain aspects of game design in REAL applications, a handy addition to forum discussion. So we could all load up a critiqued game, witness first-hand what was critiqued, and let us draw our own conclusions from it, be it getting eyes on something that doesn't work well, or being inspired by something that does work well. Maybe even allow critics to attach save games to their critiques as a tool for other developers.
Anyway, I focus mostly on the developer side of things because we are the majority voice of dissent. We still have an important job in delivering content to site visitors. I'll post more on that later.
