ON REVIEWING AND STANDARDS
Solitayre's review criteria and thoughts on why they are important.
- Solitayre
- 02/23/2010 08:44 PM
- 12068 views
Reviewing is a subjective process; it is thought and opinion, so this article and everything in it is by definition going to be biased by my personal beliefs. Some of you aren’t going to like what I’m about to say.
Some of you have very specific goals in mind when you create a game. Some of you are trying to tell a great story, some of you are trying to create fun puzzles, some are trying to create awesome and epic battles, and some are trying to show off your art or music. That’s all well and good; game-making can be a perfectly legitimate artistic endeavor and I do not intend to diminish that. But you have to understand not everyone is going to see it that way. People who download your game are looking for a fun game experience, and they’re going to have certain expectations of your project.
Some of you like to break down the components of a game in terms of their weight or importance and plug them into some sort of ridiculous math equation. {Gameplay = Story + (graphics/ music^2)} This is asinine for a couple of reasons. The first reason is it doesn’t even really make a lot of coherent sense. The second, and more important reason, is trying to imply that some part of an RPG is more important than any other is pretty ridiculous.
For you to have a fully realized game, none of these aspects can be lacking. Sorry gameplay people, whine about how story is a waste of time all you want, in an RPG there’s an expectation that there’s going to be some decent writing somewhere. I don’t understand this movement to sort of hack out parts of a game the creator doesn’t like with some kind of meat cleaver and expect the game to be better for it. It’s like a house of cards. You can’t take cards away and call the resulting mess a complete project. It might still be fun in some ways, but I think most people would agree that something was missing. The same goes for story people. If you’re going to tell a story but neglect your “gameplay”, why include it at all? If you’re going to completely forgo the RPG mechanics in pursuit of your storytelling, the resulting story might be pretty good, but you aren’t really making a “game” anymore. You’re kind of making something else.
Now, I don’t want anyone to misunderstand. I am not saying your story needs to be enthralling, but it needs to make sense. Your graphics don’t need to be gorgeous but they should be consistent and functional. Your gameplay doesn’t need to be stellar but it better not be boring.
So while I’ll judge your game on the individual aspects that make up your project, I’m also going to judge it as a whole. And if your intentions are clear I’ll even try to look at it from your perspective, as you intended it. But the bottom line is I am going to tell you what’s good and what’s bad, what’s wrong with it, even if it wasn’t a part of your project you cared about. Because it’s still part of your project and that part of it still reflected on the overall game as a whole.
Now, how can you minimize this? If you are not good at a certain aspect of a game, how can you make it so it’s not a problem? By playing down those aspects. If you’re not good at writing, don’t dump lots of long, badly written conversations on me. If you’re not good at making puzzles, don’t put a lot of puzzles! If you’re bad at balancing combat, don’t throw me into a fight every four steps. If a problem doesn’t bother me much, I’m not going to get hung up over it. If I don’t notice that there’s a problem at all, then it’s a non-issue.
Now, supposing you see a review you don’t agree with. It could be written by me or someone else, it could be for your game or someone else’s project. It is very important to remember that reviews are one person’s opinion. Getting a bad review is not the end of the world. It does not mean no one in the world likes your game. It also does not mean you should flip out and get defensive. If someone took the time to actually review your game it means that had a vested interest in it. If they didn’t care they’d simply not say anything at all. You should be thankful that someone, anyone, took the time to comment about your game and try to help you improve. Disregarding a review or claiming a reviewer has suspect motives just because they gave a negative review is not only offensive and disrespectful but ultimately a disservice to yourself and your fans.
By request, I am including my general reviewing guidelines here so that people know generally how I will look at a game and rate it.
My reviews will usually start with a short backstory about my understanding of the project and why I chose to review it. I will then provide a brief synopsis of the game’s plot to let the player know basically what they can expect going in. From here I rate things by the following categories.
Balance: This is my general impression of how battles in the game were handled, including any aspects of the game that tie into this including optional or additional systems or character abilities. I will judge by how mechanically interesting they were, if they were fun, if they were of an appropriate level of challenge and if everything generally worked the way it was supposed to.
Level Design: This is where I will talk about the general layout of the game and all the things it encompasses. Things like graphics will be discussed here, though I will generally not mention them unless they are very good or extraordinarily bad. Things like town design, dungeon design, puzzles, including aesthetics, will be included here, and if I have any concerns about the controls or interface, this is generally where I’ll mention them.
Characters: I will discuss my thoughts on the writing, dialogue, characters and their subsequent development. If I have any in-depth questions or thoughts on a character I will go into it, including any things I liked. If I have worries that a character seemed under-developed or seemed weak, it generally means I wanted to know more so shouldn’t be construed as necessarily a bad thing.
Story: I will discuss narrative structure here, including the overall plot, and any storytelling methods used. Any unconventional storytelling methods aside from basic, chronological linearity will probably be mentioned, and I will also discuss the nature of side quests or plot branches, if any. If I notice any stylistic influences, such as a famous work or author, I will usually give those a nod here as well.
Music and Sound: I will talk about how any musical choices used in the project added or detracted from the overall mood or atmosphere of the game, as well as pointing out any times I felt sound was being used inappropriately. Having “okay” music generally won’t garner much of a response from me, but having really great music will really impress me. I will generally favor music I haven’t heard before over popular commercial rips.
Finally, the general scale I use to “rate” projects is as follows. My ratings tend to be fairly arbitrary at times but this is a general indicator of what I thought.
5.0- A near perfect game. It must do nearly everything right as well as go above and beyond expectations in every way. I do not expect I will ever give this score, but you never know...
4.5- An exemplary game. Not only does it do almost everything right but goes the extra mile in terms of content, programming, or some other area that sets it above other games. Bugs are all but non-existent.
4.0- An excellent game I recommend people play. It has few flaws and those it has generally don't detract from the overall experience. Almost no bugs.
3.5- An above-average game. It does a lot of things right but unfortunately has a few flaws that drag it down. Very few bugs.
3.0- An average game. It has a mix of good qualities and poor design choices. Not everyone may like it. There may be bugs but they do not detract from the experience.
2.5- Still pretty average, but here the design flaws are very noticeable. One or more aspects of the game are severely lacking which detracts from the experience. There may be multiple obvious bugs.
2.0- A below average game, it has a lot of faults that overshadow its good points. There may still be a good game inside...somewhere, but it's not there yet. It is very hard to get a lower score than this from me.
1.5- A poor game. The game is severely lacking in multiple areas that lead to a bad game experience on my part.
1.0- A very poor game. A game with a score is generally a sign I did not think the author was trying or that they didn't care. The game is disjointed, mechanically broken and may be near unplayable. Even this is not a 0.5 from me, oh no, only a very special kind of bad gets a 0.5...
0.5- A blight on humanity. The game was offensive to me on a primal level. It has no redeeming qualities. It sends me into apoplexies of rage. I want to set it on fire. I think less of people who like it.
Please leave any thoughts, comments, complaints, criticisms, concerns, or death threats below.
Some of you have very specific goals in mind when you create a game. Some of you are trying to tell a great story, some of you are trying to create fun puzzles, some are trying to create awesome and epic battles, and some are trying to show off your art or music. That’s all well and good; game-making can be a perfectly legitimate artistic endeavor and I do not intend to diminish that. But you have to understand not everyone is going to see it that way. People who download your game are looking for a fun game experience, and they’re going to have certain expectations of your project.
Some of you like to break down the components of a game in terms of their weight or importance and plug them into some sort of ridiculous math equation. {Gameplay = Story + (graphics/ music^2)} This is asinine for a couple of reasons. The first reason is it doesn’t even really make a lot of coherent sense. The second, and more important reason, is trying to imply that some part of an RPG is more important than any other is pretty ridiculous.
For you to have a fully realized game, none of these aspects can be lacking. Sorry gameplay people, whine about how story is a waste of time all you want, in an RPG there’s an expectation that there’s going to be some decent writing somewhere. I don’t understand this movement to sort of hack out parts of a game the creator doesn’t like with some kind of meat cleaver and expect the game to be better for it. It’s like a house of cards. You can’t take cards away and call the resulting mess a complete project. It might still be fun in some ways, but I think most people would agree that something was missing. The same goes for story people. If you’re going to tell a story but neglect your “gameplay”, why include it at all? If you’re going to completely forgo the RPG mechanics in pursuit of your storytelling, the resulting story might be pretty good, but you aren’t really making a “game” anymore. You’re kind of making something else.
Now, I don’t want anyone to misunderstand. I am not saying your story needs to be enthralling, but it needs to make sense. Your graphics don’t need to be gorgeous but they should be consistent and functional. Your gameplay doesn’t need to be stellar but it better not be boring.
So while I’ll judge your game on the individual aspects that make up your project, I’m also going to judge it as a whole. And if your intentions are clear I’ll even try to look at it from your perspective, as you intended it. But the bottom line is I am going to tell you what’s good and what’s bad, what’s wrong with it, even if it wasn’t a part of your project you cared about. Because it’s still part of your project and that part of it still reflected on the overall game as a whole.
Now, how can you minimize this? If you are not good at a certain aspect of a game, how can you make it so it’s not a problem? By playing down those aspects. If you’re not good at writing, don’t dump lots of long, badly written conversations on me. If you’re not good at making puzzles, don’t put a lot of puzzles! If you’re bad at balancing combat, don’t throw me into a fight every four steps. If a problem doesn’t bother me much, I’m not going to get hung up over it. If I don’t notice that there’s a problem at all, then it’s a non-issue.
Now, supposing you see a review you don’t agree with. It could be written by me or someone else, it could be for your game or someone else’s project. It is very important to remember that reviews are one person’s opinion. Getting a bad review is not the end of the world. It does not mean no one in the world likes your game. It also does not mean you should flip out and get defensive. If someone took the time to actually review your game it means that had a vested interest in it. If they didn’t care they’d simply not say anything at all. You should be thankful that someone, anyone, took the time to comment about your game and try to help you improve. Disregarding a review or claiming a reviewer has suspect motives just because they gave a negative review is not only offensive and disrespectful but ultimately a disservice to yourself and your fans.
By request, I am including my general reviewing guidelines here so that people know generally how I will look at a game and rate it.
My reviews will usually start with a short backstory about my understanding of the project and why I chose to review it. I will then provide a brief synopsis of the game’s plot to let the player know basically what they can expect going in. From here I rate things by the following categories.
Balance: This is my general impression of how battles in the game were handled, including any aspects of the game that tie into this including optional or additional systems or character abilities. I will judge by how mechanically interesting they were, if they were fun, if they were of an appropriate level of challenge and if everything generally worked the way it was supposed to.
Level Design: This is where I will talk about the general layout of the game and all the things it encompasses. Things like graphics will be discussed here, though I will generally not mention them unless they are very good or extraordinarily bad. Things like town design, dungeon design, puzzles, including aesthetics, will be included here, and if I have any concerns about the controls or interface, this is generally where I’ll mention them.
Characters: I will discuss my thoughts on the writing, dialogue, characters and their subsequent development. If I have any in-depth questions or thoughts on a character I will go into it, including any things I liked. If I have worries that a character seemed under-developed or seemed weak, it generally means I wanted to know more so shouldn’t be construed as necessarily a bad thing.
Story: I will discuss narrative structure here, including the overall plot, and any storytelling methods used. Any unconventional storytelling methods aside from basic, chronological linearity will probably be mentioned, and I will also discuss the nature of side quests or plot branches, if any. If I notice any stylistic influences, such as a famous work or author, I will usually give those a nod here as well.
Music and Sound: I will talk about how any musical choices used in the project added or detracted from the overall mood or atmosphere of the game, as well as pointing out any times I felt sound was being used inappropriately. Having “okay” music generally won’t garner much of a response from me, but having really great music will really impress me. I will generally favor music I haven’t heard before over popular commercial rips.
Finally, the general scale I use to “rate” projects is as follows. My ratings tend to be fairly arbitrary at times but this is a general indicator of what I thought.
5.0- A near perfect game. It must do nearly everything right as well as go above and beyond expectations in every way. I do not expect I will ever give this score, but you never know...
4.5- An exemplary game. Not only does it do almost everything right but goes the extra mile in terms of content, programming, or some other area that sets it above other games. Bugs are all but non-existent.
4.0- An excellent game I recommend people play. It has few flaws and those it has generally don't detract from the overall experience. Almost no bugs.
3.5- An above-average game. It does a lot of things right but unfortunately has a few flaws that drag it down. Very few bugs.
3.0- An average game. It has a mix of good qualities and poor design choices. Not everyone may like it. There may be bugs but they do not detract from the experience.
2.5- Still pretty average, but here the design flaws are very noticeable. One or more aspects of the game are severely lacking which detracts from the experience. There may be multiple obvious bugs.
2.0- A below average game, it has a lot of faults that overshadow its good points. There may still be a good game inside...somewhere, but it's not there yet. It is very hard to get a lower score than this from me.
1.5- A poor game. The game is severely lacking in multiple areas that lead to a bad game experience on my part.
1.0- A very poor game. A game with a score is generally a sign I did not think the author was trying or that they didn't care. The game is disjointed, mechanically broken and may be near unplayable. Even this is not a 0.5 from me, oh no, only a very special kind of bad gets a 0.5...
0.5- A blight on humanity. The game was offensive to me on a primal level. It has no redeeming qualities. It sends me into apoplexies of rage. I want to set it on fire. I think less of people who like it.
Please leave any thoughts, comments, complaints, criticisms, concerns, or death threats below.
Posts
Pages:
1
0.5- A blight on humanity. The game was offensive to me on a primal level. It has no redeeming qualities. It sends me into apoplexies of rage. I want to set it on fire. I think less of people who like it.
http://rpgmaker.net/games/154/reviews/668/
Kidding!
I think this is a good essay and an interesting read to those of us who suck at writing reviews.
I think this is a good essay and an interesting read to those of us who suck at writing reviews.
Now, supposing you see a review you don't agree with. It could be written by me or someone else, it could be for your game or someone else's project. It is very important to remember that reviews are one person's opinion. Getting a bad review is not the end of the world. It does not mean no one in the world likes your game. It also does not mean you should flip out and get defensive. If someone took the time to actually review your game it means that had a vested interest in it. If they didn't care they'd simply not say anything at all. You should be thankful that someone, anyone, took the time to comment about your game and try to help you improve.
"That's right you filthy mongrels! How dare you expect someone to actually LIKE your game. You should be GRATEFUL to anyone took the time to explain to your retarded ass how terrible it was! If someone savages your game, wipe that blood off of your face, look up meekly and say..."please sir, may I have another"?"
-Drill Sergeant Max McGee
I'm sorry but I think you've hit on possibly the exact reason this community's amateur mentality holds it back. Because people spend hundreds of hours working on games and then they feel like other people are doing a favor by playing them. Besides fanfiction writers (I just read that Shadowtext article about the arrogance of storytelling) we have to be the only people in the world who work so hard on something and then are grateful (or at least expected to be) when someone plays it. Everyone else in the world who makes anything has at least some regard for the flip-side of the coin, which is that the creators are doing a huge service for everyone else, by making something that entertains them. Imagine how much better the creators would be doing if they had as much confidence as they do humility. Oh god, actually...you know what. I changed my mind. Don't think about that sentence, it's way too easy to invert. Because if I can use what I've seen as any kind of an example, we're all swinging violently from self-obsessed egomania to self-effacing insecurity all the time.
I hope my off-the-cuff comments lead directly to another thought provoking article. I can't imagine I didn't influence this one, however indirectly.
I actually wrote this article directly at kentona's request. The above might be applicable to you but it was not directed at you specifically.
"That's right you filthy mongrels! How dare you expect someone to actually LIKE your game. You should be GRATEFUL to anyone took the time to explain to your retarded ass how terrible it was! If someone savages your game, wipe that blood off of your face, look up meekly and say..."please sir, may I have another"?"
Let me ask you something, Legion. Does the phrase "Straw Man" mean anything to you?
we have to be the only people in the world who work so hard on something and then are grateful (or at least expected to be) when someone plays it.
No. This is something that extends well beyond the RPGmaker community. Amateur authors are competing with a great deal of other things for people's time, and I'm not only talking about main stream entertainment.
Everyone else in the world who makes anything has at least some regard for the flip-side of the coin,
Everyone possesses this quality. The RPGmaker community is not made up of an identical legion of automatons hell bent hurting your feelings. The world isn't that simple, Legion.
which is that the creators are doing a huge service for everyone else, by making something that entertains them. Imagine how much better the creators would be doing if they had as much confidence as they do humility.
Confidence comes with time and practice. The confidence you preach of is a false confidence; a confidence born of ignorance (Yay - Mass Effect quote). You don't gain this confidence when people are reluctant to tell you what's wrong with your game; you don't move forward when you're not challenged.
Oh god, actually...you know what. I changed my mind. Don't think about that sentence, it's way too easy
to invert. Because if I can use what I've seen as any kind of an example, we're all swinging violently from self-obsessed egomania to self-effacing insecurity all the time.
I hope my off-the-cuff comments lead directly to another thought provoking article. I can't imagine I didn't influence this one, however indirectly.
No, that's just you.
I've been sitting here for ten minutes at a loss for words. I can't decide how to close this up. My mind goes around paces around your post trying to figure out just what the hell your problem is. You take issue with criticism? Do you think people should be nicer? You want to build confidence in the community?
If you want to build confidence in the community, a good start would be advocating for higher standards, encouraging collaborative projects like Ill Will, maybe not throwing a giant fucking hissy fit when someone says something you don't like about your game.
True confidence comes with practice. It comes with tenacity and diligence; there are no shortcuts.
You do have a good point, Max. As much as a person can improve by welcoming constructive criticism, I think you'll agree that the best works require the confidence that comes with believing people will enjoy that work when you create it. However, I don't think Soli's ideas are wrong. We are a community of amateurs, and we don't have the luxury that professional writers in professional mindsets have; professional criticism. A number of a game's downloads doesn't offer nearly the same level of assurance of your product's quality as, say, sales numbers (which in itself doesn't have to correlate with the product's quality, as I imagine many creators would consider getting their message out to as many people as possible a greater victory).
In context to this article, I see no reason why we SHOULDN'T be grateful for the amount of feedback provided in a review, positive or negative. I think it's the creator's responsibility to size up the overall response to their creation. One person's opinion shouldn't affect their confidence (which I'd venture to say people either have or don't have, regardless of taking the advice of an article like this), especially if they receive another review or comments with conflicting viewpoints. The combination of these things, and to a lesser extent overall downloads, helps us determine whether or not the confidence we have in our content is enough to entertain people.
I guess to put it another way, a creator should start with confidence when they create. After a release, remaining over-confident will make one oblivious to their shortcomings. On the other hand, relying too much on everyone's feedback will dilute the integrity of the creation, where the ideas the creator wanted to convey become lost. This is inherent. I think what Soli wants to address more specifically is that there is no benefit to lashing out against criticism or even discarding it outright. The greatest benefit to all types of criticism is to weigh the strengths and weaknesses outlined in them against other comments and criticisms received, and then through your own confidence. In the end, you may dismiss a comment and conclude, "they just didn't get it," but it does more good than harm to first consider why that might not be the case.
In context to this article, I see no reason why we SHOULDN'T be grateful for the amount of feedback provided in a review, positive or negative. I think it's the creator's responsibility to size up the overall response to their creation. One person's opinion shouldn't affect their confidence (which I'd venture to say people either have or don't have, regardless of taking the advice of an article like this), especially if they receive another review or comments with conflicting viewpoints. The combination of these things, and to a lesser extent overall downloads, helps us determine whether or not the confidence we have in our content is enough to entertain people.
I guess to put it another way, a creator should start with confidence when they create. After a release, remaining over-confident will make one oblivious to their shortcomings. On the other hand, relying too much on everyone's feedback will dilute the integrity of the creation, where the ideas the creator wanted to convey become lost. This is inherent. I think what Soli wants to address more specifically is that there is no benefit to lashing out against criticism or even discarding it outright. The greatest benefit to all types of criticism is to weigh the strengths and weaknesses outlined in them against other comments and criticisms received, and then through your own confidence. In the end, you may dismiss a comment and conclude, "they just didn't get it," but it does more good than harm to first consider why that might not be the case.
I guess to put it another way, a creator should start with confidence when they create. After a release, remaining over-confident will make one oblivious to their shortcomings. On the other hand, relying too much on everyone's feedback will dilute the integrity of the creation, where the ideas the creator wanted to convey become lost. This is inherent. I think what Soli wants to address more specifically is that there is no benefit to lashing out against criticism or even discarding it outright. The greatest benefit to all types of criticism is to weigh the strengths and weaknesses outlined in them against other comments and criticisms received, and then through your own confidence. In the end, you may dismiss a comment and conclude, "they just didn't get it," but it does more good than harm to first consider why that might not be the case.
You bring up a good point; it's important to remember that you don't HAVE to do what everyone says. Nobody reasonable advocates this.
S. F. LaValle:
Paragraph #1:
I feel that the gap between professional and amateur criticism is much smaller than the gap between professional and amateur art. This is not a counterpoint just an observation.
Paragraph #2:
Yes, in theory. Of course the system could be improved so that review scores and feedback level and download numbers better correlated with game quality. But the two caveats are that game quality is not an absolute and that this is an ongoing process. Again, just an observation.
Paragraph #3:
Absolutely! On one level I am grateful for every piece of feedback I get that isn't overtly hurtful or dismissive. And I have to say that any positives from any criticism I've received that actual had COMPLAINTS about the work in question were much more inspiring to me than any kind of unadulterated, sycophantic praise which generally seems like it came from people without discriminating standards. The goal is to impress the toughest critics.
On the flip-side of course I think we should all be grateful for every minute of entertainment we get from other people's games- even the ones that are mediocre or merely "okay"- and the hours of work behind those minutes of entertainment. I suppose what I advocate above all else is respect for the creator in 99% of cases.
But I don't believe I actually disagree with anything you said, S.F. LaValle.
Paragraph #1:
I feel that the gap between professional and amateur criticism is much smaller than the gap between professional and amateur art. This is not a counterpoint just an observation.
Paragraph #2:
Yes, in theory. Of course the system could be improved so that review scores and feedback level and download numbers better correlated with game quality. But the two caveats are that game quality is not an absolute and that this is an ongoing process. Again, just an observation.
Paragraph #3:
Absolutely! On one level I am grateful for every piece of feedback I get that isn't overtly hurtful or dismissive. And I have to say that any positives from any criticism I've received that actual had COMPLAINTS about the work in question were much more inspiring to me than any kind of unadulterated, sycophantic praise which generally seems like it came from people without discriminating standards. The goal is to impress the toughest critics.
On the flip-side of course I think we should all be grateful for every minute of entertainment we get from other people's games- even the ones that are mediocre or merely "okay"- and the hours of work behind those minutes of entertainment. I suppose what I advocate above all else is respect for the creator in 99% of cases.
But I don't believe I actually disagree with anything you said, S.F. LaValle.
Pages:
1