REVIEWING

i played a game once
it was zero run gaiden
it made me hate life

Just to throw this out, many moons ago in a faraway land named "Utah"...

I heavily pushed having two types of reviews, both user and staff reviews. User reviews would pretty much be what WIP has now, with 0/5 stars and open-ended everywhere else. The staff reviews would by contrast be heavily based on the structured format of the reviews once featured on RPG Maker Database, written by guys like toadpole and others, which I was a big fan of.

These staff reviews would be massive articles that would obviously require serious dedication from the staffers writing them. The same categories of scoring would exist in all staff reviews, regardless of engine or type of game, with room for adjustment at the end to make the final score reflect the staff reviewer's final opinion. To settle differences between reviewers with higher or lower standards, final adjustments would be made by an administrator in charge. The long-term goal would be that all of these reviews would become standardized measuring for all of the finished (or incomplete, cancelled demo) games on RPGMaker.net.

Then, games could eventually be searched for based on specific staff review scores. For example, you could find a game with a high score overall, or, if you're all about gameplay and don't care about story, you could search for a high score in gameplay. Those are just some examples. The best feature would be that staffers would have to write one review every two weeks, with the ultimate goal of having at least one staff review for every game on RMN.

Erave
But sometimes some games will just fly under the radar unless someone says, ok you review these games.

With all that said, I'm not sure staff reviewers are really necessary. What might be better would be a more technical solution that relies on the community more. That is, anybody on RMN could set up a 'red flag' on anyone's game that indicates he'd like someone to play and review it. This is displayed on the front page of the website in a category titled "looking for review," similar to the latest game and latest submission categories. The games enter the queue based on the order in which they're flagged and exit when a new review is submitted and accepted for the game. Games can be flagged and reviewed as many times as desired, though logs are prepared when users flag games and abuse of review flagging would be punishable by ban.

Yes, this technical system is still open to abuse, but it sidesteps some of the maladies related to having a full-time staff. I did run a full group of staffers in all positions a few iterations of GW ago, several years ago (though WIP won't admit we did a good job :(). While it's true that having a group of staffers, a responsible boss, and specified expectations of quality work within certain deadlines WILL produce quality work regularly, there are new problems created by this.

You can't run a gamemaking website like a business, 'hiring' these volunteer staffers to write extremely high-quality material, and expect it to work fluidly, because businesses compensate staffers with pay and we don't do that here. Goodwill only extends so far. In my experience at GW, I instead created a venomous culture within the staffers where they competed over bullshit like "staffer of the week." They didn't care about creating material for the website's sake, but instead cared about being recognized as the best and eventually getting a promotion and more recognition a few months later.

RMN can't run like that. Even if we have a small group of review staffers, they'll either be a bad group of writers or they'll become so competitive as to become a liability. Even if we wanted these uber-reviewers to exist, I don't think it's even feasible in this instance: I worked with what seemed to be a much younger group of staffers at GW, and the highest performers treated it as their sole hobby. The great reviewers that we have here that are mentioned wouldn't be 'tricked' or coerced so easily into caring constantly about e-peen contests over indie game reviews.

We need to instead rely on the community. And as for those 'too-short' reviews, I say that we let them in. Obviously they need to be REVIEWS, not comments on the game (One of our most recent is a sentence complimenting the excellent screenshots...) And they need to be legible, and have a score attached, and not by pranks. Those are my qualifications for a review, really.

Brickroad
But then, WIP is a notorious drunk.

Yes... he really needs to stop hitting the bottle...

author=Fish link=topic=48.msg1410#msg1410 date=1183358145
I think there should be a spot of staff reviews, just because reviews mean more when they come from a more accountable or visible person. On a purely preference basis, who am I to care if 'randomrpgplayer232' didn't like the storyline or not, but I think I would take a note if WIP appreciated the story line's complexities.

We're all amateurs, though. I mean a lot of us know WIP to varying degrees and each of us knows (on a person-to-person basis) how much his opinion means to us specifically, but who's to say his opinion is really more important than 'randomrpgplayer232's' in general just because he runs the website?

I don't mean that as a slight to WIP, I'm just saying that running or being on the staff of RM.net doesn't really make your insight any clearer, it just makes your voice louder.

author=Shadowtext link=topic=48.msg1411#msg1411 date=1183359912
I don't mean that as a slight to WIP, I'm just saying that running or being on the staff of RM.net doesn't really make your insight any clearer, it just makes your voice louder.

Although this is the state of affairs when staff reviews are first implemented, this rule becomes generally untrue when these staffers are writing in-depth analyses at least every 1-2 weeks. Under a less stringent system, however, they could easily be what you described as simply normal members with a bigger mouthpiece. But creating mouthpieces like that might be the more responsible thing to do, if and when RMN grows from being full of relatively well-acquainted folks and into a place full of strangers. That's not something everybody wants, but it's a possibility if the site becomes even more successful, and I'd welcome it.

But like I said, what you said can be correct even in circumstances where the staff reviewers are churning out material every week. Going back to the example of GW reviewers, they were pretty knowledgeable, at least about RM2K games released recently back then, but they probably wouldn't have held a candle to some of the much more experienced #rm2k regulars who generally have played more games and have a much better vocabulary for and understanding of game development.

Darken
Ever notice how you can review a game even if it's not released? This might screw up the average rating of a game if overhyped abit.

Ew, yes. When reviews become editable and deletable this might be more satisfactory, but even then I'd prefer reviewing being available only for games with demos or releases uploaded. Comments should exist for all game pages for things like 'nice graphics lawl.'