FOUR STARS...FOR WHAT ??

Posts

Agreeing with Liberty here. You guys have already gone over everything I suppose though >_o

I'd imagine that the reviews section might turn out like Amazon if you were to allow super short reviews. But worse, since there's not a "helpful" score to show which ones can be considered game development critique and not kneejerk player reactions.

Well, I'm assuming that reviews fulfill two purposes: to inform potential players of the game's ups and downs (helping them decide if they want to play the game), and to inform game developers of what worked and what didn't work. As such, short reviews that are vague and provide no direct examples of game content and how it contributed/detracted from the overall game experience are no better than feelings rants (ala OMG!! SO GOOD U NEED TO PLAY IT this game is sooo cute<333, except with less teen girl lingo). If there are short reviews that do fulfill those two purposes, then great. If not, well, we have the comment section.

As for "How to encourage people to post more reviews?" I'd say that the template suggestion is pretty nice as a guide for people who need help organizing their thoughts into words.

Video/audio reviews might also be nice for people who prefer to talk rather than type! Podcasts and podfics are going strong, so I'd suppose podreview wouldn't be bad, right? ... Right? This also ties in with the suggestion that LPs should include a right to cast a rating, if LPs are considered like live reviews. (Which I personally think do count, since they show developers how players really interact with and experience the game).
author=Liberty
I think that it's definitely something we're going to have to bring up at the next staff meeting - that's for sure. The system is flawed, we all recognise that, but the ways suggested so far don't really work in practise so we're going to have to look at a new way to do it...

The thing is, when has any suggestion been put into actual practice for us to see if they work or not? Never. Any time an idea is put forward, it is promptly shoot down with just other ideas; opposite ideas. And at the end of the day we're all so fed up with the discussion that we'd rather turn a blind eye on these issues, rather than to work on possible solutions.

Seriously. Nothing said in this thread is something that has not been mentioned before countless times, for the past, what? 4-5 years? I find it very difficult to believe that in all that time, these issues have not been discussed at a staff level... Also, we want to feel part of the decision making process, not just hear a "We decided against it" coming from the top.
_
All ranting aside. The first thing we need to realize is that no system is perfect. There will always be room for abuse regardless of any system we adopt. The only difference between a successful system and one that fails miserably, is how vigilant we are, as community, to prevent it from being abused... But right now, the community has a very limited "tools" to make a difference.

So, yes -before anyone mentions it- even our current system could work wonders if only people were to write more reviews. But the thing is -and I've said this before- that you can twist people's arms to accomplish this. People who don't find joy/don't have time/don't have the skill to write reviews, are never going to write more reviews that are being written right now. Specially if we expect for the quality of reviews to go up, instead of down... So at the end of the day, is easier to change the site than to change the people.

On the flip side, any change done to the site falls upon the shoulders of just 2-3 guys. Not to mention that the architecture of the site may prevent for any real change to take place. This is VERY important to keep in mind. But it should not stop us from discussing alternatives. How viable those alternatives actually are, how long could they take to be implemented, etc. (No one expect change overnight, but we're long overdue for change to start happening). Improving the site's functionality is everyone best interest.
_
All ranting aside (for realz this time... or is it?). Let's talk alternatives. (Most of this has been said, but I just want to state my opinion)

I think ratings and reviews should be separated. From what I gather from this thread, this idea has finally gained enough traction for us to consider it seriously... Make it so we can see who has rated what games and what scores they have given. Perhaps even what individual score has been granted to each of a game's department (Writing, graphics, etc.) together with a small reason behind said score (ie: Graphics: 2/5 "The game mixed several graphics styles"). And before anyone says anything cute: No, this is not meant to replace or compete with reviews. This is just so ratings by themselves provide a modicum of information to the developers, and so we can use this information to ensure the fair use of system. For example, if we notice someone is giving unfair scores to competing games, we can report it. Or perhaps even allow us to thumb up/down any scores ourselves to further offset any possible biases.

Also, stars need to be replaced for a less subjective metric, because not even four stars cut it, specially if we'll still be able to give half-stars. For example, right now if you by the literal meaning, "average" means 2.5 stars; but if you go by the colloquial meaning, it is 3 stars. And this sort of discrepancies will be inherited by a four stars system... No, you need to spell these things out for people. Something like: "Bad, regular, good". Anything from three to ten stages could work. Anything but a binary, please! xD

As for reviews, you know all for raising their standards. The current minimum is too low for a review. But more than word count I think we should focus on content. It is very easy right now to find a review that lacks in content in one way or another, even if they're several paragraphs long rants essays... I like the template idea, but not so much a template for reviewers, because then people get the idea that every review will read like a carbon-copy of each other, even if that's not necessarily true. The template should be more for mods, a laundry list of sorts to determine if a review covers enough ground to be accepted or if is sent back to be improved.

Well, I guess that's all, from the top of my head. But obviously, these subjects are too nuanced for any single person to cover everything.
What people don't realise is that we do allow LP/video commentary reviews! They've been allowed for a while now. All that we ask is that you include a summary of your thoughts along with the review~ So go forth and video if you so wish. I don't know about Soli, but I've yet to see a video review in the queue (but then I'm not in charge of reviews and I've only been on the queue for a while.)



Also, I forgot to add that this is now up. If you do have problems gaining the 300 word mark, do check it out. I might actually do a general writing tips article at a later date for our non-English-as-first-language writers so that they can refer to it for common mistakes when writing the terribad language that is English.
Rave
Even newspapers have those nowadays.
290
Kaempfer, your D3 example is bad. People didn't like it because it wasn't D2. Can you say Auction House? That's why people didn't like it. Now it will be/is closed (don't know as I don't like hack'n'slashes) I expect it to spike up. And reviewers of AAA titles are paid anyway.

//edit:
author=Liberty
All that we ask is that you include a summary of your thoughts along with the review

author=accha
Video/audio reviews might also be nice for people who prefer to talk rather than type!

http://tvtropes.org/pmwiki/pmwiki.php/Main/Comicallymissingthepoint
A summary is simple. It's not even the 300 character limits!

For example:
Overall the game was decent, the graphics were well-utilised and the gameplay was fun. I highly recommend it for anyone who enjoys cute characters and puzzles galore!

Is that really gonna kill anyone?
@alterego The individual score ratings sound pretty cool and also useful!

@Liberty When you guys fix the characters->words thing on the review page you should also include that bit about video reviews then, so people who don't roam the forums can know too XD
It was suggested adding a "Was this helpful" option to the reviews, and the more helpful a review the higher up it would be.

What about doing something like this for comments, and then letting the viewer sort comments either by first to last or by most helpful?

That could give the developer an opportunity to sort the best comments first if they're looking for immediate feedback that may or may not have been covered in the review.

Fleshing out the score to a point value would also be nice, or at minimum including quarterly points. I had a game that was 2.83 and had to choose a final star rating of 2.5 (which I felt was too low) or 3 (which I felt was too high). I wound up with 3 since the story was very well done and didn't want to cast the game negatively anymore than I may have done. Being able to select 2.75 would have been a better choice because it would have shown it's a bit better than subpar, but still needs quite a bit of work.

I take scores with a grain of salt, but not everybody does. So I think it boils down to people being able to select the most reasonable score, and readers understanding that scores are going to be biased and opinionated. However, being able to have more flexibility in scoring and being able to vote up or down whether a review or comment is helpful may help.

An end of the year contest for best written review could also encourage more.

Finally, maybe a staff position for review writers could help. Public reviews could continue, but any review that has an official staff label would tell the audience it's written with the least bit of bias as possible, as the staff would be trained to review the game with the mindset of the audience it's targeting.
LockeZ
I'd really like to get rid of LockeZ. His play style is way too unpredictable. He's always like this too. If he ran a country, he'd just kill and imprison people at random until crime stopped.
5958
I actually think our current review system, if we changed the characters back to words, is perfectly fine and does everything that matters. It discourages people from submitting crappy reviews and encourages them to submit good ones. The only people "upset" about it are the ones who want their voices to be heard with minimal effort on their own part, but your voice being heard doesn't benefit the community. You should be sharing a review because it will help inform other people about things they couldn't learn just by looking at the game page, not just because you want to give a game a score.

In short, it is causing zero problems.

If it ain't broke, don't fix it.
author=LockeZ
I actually think our current review system, if we changed the characters back to words, is perfectly fine and does everything that matters. It discourages people from submitting crappy reviews and encourages them to submit good ones. The only people "upset" about it are the ones who want their voices to be heard with minimal effort on their own part, but your voice being heard doesn't benefit the community. You should be sharing a review because it will help inform other people about things they couldn't learn just by looking at the game page, not just because you want to give a game a score.

In short, it is causing zero problems.

If it ain't broke, don't fix it.

But it would benefit the community.
For example take a game that two people liked so much they felt like reviewing it. So they both write pretty in-depth reviews on why they liked it so much and rate the game 5/5 both. Now it seems like this is a very good game.
But now if you allow people to rate the game without reviews (let's just take this as theoretical example), instead of just 2 review rating the game we would have say 200 votes on the rating and it averages to 3.45, so the game is not as perfect as the reviews made it seem.
So the benefit is that the rating is much more accurate.
LockeZ
I'd really like to get rid of LockeZ. His play style is way too unpredictable. He's always like this too. If he ran a country, he'd just kill and imprison people at random until crime stopped.
5958
That's not more accurate. That's just more people.

It was better as just the two good reviews.

Reviewing or rating a game changes the presentation of a game and is an addition to the game page. It should therefore be held to at least the same standard of quality as submitting a game page, and probably much higher especially since you're doing it to other people's games without their permission.

Changing other people's game pages and altering one of the most visible aspects of public perception of their games without their permission is not a right people just automatically deserve to have. It should be reserved for those whose submissions are exceptional.
The more people, the more accurate the score.
LockeZ
I'd really like to get rid of LockeZ. His play style is way too unpredictable. He's always like this too. If he ran a country, he'd just kill and imprison people at random until crime stopped.
5958
Visiting the theater doesn't make you qualified to be a film critic.
Thank you, LockeZ - that is what I've been trying to put forth. Putting effort into a review because it affects another persons game's reputation. I'm not one of those who believe that allowing everyone an instant vote will equal equality. For one, older games will be denied the chance to be 'loved'. For another, people are stupid and judge only on what they see. If we open it to 100 words or less then people will just comment on what they see without any thought or even having to play the game, they won't bother giving real feedback or any kind of depth to their reviews and that will not skew well.

It really doesn't take much to write 300 words. Why are we arguing about this still. There are tutorials about writing reviews for anyone who can't - pretty good ones too, to help you pump up your word count and put down thoughts that can help the game creator.

300 words isn't a wall of text and if you can't be arsed to write that minimal amount then you shouldn't be giving thoughts on games outside of the comment section. It requires a little thought and if you don't put at least a little thought in, you definitely should not be rating a game.

How many times does that need saying, really?
author=LockeZ
Visiting the theater doesn't make you qualified to be a film critic.

What's the point in a professional critic saying a game/movie is great when the majority of players/watchers doesn't like it?

Well, to be more exact, a review or critic is usually most useful for the creator. He can learn what he did right and what he did wrong and learn from mistakes for future projects. But when you are the player/watcher, what matter more for you is "How high is the chance that I will actually enjoy this" and in this case, votes from many persons give you a more accurate result.


@Liberty
I think you are still missing the point. 300 words isn't much and anyone who really wants can easily write a 300 word review. That's for sure and nobody doubts that either. It will however not change the fact that not many people will write reviews at all because many of them are too lazy to even write 300 words, whereas they wouldn't be too lazy to click on a score.
Then they shouldn't get to. If they are too lazy to write something about the game why should they be allowed to change the score of it without at least having played it or showing that they have an opinion besides "I clik butun durrrrr"?

Frankly, after reading and writing in this topic I've changed my mind. The current system is actually pretty good for what we aim. Most of the members on the site are creators and thus request information about their games. If linking reviews to scoring allows for more reviews (because, honestly, if we opened scoring up to just button clicking you can bet the amount of reviews would plummet) then we should stay that way.

And don't think for a moment that people would continue writing reviews if they could just give their opinion with a button click. Some would, yes, but for the most part people would just hit a button and maybe leave a comment like "Nice game. Keep it up. 5 stars."

No. Fuck that noise.

You want to rate a game? Write a fucking review.
You think a score is too low or too high? Write a fucking review.
Have something to say about a game? Write a review.

Do the work, get the reward - that reward being your opinion making a change on someone else's game. Otherwise leave a written rating in a comment.

Oh, and for your argument that the score would better fit the game? No. No it wouldn't. Why? Because people would just look and click like the lazy fucks they are (how many games which have great CSS and mapping have more downloads and comments like "This gam r luk grate!!!!!" than ones like Manifest which has solid everything, as an example?)
Maybe they would play - maybe - but they would instead base the game on how it looks on the game page when they pass through. Maybe offer a thumbs up/down feature for passers-by but not an actual score. That should be for those who actually take the time to consider the game and write something (300 fucking measly words folks) about it, because fuck fly-by scoring. It is NEVER an adequate judge of what a game is like.

It might work for an image you look at but not for something you have to actually invest time to download and play. We aren't DA or a site like that where the item in question is right there in front of your eyes and you can just tap a score and know that your (and others') opinion/s were informed because what you were judging was right in front of your fucking face.

No.
Games require a download. They require playing to actually gain a proper opinion and you can't police that aside from actually getting people to write about their experience in the game itself (hello reviews). You can't just allow for a simple button press based on first impressions to rule how a game is ranked. It does not work.
I can't think of any single situation where allowing more voices to be heard it is not a good thing. Of course, there's the concern that not everybody is going to use their voice adequately... But for one I can't believe that I, as the "wet blanket" I'm known to be, have more faith in the rmn userbase as a whole that some of you. xD Not to mention that ways to help quell those concerns have been suggested, and I'm sure there's plenty more to think of. But you're just too bend on your "It's my way or the highway" ...ways.

Ratings by themselves may not seem like a significant piece of information, but they're still an important piece of information. Try to remove them and see what happens. And in their current form, attached to reviews, they're being underutilized or even abused. Maybe if you were to really put your money where your mouth is, if you really enforced review standards, if you really cared for other people's games, then perhaps people like me wouldn't feel so powerless. Because it shouldn't be possible to rate a game zero stars out of spite for what a game represents: "This game is utter trash. 0/10." And like that there are plenty, albeit milder, examples.

Also, the idea that there will be less reviews if scoring are made more accessible is one that I just don't see supporting evidence for. Something that people need to understand is that those who are write reviews right now are doing so out of their own volition. If it were true that keeping the score as a "reward" is helpful to encourage more reviews, I would have more reviews to my belt, LockeZ would have more reviews to his belt, Liberty would have more reviews to her belt (Even though she totes could =p). etc.......

But no. People write reviews do so because they can/want/enjoy writing reviews. There's always going to be people who write reviews periodically like Addit, enthusiast newcomers like Cashmere, pro-active people who organize/participate in Review based events like Liberty. And there's always going to be your average joes who writes reviews whenever they goddamn feel like it, pm everybody else. There's not going to be a significant decrease or increase in reviews no matter what you do or stop doing.

Also, also, c'mon. It's not just "lazyness". Allow me to put things into perspective. Let's say that to rate games you have to draw a piece of fanart. This is something that I, who has some experience and -very important- like doing, could do in a few minutes to a couple hours. But for anyone else it could take significantly longer. And I could write tutorials to make the task easier for you. And I bet you could do it. I mean, drawing something is not like super hard. But assure you that at some point, while you're struggling drawing a stick figure, you would be thinking: "Why do I have to put up with this? Why can't I use this time to play more games instead?" xD

So it's not like 300 words is too much. This very post is longer than that. I've written comment-reviews longer than that (link), I have a few draft reviews sitting on my hard drive longer than that. But as a non-native English speaker, who is not very good with words in his own language to begin with, writing a review for every game I play is something that I just cannot afford doing (no one cans, really). But this does not mean I or anyone else lack the ability to rate games fairly, and I would very much like if we were allowed to do it.

Like I said, there are ways to address any possible "worst case scenario" you can think of; you just need to be willing to discuss them.
author=Liberty
*Drops mic*
Well put. I don't think the current review system should change. A one-click voting system IMO caters to the lowest common denominator, and people would end up gaming the shit out of it to bump up the score of their game. The entire rating/review system would be meaningless if every game was between 4 and 5 stars.
Rave
Even newspapers have those nowadays.
290
author=LockeZ
That's not more accurate. That's just more people.

Lockez, learn basic statistics, mostly that funny thing called statistical significance so you can see why asking 1000 people to see which political party (example) is more popular is better than asking 10 people same thing.

//edit:
author=RyaReisender
What's the point in a professional critic saying a game/movie is great when the majority of players/watchers doesn't like it?
This, so much this.
Linkis
Don't hate me cause I'm Cute :)
1025
JUST A THOUGHT, DON"T HATE ME :)

What if there were reviews for the eyes of the developers only, but no star reviews.

The game page and screenies is what I use mostly to see if I will like a game.
Most people I feel like a few particular types of games. The search/sorting program lets us pick a group of games we like.
Then we go to the game page, read what it's about and check the screenies.

Like I said.....JUST A THOUGHT :)
LockeZ
I'd really like to get rid of LockeZ. His play style is way too unpredictable. He's always like this too. If he ran a country, he'd just kill and imprison people at random until crime stopped.
5958
The fact that the people with legitimate opinions are statistically insiginificant in the "anyone can rate" method is not a selling point. All you have to do is glance at ratings on places like Amazon and Ebay and Yelp to see how this would end up.

I died to the second boss. Zero stars!
I don't recognize the game you ripped the graphics from, so I assume they are original. Five stars!
Couldn't find the download link. Zero stars!
Used music from my favorite PSX game. Five stars!
Made an FF fangame, but Sephiroth isn't in it. Zero stars!
My friend made this game. Five stars!
I didn't realize this would be a horror game. Zero stars!
Haven't played it yet, but nice screenshots. Five stars!
No Portuguese translation yet. Zero stars!
Opening cut scene is nice, I'll play the rest tomorrow. Five stars!

This is exactly what will happen, I promise. Data needs to be verified for accuracy before being accepted, at least. Having quality control on reviews ensures that.