ARE WE TOO GENEROUS WITH GAME REVIEWS?

Posts

Solitayre
Circumstance penalty for being the bard.
18257
author=Sated
We're too generous with the standard applied to reviews that are allowed onto the site.

Why should someone have to aspire to professional review standards to be allowed to write a review?
Sooz
They told me I was mad when I said I was going to create a spidertable. Who’s laughing now!!!
5354
author=Pancaek
I more or less just meant: "Do some people give better reviews than what they actually think the game is worth just to be nice?"

Oh, is THAT the real question?

No. The answer is no. That is not a thing here.

Whatever one may believe re: the accuracy of a score, the issue is going to be more a problem of each reviewer's perception and personal definitions. Por ejemplo: What does "average" mean? Is it "conforms to the quality of the majority of games on the site"? "Conforms to the quality of the majority of games, period"? "...of all amateur games"? "Fulfills what I expect out of a competent game"? "Has no standout qualities, good or bad"? If a game's quality is extremely uneven, and I score my hypothetical review by adding and subtracting points, and then arrive at a 3, is that too nice because parts of the game are terrible?

Review scores will always be a difficult subject because it's trying to apply an objective measure to a subjective experience. Some people will think that a cool story or lovely graphics or great writing make up for shitty, shitty gameplay. Some people will be turned off of a game that plays beautifully but has nothing else going for it. Some people will latch onto a particular detail that, for them, increases the game's quality immensely. This is not a case of being "nice," just different people having different standards of quality.

And that is why, if you're not sure about a game, you should at least skim the reviews, rather than relying on the score to give an impression of how you'll like a game. A reviewer's opinion is only helpful to you, the individual, if you share that reviewer's general outlook on games.

Or, if you're like me, it's useful if the score is 2 or below and you like reading about terrible works. :V

OK realtalk mode off, back to shitposting.

ETA: Yes what this site for amateur games made with an easy-mode engine really needs is professional reviewing standards.
iddalai
RPG Maker 2k/2k3 for life, baby!!
1194
author=Solitayre
author=Sated
We're too generous with the standard applied to reviews that are allowed onto the site.
Why should someone have to aspire to professional review standards to be allowed to write a review?

That's not what Sated said in this particular post.

What he said is he believes review standards are low.
They're not. Just asking for good grammar/spelling/writing in general, over a word number threshold and making a case for or against the game is fine.
Corfaisus
"It's frustrating because - as much as Corf is otherwise an irredeemable person - his 2k/3 mapping is on point." ~ psy_wombats
7874
author=Isrieri
Here's an idea: Have our biggest reviewers (like, the ones who make it onto the monthly stats) be given a "RMN Seal of Quality" or whatever. Problem solved forever.


Most of those people made it big by spamming acceptable reviews when expectations were low. I wouldn't want them to give a "Seal of Quality" on anything.
author=Solitayre
Why should someone have want to aspire to professional review standards to be allowed to write a review?

Because, reasons.
author=Pancaek
...How would you know a game is that bad if you haven't played it?

Granted it is pretty easy to spot a low quality game, but at the same time it's hard to really accurately judge a game if you haven't played it. Maybe the art is really bad but the writing and game-play is outstanding.


This is one reason I really think any game which rests to any significant extent on the quality of its writing really should contain screenshots of dialogue or other significant text on its game page. Even if it doesn't reveal anything about the narrative structure, you can tell a lot about the quality of writing in a game from just a few lines.

Lacking any particular sign that a game is good though, I'll tend to assume most games are not, and I suspect most people on this site do the same. Most people don't touch the vast majority of games on the site, after all. So it's no surprise if review scores skew higher than "average," since not only are people less likely to review games they weren't enthusiastic enough to finish, but they're going to tend not to bother playing games in the first place which they don't expect to be better than average. If anything, the average unreviewed game would probably garner ratings significantly below 3 stars if members were forced to play and review them.
Sooz
They told me I was mad when I said I was going to create a spidertable. Who’s laughing now!!!
5354
I propose a supplementary score- rather than having the players write a full review, just put in a three-point scale: Crap/Meh/Good.

Then watch it turn into a binary scale because even if it's low effort barely anyone bothers to react to a thing unless they already have strong feelings.
No system is perfect.
I prefer: Stop looking at your ratings and just work on that damn thing.

I like the system used on Steam.
Thumbs-up or thumbs-down and description.
Everything can be misused but if you see something like: "50/10 would bang the heroine" and a thumbs-up, you know that one is not serious.
Although it can still be important to some people..... Hm.

It is either good or bad to a person and you can read why.
What is 4? Slight worse than 5 but slightly better than 3? Too many variables...
When I see a thumbs-down, I can read why the person does not recommend it and decide whether those points are fine with me personally.

And that is what it comes down to, I guess.
Do not look at the score/stars/thumbs but read what people have to say about it.

And objective rating is well... probably nonexistent.
Every person shows more compassion towards buddies.
I have also noticed in other communities that people who have no content there and are just "consumers" tend to rate more how they see it because they have nothing to lose or gain.

People do not like to be jerks. Lower ratings usually mean that but that depends on how the community nurtured itself, really.
Then they only rate good. Or they rate good so that their own work will not be lower rated or ignored. Fear of that when you have your own work at stake.

Experiences I guess. I am pretty surprised by this place. I see a popular name and their games are rated a varying amount of stars.
I come from a place where... everyone rates five and anything lower could be a problem.
You mean the broken-ass system on steam where people get others to up their game for them with no knowledge of who is giving their opinions? Where it's dead easy to pay people to write reviews that are unchecked and thus always have glowing recommendations despite the fact your game is shit? Where there's major discrepancy between truth and fiction? Where a ton of thumbs up are from who knows who and getting, say, a tumblr fandom to dive on stage and like your game for easy points? Where it's completely unfair and no-one has to give their reasons for why they like/dislike a game, and even when they do, there's no change in the overall 'score' of the game even though it might be a buggy piece of shit?

I think I prefer our 'broken' system, thanks. At least people can't cheat it so easily.

Lower doesn't mean bad. It means there's promise and work needs doing to get it up because it's not amazing yet. Where do you come from that 5 stars is the norm (so I can keep the fuck away from there)? That's completely horrible to use as your baseline. Bloody hell! How does anyone there ever improve? :/

It's not being mean to point out the issues with a game and give points to encourage people to build better shit. Just... noooooooooooooooooooooooooooooooo
The system on Steam isn't any more broken than the one we have here. If steam had the same five star voting system as this website, people could still pay people to give their game five stars, or "get others to up their game for them." The "thumbs up or down" system has nothing to do with it.

I'm not personally educated enough on the whole situation to say if people really do pay people, or encourage people without reason, to up their games, or not, on steam, however I can say that I personally have never played a game on steam with good reviews that I didn't enjoy.

Also, if that where the case, I doubt games like Umbrella Corps, made by large companies, would have a review rating of "mostly negative" when they have the money to pay people off, and have a huge fan following.

I find the Steam voting system to be really good, actually. It's easy to pay off 10-20 well known reviewers, but when everyone can be a reviewer, it's hard to throw cash at literally everyone who plays your game. But then again, like I said I'm not really educated on the whole thing, so take my words with a grain of salt.
No, it's extremely broken. The one here actually ties reviews and scores to people. If they deliberately try to flaunt the system, they get called on it (as has been done in the past). There is no accountability on steam at all. It's a broken piece of shit that serves only those who know a whole bunch of people and/or are capable of bribing people to give their game a good review. That doesn't happen here because the moment anything like that is found out, it's lights out for the person doing it. Banned.

They do pay people - it's a well known fact. I've seen it happen. I've seen games that have had stolen and ripped assets put up for sale and had the developers remove reviews by sending in hordes of idiots to downvote the reviews to oblivion so that they don't show up on the front page. If a review isn't seen - if the effects of a review isn't seen - it might as well not exist at all.

On RMN reviews are actually well monitored. People read them, they play the games and then they comment on them.

The reviews also mean something, unlike on Steam. They can't be hidden or easily ignored. They actually contribute to the score of the game - which is what they should do. Informed opinion, backed up by an actual argument, is much better than nameless faces just hitting a button. You have no idea or reference for anything with a button press, but with a review that has a tied score the reviewer actually has to make a case for the score they gave.

They can't just hit 5 stars and say it was great, they have to say why it was great - and woe betide them if they lie because that review is tied to their name, their account. They are -known- and if they lie and try to cheat the system they become known for doing so. Their reputation is tied into their reviews, so there's a big problem for them when they review if they don't do so truthfully. Again, as people have found out when they tried to cheat the system.



Considering the laxity of our past review standards and the amount of games that still didn't have reviews VS the amount of reviews we get now with actual standards... we get a hell of a lot more now than we used to.

We've already discussed this topic to death, though, and always come up with our current review system being the best FOR THIS SITE. What works for Steam or Whatever-Other-Bullshit-Site doesn't work for a site of game developers who also happen to be players. Go read up on other topics if you want to know more - there's a literal fuck-ton of 'em around.
You're basically saying the "thumbs up and down" system on youtube is broken, because it's the same thing. Youtube use to have a 1-5 star rating system, but they changed it for a reason. If people are paying off other people to bump up their games, then that's a problem with humanity, not the rating system. If you're suggesting that steam should have to approve reviews before they're posted, then I hate to tell you that with the amount of games and reviews, that's basically impossible.

I personally agree with Madjak91, that a simple "vote up or down" system, along with the writing why you feel that way, is better than the five star rating system. However I can see where you're coming from, and really I don't think it would make much of a difference either way.

It's fun to talk about, but at the end of the day it doesn't really matter a whole lot.
It is. It very much is still a pile of festering shit. That's why no-one pays attentions to ratings (if they've got half a brain at least).

The difference between the youtube 5 star system and the one here is that there was no culpability on youtube. People could and did rate shit without any reason left as to why. That's why it was shit and replaced. They didn't replace it with something better, though. Just more shit.

There's a huge difference between dropping a like/rating and actually writing a review to defend/explain the rating you give a game. A huge difference. If you can't see that then you might need some glasses or something, because it's pretty damn easy to see.



On second thought, the up or down rating system probably wouldn't be great for this website, since the community is so small. If one person up voted a game, it would show the score as "overwhelmingly positive" and if one person down voted a game, it would show the score as "overwhelmingly negative." Which would be a problem since most games here don't even get more than one or two reviews.
I.... I wanted to discuss this but...

It is pretty obvious to me that to you personally, everything around here is perfect. Maybe because you are here for a long time or some "Internet home" thing. I get that actually.
See, communities usually devolve into a selected group of repeating users you see everywhere and new guys will not stay around for long because they are not in the group while the comfortable group thinks everything is perfect.

I am saying this because outside people might see it differently and I guess that is what Pancaek wanted to discuss.

Also, I can call an army of friends here to either harass or praise a game. Chances are you will not be able to tell.
Also 2: Pom Gets WiFi cough cough

Steam or bigger places are just well, bigger and more attractive for developers to target but also misuse.
While small communities stagnate and revolve around the same users over and over again and develop a "depends on who you are" kind of rating/reviewing/etc.

In the end, people always kick for their own team.

A system which is best for its own site is best for certain users and nurtured by certain users.
Steam or RMN.

Again, on Steam you can click the thumbs-down ones and read yourself.
Just as I will not immediately trust that a game is 5 or 2. I will read what people have to say and they can suck up to it or hate it just the same.
Steam is just bigger than a small group controlled community.
@Pancaek:
Yes. That is why actually tying scores to reviews - where people have to explain why they're giving that score to the game - works for this site more than it would for bigger sites. Though even if we did get bigger, I'd still push for that system - imagine the curves that would appear with more people writing more reviews! It'd be a lot more fair than any up/down nonsense could ever create since you'd know the scores had to be explained, not just dump-and-run.

I mean, you'd know that someone didn't just look at the images and decide to hit like, despite the fact it might play like a piece of buggy bullshit - they actually have to play the game and prove that before they can score, because the comments in the review would show whether they actually did play or not.

Reason > numbers any day


@MadJak:
Again, on steam, thumbs down are often drowned out by fuckwits who are paid or convinced to play a game or review because they're friends and there's no-one there to tell them not to do that or hold them accountable. The amount of games I've had to refund because the reviews actually made them sound good is astonishing, sadly. That aspect of steam is a broken pile of lies and shit. Seriously. It's incredibly stupid and doesn't work at all.

I don't think RMN is perfect, but again, the review system is fine. It can be improved (which we've talked about in other topics - basically tying scores to versions of the games instead of leaving old build scores to drag down game numbers, but most people are in agreement that the reviews + scores tied together work for the site and the amount of people on it. Reason beats fuckwits who never even played a game hitting like because they think it looks pretty.)

I've been on many other sites and most of the ones that have had decent games show up over and over again are the ones with starred ratings+reviews. Why? Because someone actually putting the effort to write about a game when giving a score and describing and explaining why the game deserves that score will ALWAYS beat out some random bunch of idiots pressing a button. Always.

I don't get the dig at Pom. It was a legit good game. People didn't like it because they were jealous of the attention it was getting, not because it was a shitty game with good reviews. You might need to find another example or something because that one doesn't fit.

Here's a good one: Pocket Mirror - on bigger sites it's hailed as some kind of god but here it's been given an actual fair score because it's not that great a game. Yet. It has the potential to grow into one, but currently it's all flash, no substance. The score it has on RMN is more accurate a representation of what the game is really like than the thousands of likes it has elsewhere.
Sooz
They told me I was mad when I said I was going to create a spidertable. Who’s laughing now!!!
5354
You kids really have strong feelings about rating systems. Like, "need to go outside and quit typing multiple paragraph rants over and over" feelings.

There is no perfect or ideal system. If you don't find the scores trustworthy, just skim the damn reviews that people spent actual time and effort on and decide whether you think their taste is similar to yours.

Yeesh.
I think most people here are just going back and forth for fun. I don't think anyone is really getting worked up or emotional about this. I hope not, at least.