Full Games List
Featured Games
New Games
Reviews
Media
Images
Blogs
Game Updates
Images Feedback Summary
Game Design Highlights
Tutorials
Articles
Scripts
Utilities / Plugins
Resources
Engines
Engine Bulletins
What is Makerscore?
The Simple Times has started!
Past Events
Achievements
Misaos
Forums
Post History
Chat
New to RMN?
Full Store List
RMN Music Pack (free)
REVIEW SCORING: STANDARDIZATION, PROFESSIONALISM, ETC.
Posts
Max McGee
View playlists
CloseMax doesn't actually think that, he was making a point.
This.
For example, Sai's review. The score was very low, to the point where I personally do feel it fabricated. But the intent wasn't necessarily, "Nico's game is too successful, time to take it down a notch." It was clear in reading the review, that Sai was saying, in so many words, "HEY! Everyone is gushing about your game, but there are VERY SERIOUS PROBLEMS with it." The score was to get people's attention. It was meant to ask people, "why does no one else have a problem with x, y, and z? This is bad design."
It's worth asking the question, 'if everyone but you likes a game, does it really have 'very serious problems' with it, or do those things not matter as much as you think? I'm not trying to argue against dissenting opinions, however.
People will still have different opinions of what average and good is. What is different is you know this reviewer isn't giving a game one star and telling you
"Lol, do not feel bad. 1 star is what I call average so its not a bad score." By giving it one star, he has now confirmed he is telling you it is inferior crap and this is what he means to say.
This will definitely at least somewhat help solve at least part of the problem. We can all agree on that, right?
reviews have accompanying thumbs up or thumbs down instead of grades or stars or scores
on game profile: 'this game has 71 thumbs up and 2 thumbs down'
or
'this game has 4 thumbs up and 564 thumbs down'
numerical or grade scores are stupid. when you are telling your friend about a game you say 'ya it was good you should play it' or 'not so good dont bother'. all you need to know is which direction the thumb is pointing. being told that the game is a C+ is not useful to anyone in any way because the ONLY decision to be made is whether or not you are going to fxckin play it or not. (!!)
the variability of reviewer interpretations of scores makes this impossible; if your personal mandate is to only play games that are at least 'average/5 out of 10/2.5 stars', what that means to you is going to be different than what it means to the reviewer, so the entire exercise is futile and should be abolished.
this way nobody has to worry that their AVERAGE STAR COUNT sitting on their main profile+search listing will be brought down by some d-bag trying to seem cool by giving an inordinately low score. likewise a game's STAR POWER will not be undeservingly bolstered by some friend of the creator or other forms of sycophant
the number of thumbs-ups a game has could also potentially affect its visibility but that is another matter
sorry just delivering my golden logic on wings of diamond they call me jean claude van 'games'
ahaah you get richer with every passing year
on game profile: 'this game has 71 thumbs up and 2 thumbs down'
or
'this game has 4 thumbs up and 564 thumbs down'
numerical or grade scores are stupid. when you are telling your friend about a game you say 'ya it was good you should play it' or 'not so good dont bother'. all you need to know is which direction the thumb is pointing. being told that the game is a C+ is not useful to anyone in any way because the ONLY decision to be made is whether or not you are going to fxckin play it or not. (!!)
the variability of reviewer interpretations of scores makes this impossible; if your personal mandate is to only play games that are at least 'average/5 out of 10/2.5 stars', what that means to you is going to be different than what it means to the reviewer, so the entire exercise is futile and should be abolished.
this way nobody has to worry that their AVERAGE STAR COUNT sitting on their main profile+search listing will be brought down by some d-bag trying to seem cool by giving an inordinately low score. likewise a game's STAR POWER will not be undeservingly bolstered by some friend of the creator or other forms of sycophant
the number of thumbs-ups a game has could also potentially affect its visibility but that is another matter
sorry just delivering my golden logic on wings of diamond they call me jean claude van 'games'
author=Max McGee
Of course, since this is the internet, the most popular and socially acceptable response is to say BFD, NicoB needs to get the fuck over it, and stop being a whiny, butthurt pansy.
ahaah you get richer with every passing year
Kef
0
Ciel is right. A game with 500 thumbs up and 2 thumbs down will be, at worst, at least playable and mediocre. A game with 2 up and 500 down will probably be a trainwreck. it does not need to ever be more complicated than that. the wordy reviews about how unbalanced armor class is only matter to three people; 99.9% of the people reading the games page just want to know if they should play the freakain game.
Pokemaniac
3559
I obviously agree with Ciel(seeing as I suggested the same thing one page earlier...). Reviews shouldn't go away, and I don't think numeric scores should either; but thumbs up/thumbs down(I think we need both; saying "X number of people like this" is fine and dandy but if you don't have anything to compare that with it's a useless statistic) should be included. My original suggestion was having a thumbs up or thumbs down attached to each review, but a more general version that let any member rate the game without writing a full review has its merits.
Neophyte
155
author=Ciel
on game profile: 'this game has 71 thumbs up and 2 thumbs down'
or
'this game has 4 thumbs up and 564 thumbs down'
numerical or grade scores are stupid. when you are telling your friend about a game you say 'ya it was good you should play it' or 'not so good dont bother'. all you need to know is which direction the thumb is pointing. being told that the game is a C+ is not useful to anyone in any way because the ONLY decision to be made is whether or not you are going to fxckin play it or not. (!!)
Hey this person is right you should probably do this.
*tilts thumb 37.46 degrees to the left*
eplipswich
2728
author=Ciel
on game profile: 'this game has 71 thumbs up and 2 thumbs down'
or
'this game has 4 thumbs up and 564 thumbs down'
numerical or grade scores are stupid. when you are telling your friend about a game you say 'ya it was good you should play it' or 'not so good dont bother'. all you need to know is which direction the thumb is pointing. being told that the game is a C+ is not useful to anyone in any way because the ONLY decision to be made is whether or not you are going to fxckin play it or not. (!!)
Hmm, I have to so agree with this one. A lot fairer and better than the numerical ratings.
EDIT: Also, thumbs up/down are so much more straight forward than a review. People don't have to freaking crack their brains to write a review and give a score. They just simply indicate whether they like it or not. As simple as that. As such, the gauge and popularity of a game will be more accurately measured.
Sated
4005
saying "X number of people like this" is fine and dandy but if you don't have anything to compare that with it's a useless statistic)
Wrong. You have the values for other games to compare it with.
Pokemaniac
3559
No, think about it. Game A gets 1000 downloads. 10 people like it, 990 hate it. So it gets 10 likes. Meanwhile, Game B gets 20 downloads, and is legitimately good. It also gets 10 likes. Statistically, (assuming most people aren't going to work out percentages in their heads), the games seems equal, but one is obviously much better than the other.
deboo
85
author=Max McGeeoops it's painfully obvious re-reading it haha, sorryMax doesn't actually think that, he was making a point.This.
author=Cielwouldn't this be incredibly off-putting? i'm not hugely against a thumbs up thumbs down idea, taking into account the argument, but it seems so easy to abuse.
'this game has 4 thumbs up and 564 thumbs down'
if i was a first time game developer and put my game up and it wasn't exactly great and got like 500 thumbs down it would probably make me stop trying altogether. i'm not sure about a system where you'd bypass the encouragement needed to improve a game and just get straight to 'this is good/this is bad.' i suppose if people still bother writing reviews to go along with it then that would balance out the vague thumbs up thumbs down thing, but i'm not sure about a system where it's made easier to officially say you hate a game on the game page.
I'd like to point out one slight logical fallacy in Ciel's argument there.
It assumes we have 500 people to up-vote/down-vote every game.
You might see 48254392846928 Thumbs Up and 3 Thumbs Down on a game like, say, Hero's Realm, but for the vast majority of them? It's gonna be like:
2 Thumbs Up
1 Thumbs Down
Oh wow I feel my decision being influenced already.
Then again, assuming we did have enough people using it for a thumbs up/down system to work, then it would be more like he suggested. And it's not like it would hurt to try. But I still wanted to point that out.
It assumes we have 500 people to up-vote/down-vote every game.
You might see 48254392846928 Thumbs Up and 3 Thumbs Down on a game like, say, Hero's Realm, but for the vast majority of them? It's gonna be like:
2 Thumbs Up
1 Thumbs Down
Oh wow I feel my decision being influenced already.
Then again, assuming we did have enough people using it for a thumbs up/down system to work, then it would be more like he suggested. And it's not like it would hurt to try. But I still wanted to point that out.
Pokemaniac
3559
author=halibabicaIt's still better than "Zero Reviews". :|
I'd like to point out one slight logical fallacy in Ciel's argument there.
It assumes we have 500 people to up-vote/down-vote every game.
You might see 48254392846928 Thumbs Up and 3 Thumbs Down on a game like, say, Hero's Realm, but for the vast majority of them? It's gonna be like:
2 Thumbs Up
1 Thumbs Down
Oh wow I feel my decision being influenced already.
Then again, assuming we did have enough people using it for a thumbs up/down system to work, then it would be more like he suggested. And it's not like it would hurt to try. But I still wanted to point that out.
Sated
4005
author=Pokemaniac
No, think about it. Game A gets 1000 downloads. 10 people like it, 990 hate it. So it gets 10 likes. Meanwhile, Game B gets 20 downloads, and is legitimately good. It also gets 10 likes. Statistically, (assuming most people aren't going to work out percentages in their heads), the games seems equal, but one is obviously much better than the other.
We can see download numbers and make a "like":download comparison ourselves. I know that people are stupid, but c'mon. Besides, the number of "likes" will be backed up by reviews. If there are a load of "likes" and the reviews are terrible, something is obviously up.
Besides, it is far too easy for people to troll developers that they don't like by "disliking" a game because they don't have to give a reason to hit that button. I don't want that to happen. And don't try and say that it won't, that's way too naive.
Pokemaniac
3559
But still, if you work out that there are 10x as many downloads as likes, that doesn't mean that everyone else disliked it, does it? Just like how you wouldn't be able to tell how good a game is if only positive reviews were accepted. Having dislikes is a much fairer way of judging these things than just having likes and working it out based on the downloads.
Also, some people would abuse it, but people abuse the review system as it is, and having a higher sample of opinions(which we would have since it would be easier to have an opinion) would easily balance out the one or two unfair votes a game may get.
Also, some people would abuse it, but people abuse the review system as it is, and having a higher sample of opinions(which we would have since it would be easier to have an opinion) would easily balance out the one or two unfair votes a game may get.
Sated
4005
It's much harder to abuse the review system. Much, much harder. Clicking on a button that says "dislike" is a really easy thing to do. Writing a review that is coherent out of spite is not something that is easy to do.
If people really liked the game, then you'd hope that they'd make the effort to hit the "like" button. Therefore, although you cannot tell how many people disliked the game, you can tell how many people liked it so much they were willing to put in the effort to give it more visibility. I reckon that the % of people willing to put in the effort to "like" a game remains constant across the community, so the % of likes to downloads should remain constant given how small the community is. Maybe not. I prefer this ambiguity to trolling, though.
Because of how spite and trolling work, the same cannot be said for people disliking the game so much they take the effort to hit the "dislike" button. People are much quicker to react to things they don't like than they are to things they do like. This is why there are so many vocal minorities who are against things and there are rarely vocal minorities who like things.
But still, if you work out that there are 10x as many downloads as likes, that doesn't mean that everyone else disliked it, does it?
If people really liked the game, then you'd hope that they'd make the effort to hit the "like" button. Therefore, although you cannot tell how many people disliked the game, you can tell how many people liked it so much they were willing to put in the effort to give it more visibility. I reckon that the % of people willing to put in the effort to "like" a game remains constant across the community, so the % of likes to downloads should remain constant given how small the community is. Maybe not. I prefer this ambiguity to trolling, though.
Because of how spite and trolling work, the same cannot be said for people disliking the game so much they take the effort to hit the "dislike" button. People are much quicker to react to things they don't like than they are to things they do like. This is why there are so many vocal minorities who are against things and there are rarely vocal minorities who like things.
Diaskeaus
44
author=halibabica
I'd like to point out one slight logical fallacy in Ciel's argument there.
It assumes we have 500 people to up-vote/down-vote every game.
You might see 48254392846928 Thumbs Up and 3 Thumbs Down on a game like, say, Hero's Realm, but for the vast majority of them? It's gonna be like:
2 Thumbs Up
1 Thumbs Down
Oh wow I feel my decision being influenced already.
Then again, assuming we did have enough people using it for a thumbs up/down system to work, then it would be more like he suggested. And it's not like it would hurt to try. But I still wanted to point that out.
This deals with the concept of incentive, and whether people browsing the site have incentive to actually post whether or not they liked the game. In other words, is it possible to keep track of these things on a user profile? So by checking so and so's profile, I can see which games they liked and which they didn't? Requiring a one-line critique obviously will help quell issues of spamming as well as issues of trustworthiness.
The other ideas mentioned all sound super fantastic:
-lists of favorite games (with auto-links,
-and when searching being able to find out which games were favorited the most),
-setting up a rubric for reviewers by using words and descriptions of what a number score means to indicate a more solid foundation during the writing of the review,
-helpful/unhelpful checks of reviews (especially in regards to scathing reviews, which does mar the chance the developer needs to attract much needed criticism),
-one-line reviews which award commentators with maker points (maybe 1?),
and really, props to Kentona and folk for really sprucing and improving this site over the years, while staying true to its roots as a site for people of all levels to participate in. I remember from years ago, and your constant effort in improving the site continually is inspiring. While there seem to be some sour grapes and elitists wandering the halls, thankfully this is still a wonderful and easy to navigate site that offers great gaming experiences without most of the tension that other RM-groups have devolved to.
More to the point about the issues presented here so far; even the above argument presented by Hali, while yes it's an issue for any site which has lower than desired numbers, but like Pokemaniac said, it's better than nothing, and over time will grow, especially if site users are given incentive to go back to older games and give them a try. The sad fact is that RM makers are a needle in a haystack (especially good ones) and in the indie world, pretty much given no cred for the vast amounts of work they do outside of the various communities.
I argued fruitlessly on Tigsource for months for more recognition to be given to RM users, but to little end. While they did add A Blurred Line and Sunset Over Imdahl, those two games only garnered aggregated reviews of 3.6 and 3.5 respectively (both only having been rated 14 and 13 times), whereas games on the top 10 list (such as Cave Story) have been rated 50-100 (on average).
I've probably gone off topic a bit, but a slight reform in particular areas would likely be helpful, but definitely not an overhaul.
author=boobledeebooauthor=Cielwouldn't this be incredibly off-putting? i'm not hugely against a thumbs up thumbs down idea, taking into account the argument, but it seems so easy to abuse.
'this game has 4 thumbs up and 564 thumbs down'
if i was a first time game developer and put my game up and it wasn't exactly great and got like 500 thumbs down it would probably make me stop trying altogether. i'm not sure about a system where you'd bypass the encouragement needed to improve a game and just get straight to 'this is good/this is bad.' i suppose if people still bother writing reviews to go along with it then that would balance out the vague thumbs up thumbs down thing, but i'm not sure about a system where it's made easier to officially say you hate a game on the game page.
if you are the kind of person who gives up due to negative feedback then i'm sure an abysmal star rating would have the same effect. maybe we should not tailor our societies to accommodate the most fragile possible egos. and yeah only people who wrote a review would be able to give a yes/no recommendation.
author=halibabica
I'd like to point out one slight logical fallacy in Ciel's argument there.
It assumes we have 500 people to up-vote/down-vote every game.
You might see 48254392846928 Thumbs Up and 3 Thumbs Down on a game like, say, Hero's Realm, but for the vast majority of them? It's gonna be like:
2 Thumbs Up
1 Thumbs Down
Oh wow I feel my decision being influenced already.
Then again, assuming we did have enough people using it for a thumbs up/down system to work, then it would be more like he suggested. And it's not like it would hurt to try. But I still wanted to point that out.
it doesn't need 500 people. idk if you are the type of person who needs other people to tell you whether or not to play a game you should read the written reviews. if you insist on at-a-glance recommendations then 2 thumbs up and 1 down is a lot easier to interpret than '2.5 average stars'. you know for certain that two people thought the game was good enough to recommend playing. that is more useful data than an average score based on nebulously defined rubrics. but let us not forget that we are talking about people who are both too lazy to read a review and completely incapable of making choices based on their own observations so meeting the needs of these individuals should not be anyone's primary concern.
deboo
85
author=Cielwell this changes things, you must've been wildly over exaggerating when you came up with the 564 thumbs down figure haha. i think it's a much better idea to allow thumbs up/thumbs down along with a well-written review instead of just having people drop in and rate the game without any real basis.
yeah only people who wrote a review would be able to give a yes/no recommendation.
just a couple of things, if it's a problem of the star score being averaged out (scores aren't difficult to interpret at all when you can see them individually) i can see where you're coming from, but that's assuming people reviewing have vastly different opinions of the game, making the average unreliable. however, in that case you would get as much differences between thumbs up and thumbs down as you would with the star score; making it just as hard to decide whether the game is good or not. at least with the star rating you can look through all the reviews and see where each stands in terms of how much they would recommend that particular game.
plus it's misleading to only use thumbs up/down when dealing with games that are mediocre. it seems like it would only be a good thing to be able to put forth an in-between score.
all the games would end up being at one end of the spectrum, eg. a very good game having as many thumbs up as an above average one. i guess in terms of 'yeah i would recommend both games despite this one being somewhat better,' that's a good thing but then it becomes harder to distinguish the differences of how good a game it really is unless you base it on the amount of reviews ( = more thumbs up, which still doesn't necessarily mean it's better).
S. F. LaValle
1069
The thing with thumbs up/thumbs down counters is that it will always err on the side of a favorable position. Meaning, someone can thumbs up a game if it meets one criteria they personally fell is important, such as being extraordinarily pretty even if the story is a bit boneheaded. This isn't a bad thing, though, because one redeeming quality about a game may be all that several people are looking for in a gaming experience. In the end, however, the mediocrity of a game ideally will balance itself out with thumbs up/thumbs down counter (5 up/5 down might be perceived as mediocre). Or, we could direct would-be thumbsmiths to point in the direction of whether or not they simply "had fun."
The sample size wouldn't be great, no, but it would be greater. As pointed out above, it's better than 0 reviews. People make comments on game pages all the time about a game after it's released. If we made it that easy, just putting a "thumb up/down" option in a comment box, then there ought to be a a great deal more thumbs than reviews. Review drama would cease; imagine Forever's End, based on the reviews now, let's say there was 14 thumbs up. He gave it its first and only thumbs down up to now. Hardly a tragic event.
As an added bonus, the concept of "thumbs up/thumbs down" inherently appeals to one's experiences playing the game, and not necessarily their critical viewpoints. It implies, "would you recommend this game to others?" Yes, people could still be disingenuous and give a game a thumbs down for any number of nefarious reasons. But compared to reviews, this is hardly a threatening proposition.
Even if the sample size is the same as reviews, the barbs are gone. "Thumbs down" means either "I can't recommend the game in its current form" or "this game is an irredeemable piece of crap." Inversely, you might worry that that "thumbs up" means "this is a masterpiece, best game EVER" or "a serviceable if not particularly engaging experience." I'm not that worried about it; those games that bring in a wide range of those comments are typically the most popular, and will make up for it in sheer number of upvotes. Plus, people will still be entitled to their short commentary.
@Pokemaniac, that is a valid point, but I would think that if this system goes into place, there would be metrics for displaying games based on number of likes (or dislikes if people want) as WELL as the percentage of likes over dislikes. If we only use one of these metrics, the percentage of likes would probably trump the number.
If it wasn't obvious, I would be in favor of such a system. It touches on the reason we all considered Racheal's idea good when it was first said. One person's average might be 3.5, another's might be 1.5; but if you need to give a game either a thumb up or a thumb down, average is playable, so thumbs up from all parties. The more granularity you add to the scale (adding below average, average, or above average), the more you are opening the door to outrageous differences in opinion and ensuing drama, thus complicating a very simple purpose.
The sample size wouldn't be great, no, but it would be greater. As pointed out above, it's better than 0 reviews. People make comments on game pages all the time about a game after it's released. If we made it that easy, just putting a "thumb up/down" option in a comment box, then there ought to be a a great deal more thumbs than reviews. Review drama would cease; imagine Forever's End, based on the reviews now, let's say there was 14 thumbs up. He gave it its first and only thumbs down up to now. Hardly a tragic event.
As an added bonus, the concept of "thumbs up/thumbs down" inherently appeals to one's experiences playing the game, and not necessarily their critical viewpoints. It implies, "would you recommend this game to others?" Yes, people could still be disingenuous and give a game a thumbs down for any number of nefarious reasons. But compared to reviews, this is hardly a threatening proposition.
Even if the sample size is the same as reviews, the barbs are gone. "Thumbs down" means either "I can't recommend the game in its current form" or "this game is an irredeemable piece of crap." Inversely, you might worry that that "thumbs up" means "this is a masterpiece, best game EVER" or "a serviceable if not particularly engaging experience." I'm not that worried about it; those games that bring in a wide range of those comments are typically the most popular, and will make up for it in sheer number of upvotes. Plus, people will still be entitled to their short commentary.
@Pokemaniac, that is a valid point, but I would think that if this system goes into place, there would be metrics for displaying games based on number of likes (or dislikes if people want) as WELL as the percentage of likes over dislikes. If we only use one of these metrics, the percentage of likes would probably trump the number.
If it wasn't obvious, I would be in favor of such a system. It touches on the reason we all considered Racheal's idea good when it was first said. One person's average might be 3.5, another's might be 1.5; but if you need to give a game either a thumb up or a thumb down, average is playable, so thumbs up from all parties. The more granularity you add to the scale (adding below average, average, or above average), the more you are opening the door to outrageous differences in opinion and ensuing drama, thus complicating a very simple purpose.
