MAX MCGEE'S PROFILE

I guess that's kind of a valid point...it still pisses me off that anyone might reject games with such a broad brush, but you make a good point.

I agree with everything Adon said although I'd put the threshold of Mediocre at 2.5 Stars or less.

Actually looking through all of her reviews the other day I noticed that nhubi had given almost NO games a rating of 4+, possibly actually none (I don't recall at the moment). I remember thinking... "MAN, what is she LOOKING for? doesn't she love ANYTHING?"

oh well. /tangent

Oh, don't get me wrong. Scores, in general, bother me. Look at one of my own games and you have two people who thought it was stellar and amazing and one person who thought it was crap. The third person also thought that the critically acclaimed Grave Spirit and the wildly popular Legacies of Dondoran deserved 1.5 Star reviews, so clearly my vicious panning at least put me in august company.

The way that review averaging works means that because none of the scores are weighted, the game winds up with an aggregated score that is 'slightly above average'. I should be happy enough with this but...the truth is, two of the people who bothered to review the game thinking it was great and the other one person who bothered to review the game thinking it was crap doesn't mean the game is, in some objective sense, slightly above average. It doesn't mean anything except that two of the people who bothered to review the game thought it was great and the other one person who bothered to review the game thought it was crap. Basically, scores are nothing but a useful evaluative shorthand. They're not a "true measure of a game's worth" whatever that is. I don't think scores are problematic in and of themselves, I think it's the weight that other factors, like the site's structural interface/design and our own psychological design, give to these scores that creates a problem.

An interesting idea I just had is what if instead of using an average review score, when there was more than one starred review, the site staff picked one of the reviews to be the "featured review" for that game and used the star score of that one to display. This would obviously set one person's opinion above all others, and the staff would make a judgement call based on the quality of the review. I don't know, this obviously opens a whole can of worms on its own and I thought of about half a dozen problems with it just writing this paragraph, so I'm not actually advocating for it I'm just thinking out loud.

I don't think that getting rid of stars for everything entirely is feasible because honestly when you search the site for a game to play as a lot of our 'silent user majority' do, one of the parameters you want to be able to filter is 'overall goodness of game'. And let's say you arbitrarily set the minimum rating you were interested in to Four Stars (****), well even if you were looking for a science fiction RPG you'd automatically miss out on my game which one person thought was a 4.5 and another person thought was a 5 just because one other person thought it was a 1.5. So yeah the system as it exists is not awesome but abolishing stars entirely is not a practical solution because end-users wanting to search games by rating is a reasonable request. Hmm...

I think the idea of a game creator being able to opt out of displaying its score is fascinating...but I do worry about how the public would perceive games with 'hidden' scores, and this idea opens up its own whole host of issues.

But all that is a bigger and thornier issue than the question of whether or not to disable scores just for reviews of incomplete games specifically. I do have specific arguments I want to make here--namely an argument that unstarred reviews are "good enough" for end user evaluation of demos but not complete games--but it's going to have to wait until I am substantially less tired.

One question: how am I supposed to proceed if I want to find a good demo to play, then?

I did actually invest literally hundreds of words into answering this question in the original article. Short version: use thine eyes.