Dr. Black Jack
Member
So fuck Metacritic. Why do the reviewers care? Less payola from publishers?
-
Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.
Numbered Reviews Must End
- Thread starter TheCloser
- Start date
So fuck Metacritic. Why do the reviewers care? Less payola from publishers?
Haha. Though I agree, I don't think console warriors are the only problem. It could be anyone. Imagine if half life 3 gets an 8.5 (I'm not sure which is more unlikely, the game or the score). Don't you think there will be a unanimous uproar against reviewers? The issue here has to do with misguided passion, be it platform, genre, developer or game.
Also, game publishers are over reliant on these scores as well. People get fired, projects suspended if a game isn't the critics' darling. That's just ridiculous. I can see the metacritic score influencing sales a bit, but that can't and shouldn't be given so much significance.
davepoobond
you can't put a price on sparks
Why not both
Actually, a single number score isn't a bad idea for a review site.
No reading or whatever the fuck
boredandlazy
Member
I'm glad I didn't read any of the Infamous: SS reviews, sounds like most of them spoiled stuff.
Aaron-Lagann
Banned
I don't have as big a problem with the numbers as I do the inconsistencies with the numbers. An 8 is only an 8 if the average review score is 5-6! Otherwise the 8 is the new average and therefore meaningless!
Tom Nook Sawyer
Member
We have that today: it's called reading more than one review.
NervousXtian
Thought Emoji Movie was good. Take that as you will.
Let's Plays spoil games.. also unless you watch the whole thing how do you make an opinion of if the whole things holds up.
I'm glad it's an option for those who want it.. but to say the standard review system is broken because YouTube & Twitch exist is absurd.
So much ether in this here thread. Love it!
IncandescentPrecept
Banned
The problem then arises that 3/5 which translates to 6/10 is seen as a bad even though on a 5 star scale, it's a good score.
So, 5 star scale is even worse in that sense.
Metacritic does that on a wider scale. . .
I like the movie review system of generally 1 to 4 stars and the whole scale gets used. I especially hate breakdown and "averaged of score criteria" style reviews, I could care less about the graphics, if the game is fun (and I don't care how pretty a lousy playing game is)
Has anyone seen a movie review with breakdown numbered scores for acting, directing, story, and special effects? While characteristics will lend themselves to final verdict, in the end it's about how everything comes together, and you dock a bit if there is something really bad that stands out to lessen the experience, and bump a bit if something sticks out as exceptional in an otherwise mediocre experience.
Has anyone seen a movie review with breakdown numbered scores for acting, directing, story, and special effects? While characteristics will lend themselves to final verdict, in the end it's about how everything comes together, and you dock a bit if there is something really bad that stands out to lessen the experience, and bump a bit if something sticks out as exceptional in an otherwise mediocre experience.
Not in the way where they would play off each other in the reviews. The last person could disagree with the first, make comments on something the 2nd one said.
killer rin
Member
This really is the only way to rate a game. We need to take a Technology Review standpoint where everything is broken down into parts, like how a review for a Cellphone breaks it down into Design, Camera, Display, Call Quality, ect. A score for games should be broken down into Scores for graphics, sound, gameplay, ect. In addition, numbers should never be combined into an overall score. Keep everything partitioned.
Until everyone does this, Numbers are completely subjective.
Most of those reviews were very short, and any interplay came at the expense of criticism and analysis.
I used to write for EGM when it was still published by Ziff. I want to say the small reviews were about 80 words long. It's hard to do much of anything with that limited word count.
Tom Nook Sawyer
Member
And those individual numbers would still be just as subjective. This doesn't resolve anything.
Also, I hate the idea of games be reviewed as products instead of as an artistic whole. Films aren't broken down into separate Script, Acting, Special Effects scores, and I don't think games should either. It's about how it all comes together; rating the individual pieces outside of that context provides no additional value.
That is a fair point, but this raises other issues:
1) What do you do if all of them have opposite opinions, which is quite common in this medium? The average score ends up being even more meaningless than the individual scores.
2) This implies that more than one person needs to be paid to review the game, thereby increasing costs. The more it costs the website to do this, the more they transfer that cost to the reader. Enter subscription charges and/or more advertisements (as if we don't have enough already).
3) Committing more resources to review the game and play off each other implies a significant amount of time spent on the review process itself, which makes reviewing games before release difficult. With increasing consumer expectations for early reviews, this just adds pressure on everybody and could affect the quality of the reviews.
Basically, there are down sides to any method. People just need to make the best of whichever they choose.
This is ironic and frustrating. You're implying a causal relationship. Eg - gamers buy games because of a high metacritic score. What if good games tend to sell more, and good games tend to also have above average review scores?
This isn't just about semantics. These are two far different possibilities and it cuts to the core of the supposed problem, which I do agree is a problem, but for different reasons (more on that in a minute). If metacritic scores simply correlate with the quality of a game as do sales, then getting rid of metacritic scores would do absolutely nothing except remove a convenient resource for players. Now on why I actually agree it's a problem, but for different reasons. Publishers have also assumed a causal relationship and they're starting to develop games based explicitly around achieving metacritic scores. The games media and game players don't exactly see eye to eye. When publishers start going overboard and doing things like scrapping concepts based on mock reviews and the like then they're no longer making games for players, but games for the games media. This is something that may potentially hurt the correlation between "good games" (from a players perspective) and above average review scores. And that is a problem.
If publishers believe that there is a causal relationship between reviews and sales then getting rid of scores wouldn't actually do that much. They'd still engage in the same sort of behavior aimed and targeting the games media with their games. And for that matter the games media rarely works as an actual media outlet and instead are more of a PR branch. Here is IGN's lead paragraph for their Titanfall review:
That's splattered right below some giant audio/visual hype ad for the game that auto plays when you visit the review page. You don't need to see some number to know it's going to get an absurdly high score from them. Imagine if Ebert had wrote a review that began: "With the action packed sequences from the first moments my heart began to palpitate - I literally could not sit still in my chair. The excitement was palpable - the experience unprecedented.. blah blah blah" But he wouldn't because that's retardedly unprofessional. That, like the games media already long since has, goes well beyond the realm of objective critique and into the realm of fanboy PR. The problem isn't about the numbers, it's about assuming a casual relationship which is driving publisher influence on the media - both coercively and simply in terms of what they release. Ironically, you assumed the exact thing in your critique of the problem!
The scale of 1-10 is too large. Anything below an 8 no one wants to buy. When something get's 7.9'd it's the end of the world for some.
I prefer the star system myself, though I think 5 stars is too much.
1 star = bad, don't rent, don't buy
2 stars = average (good things, bad things, worth a rental or bargin bin purchase)
3 stars = good (primarily an excellent game with very few flaws, worth a full price purchase if the type of game is your style)
4 stars = exceptional, does nearly everything right and you can only nit pick on most things. This would be the equivalent of the coveted 10.
I prefer the star system myself, though I think 5 stars is too much.
1 star = bad, don't rent, don't buy
2 stars = average (good things, bad things, worth a rental or bargin bin purchase)
3 stars = good (primarily an excellent game with very few flaws, worth a full price purchase if the type of game is your style)
4 stars = exceptional, does nearly everything right and you can only nit pick on most things. This would be the equivalent of the coveted 10.
Mr. Waffle
Member
Why would you need to watch the entire game to decide? If you did then it's a good sign that the game isn't worth your money. How did you know if a game was good in a video arcade? You noticed the gathering, and you watched the game being played... if it looked like it was worth your quarter you stuck it on the screen seal. This will only make sense if you are old enough to remember arcades before NES.
Ravensmash
Member
Honestly, I think reviews are fine as is.
What needs to happen is for people to understand what they are, what an opinion is, and stop getting so invested in the number hit on at the end!
Scores exist as a way to quickly summarise a game review or organise it into a category (good, amazing, average, etc.), what happens is that people cling onto that number and then proceed to make a judgement regarding the entire game without paying attention to the actual review itself.
What needs to happen is for people to understand what they are, what an opinion is, and stop getting so invested in the number hit on at the end!
Scores exist as a way to quickly summarise a game review or organise it into a category (good, amazing, average, etc.), what happens is that people cling onto that number and then proceed to make a judgement regarding the entire game without paying attention to the actual review itself.
Strongly disagree. Think rotten tomatoes.
Gattaca is viewed as a classic of enormous import by many. It received a 64/100
Titanic is the GTA of the movie industry. Huge budget, huge advertising, TITS, LOVE STORY, ACTION, primarily positive reception: 74/100
Avatar is the highest grossing film of all time: 83/100
Unanimous agreement is typically the sign that either you have a one in a million treat on your hands or that something has gone terribly wrong in the system. In video games I think it's obviously the latter almost always. GTA4 received a 98. That score is just stupid. Disagreement is the heart of consensus! If at least one person isn't disagreeing then your sample sucks or there are shenanigans at play. In the current state of video games - it's both.
Mister Chippy
Member
I don't mind numbers, I just ignore them. I read the meat of the review then make my decisions based on that, I normally don't even look at the score. Still, I like formats without scores better, like how Totalbiscuit does it.
Doctoglethorpe
Member
I like simple Yes or No system. Buy or Don't. Thumbs up, thumbs down. It makes sense for reviews, not just some completely arbitrary nonsensical number that doesn't even hold up as a relative rating against other games due to the fact most sites have multiple people with ranging opinions doing the reviews rather then one unified perspective.
That said, I don't really care about reviewer opinions anyways, I only watch video reviews occasionally to treat them like glorified gameplay trailers, or in the cases of people like Totalbiscuit to get info on features and how broken they might be.
That said, I don't really care about reviewer opinions anyways, I only watch video reviews occasionally to treat them like glorified gameplay trailers, or in the cases of people like Totalbiscuit to get info on features and how broken they might be.
Tom Nook Sawyer
Member
It should be noted that MetaCritic has far from every review site on there--which is kind of a problem in and of itself. They selectively choose which sites to represent, which could also be a factor in the perceived consensus.
Rogue_Ledr
Banned
The problem is that a Buy It/Don't Buy it scale is too vague a gauge of quality.
Tom Nook Sawyer
Member
Indeed. Can you imagine if you bought every game they recommended that you play? That would be the majority of games...
Hmmm? I totally agree with everything you said. You are taking my point out of context. I was specifically referring to the suggestion that having multiple people collaborate and give a single average score isn't any better than just one person giving a score. If one person gives a 0 and the other gives a 100, then those scores are useful to process separately. The average of 50, is meaningless. That's all I was trying to say. All your examples are spot on. They highlight exactly why people shouldn't take scores too seriously.
Mr. Waffle
Member
How about week prior to launch paid demo's for all games.
$1 for an hour of play. Then you can decide if you like it, or not. You really don't need someone else to tell you whether you like something or not. For those that played the beta of Titanfall do you think any of the reviews meant anything to them... playing the beta made the grade or didn't.
$1 for an hour of play. Then you can decide if you like it, or not. You really don't need someone else to tell you whether you like something or not. For those that played the beta of Titanfall do you think any of the reviews meant anything to them... playing the beta made the grade or didn't.
NipplesAndToes23
Member
Firstly, I've never seen a reviewer give a game a 8.1 or 8.2. Those numbers only show up when averaging reviews from multiple reviewers. In that case it's fine if you want to know where the general consensus is on a game. In a single review a number isn't as important.
Doctoglethorpe
Member
Well but the point isn't to go strictly by the "yes/no" rating. I hope you don't just buy every 8.0 or better game just because that. The rating is just supposed to be the reviewers "sum" of the whole equation of pluses and minuses they discuss in the review, and only a guide post for you as you do your own math on that equation and come up with your own answer.
The benefit of a yes/no system is just that its much more easily explainable in a review. You can add up what you like and don't like and decide if it balances out for a yes or no. When you do some arbitrary number or stars or whatever theres no real structure to it. They are just pulling a "percentage goodness" out of their bums. I kind of like it, its good but not great, 8.3. What? A few places do it more strictly where perhaps like 5 categories take up 20 points each, add up to 100 for final score, but then your just making arbitrary point choices on individual assets and adding them together, same thing in the end. None of it makes sense, but they do it because its good for marketing, when people just see the score and not the review behind it.
I think if you actually read a review you should completely ignore its score. Those aren't really meant for you, they are meant for metacritic and box art.
Doctoglethorpe
Member
IGN and a lot of other sites do it all the time.
And in each case they're doing games a favor by assuming it's the top of the segment on a percentile scale, rather than the bottom (41% or 61%). It's fine.
The use of a 5pt scale by any review outlet expressly implies they don't want to get too granular with review scores, so I don't see the problem in representing their scores in a rough order of magnitude on a percentile scale; it abides by the spirit and intent of their actual rating scale.
Tom Nook Sawyer
Member
Yep, thus rendering it even more meaningless.
Only because it's sufficiently vague. I'm not an advocate at all of the 100-point scales, but take something like Giant Bomb or Joystiq's 5-point scale, which I find to be not only much more useful than a binary scale in how it more adequately summaries the reviewer's opinion, but is also just as easily justified via the review text.
Mr. Waffle
Member
I give this thread a 4.2/10
It's just not next gen enough for me.
It's just not next gen enough for me.
That's one reason Metacritic pulls from such a large base of reviewers. If ninety reviewers think a game is good and ten reviewers think it's horrible, the score will reflect the mostly positive rating. In practice, there are very few games where the distribution of scores is bimodal; in general, review scores will be clustered around a single peak with a few outliers.
All the movies mentioned were competent and scored well. The vast majority of moviegoers would agree that any one of those movies (scoring in the >60 range) is better than the movies that score in the <30 range.
There were over 100 Metacritic-rated games released in 2013 on the Xbox 360 alone. Would you be willing to spend $100 and 100 hours trying each out for yourself?
Kotaku does effectively use a explicit binary review scoring system (Yes/No) with the addition of simply ignoring games that it deems unworthy of reviewing. We can quibble over how broad the range of scoring should be, but to pretend that Kotaku doesn't use an "authoritative" or "definitive" voice because it uses a word to represent each of its two explicit scores is absurd.
almost all of the big anticipated PC releases from last year (rome 2, company of heroes 2, sim city, arma 3, x: rebirth) received a mix of luke warm and outright savage reviews. i don't remember a similar reaction in any of their review threads. in many cases (CoH 2 and sim city in particular), the audience were by far the loudest critics.
even the titanfall thread was markedly even handed considering the weight of that release.
Fixed2BeBroken
Member
What I don't understand is why the "number" effects you? Did you enjoy a game? what did you think? Based on what you gathered, if you care enough about the score it got, it's obvious you have enough interest in the product to warrant a purchase. So why should the number that might not reflect your opinion matter to you?
That's my whole thing. Its comical that people get worked up over a score and the OP in this sense is comical. The score is what it is. Society rates shit and people use this to help make a judgement. But if it's the only thing they are using to help make a judgement then that is their own ignorance.
Point is, what you think and feel about a product should be most important, not what others think about or how others quantify it.
Oh please, it wasn't the titanfall official thread, but it was all the other threads that eventually got locked.
That's my whole thing. Its comical that people get worked up over a score and the OP in this sense is comical. The score is what it is. Society rates shit and people use this to help make a judgement. But if it's the only thing they are using to help make a judgement then that is their own ignorance.
Point is, what you think and feel about a product should be most important, not what others think about or how others quantify it.
Oh please, it wasn't the titanfall official thread, but it was all the other threads that eventually got locked.
here's the titanfall review thread: http://www.neogaf.com/forum/showthread.php?t=782150
feel free to compare and contrast.
TheSpaceBetween
Member
Numbers have to go. It would be one thing if there were some standard to which points are added and taken away, but it's almost completely random. Certain games have points taken off for this or that while other games aren't penalized for having the same issues. This happens not only across an individual publications, but even individual reviewers. Until there is some consistency with how games are reviewed individually and collectively, numbers have absolutely got to go.
NervousXtian
Thought Emoji Movie was good. Take that as you will.
Yes, I am old enough to remember that. I didn't need to see all that much of most arcade games back in the day... if you watched them for a couple minutes you got everything you needed to know.
The nickle arcade was a multiple time a month journey for us as kids. A lot of games aren't like that anymore.
Why exactly is that IGN paragraph PR? Pretty much captured my exact feelings while playing the game on my PC.
So automatically if someone likes something, they are just part of the PR machine?
Isn't it really just easier to believe that reviewers just like or dislike things, because they like or dislike things.. instead of some grand conspiracy theory?
Fixed2BeBroken
Member
no thanks, aint nobody got time for that. and it's cool how you just ignore the whole thing I said, thus your response makes no sense. good job on that
i guess there wasn't enough bold.
Tom Nook Sawyer
Member
Uhh, I don't think it works like that for most sites. It's not like "long load times = -1 pt." The score is a representation of a reviewer's overall opinion, hence why they often have a word that goes along with them saying exactly what the number represents.
Yup. Agree with both your points. I don't see any issues with the way metacritic compiles these scores. My point was specific to individual sites making aggregates based on a couple of reviewers. The only issue I have had is the way those scores are perceived in the gaming industry. Like it's some kind of horse race and every point matters! It's totally bonkers.