Splash Damage: Metacritic is a strain on devs
Splash Damage, the developer of Quake Wars, claims that Metacritic is becoming a problem for developers.
The Quake Wars developer said that the entire process was flawed and that journalists should abandon percentile scoring systems and instead use 'out of ten' scores. Otherwise, Wedgwood reckons, journalists are under pressure to justify such precise scores, which can lead to overly negative reviews.
"We know that some websites score quite high and some quite low, but in general, all websites tend to score between 60 and 100. There's never a 37. It's as if that whole section doesn't exist, so zero starts at 60, so three stars, and goes up to five. It's just not really an accurate enough measure," said Wedgwood in an interview with GI.biz.
As an aside, the man obviously hasn't been reading bit-tech.net lately as we make sure we use the whole scoring range, with even a 5/10 score going up today. We're certainly not afraid to call bad products out. Ho-hum.
"Percentiles put too much pressure on a journalist to justify an exact score. It puts too much pressure on the developer to try and identify these criteria that lead to very specific point increases or decreases, which is not at all what the developer should be focusing on," said Wedgwood.
Wedgwood, who admitted that he'd never agreed to such a deal himself, said that Metacritic-related bonuses are common in the industry and aren't really a problem. What really gets him worried is when royalty rates start changing based on Metacritic scores - shouldn't game sales alone determine that?
What do you think of the bit-tech.net scoring system, or the scoring trends of other websites and magazines? Let us know your thoughts in the forums.
Previous Article
12 Comments
Discuss in the forums ReplyOn a usage note, shouldn't it be "affect" in the tagline?
The bit about percentile scores seems .. biased, maybe. I mean if a developer needs to score 7's or 8's to hit a royalty level and they get 69 or 79 I'm sure they'd be extremely annoyed as in the /10 system those would have probably rounded up, but it does let reviewers have a bit more granularity in review scores, especially when as he himself points out, most sites/magazines won't use anything less than a 6/10 or 60/100 anyway.
The blame shouldn't really be put on the reviewers though (for putting pressure on devs anyway, the 6/10 problem is mostly there own) as they are just doing there job, it's the publishers making the crazy deals and the devs accepting the contracts.
And with the bonuses related to scores instead of sales, GOOD. Movie tie ins don't deserve anymore money than they get, but sometimes critically acclaimed titles slide under the radar of most gamers.
Wasn't there a recent bout of metacritic scores being screwed about with by groups with specific agendas?
**cough** Fanboys!
It's stupid how there are web sites say they are out of ten, but because they use decimals all the time and rarely give out a whole number it's actually a 100 scale.
No, they have both, the score they display next to every games name is the critic score, they take scores from loads of sources and average it. Then there's the user score which is displayed next to the critic score when you open the page for the game.
Because we have to be either gaming-focused or do something like 12 game reviews a month. With just me and Andy doing reviews and with us not really reviewing all the Wii and DS crap that comes out, that's pretty hard work.
Egos, it seems, are still easily-bruised.