« | Home | »

Lame Discussion: To Pull a Review

posted on November 8th, 2006 by jay

It’s been a while since the last discussion. The format will be the same as always; I just drop you into our chat room. Today we are talking about the pull of 1up’s Neverwinter Nights 2 review. If you aren’t sure what I’m talking about, don’t worry, Horatio wasn’t, either. For those of you who would prefer to read opinions in a more traditional format, check out Craig’s editorial on this issue.

______________________________________________

Horatio: Before we start, I quickly read the article, but can someone do like a three sentence explanation of what happened?

Christian: Basically, Matt Peckham wrote a scathing review of NWN, 5/10. People thought it wasn’t a fair review, and 1up’s editorial staff pulled it. Later it was explained that their editorial process was a bit different for this one. It might not have been read by as many people as normal.

Horatio: Also, unfair in the sense that like, he just doesn’t like the genre?

Jay: It happened to not be well written so a lot of people argue that’s why it was pulled. But these are all points of contention. He did cite Icewind Dale and Torment as good games.

Christian: Anyway, the editor still didn’t like the review, told Matt to redo it a few times, and posted a final draft he still didn’t like much. And later decided he couldn’t in good faith keep it up.

Jay: The first question: Is pulling something because it’s poorly written acceptable? (ignoring the possibility it was pulled for other reasons)

This is all his fault.

Craig: Pulling something for the sole reason that it’s poorly written is perfectly acceptable, but the editors shouldn’t even have let it get that far. It’s an editor’s job to make sure the article is top notch before it goes live, not afterwards.

Horatio: In theory I agree with what you say, and I know, Jay said we should ignore other factors. On the other hand, if you let people pull articles because they’re “poorly written,” it seems much harder to trust a site that that’s the reason they’re actually pulling an article. As opposed to inadvertently pissing off their fan base or advertisers etc. etc…

Jay: I think it’s absurd to pull something for poor quality and makes it look worse than not pulling it. But then the beauty of the internet is that you can edit things subtly. Print mags can’t change stuff, but we can. If it’s nothing of material (which is important) and just grammar fixes, just go in during the night like a ninja and fix it.
Don’t retract it.

Craig: There’s another problem with pulling things in such a way – it’s opening the floodgates for the rest of the site. Now the community has seen they’re prayers answered so to speak, they’re going to request this every time they see something they don’t like. That review wasn’t that badly written, just mediocre.

Christian: On one hand, the explanation was quite detailed. I appreciate that a lot; its not something mysterious like a lot of pulled reviews. On the other hand, if it was that bad, it should have been caught. That’s just sloppy on the editor’s part. Plus, 1up is already on shaky ground with some people. Examples: accusations of their DOA4 guide being ripped from a fansite. Dan Hsu’s call out of game journalists without giving names. They seem to make huge blunders, make a fancy excuse, and wait for the fans to forgive them (after all, those editors are like rock stars to them). I just don’t trust them at this point.

Jay: Does anyone want to defend the pulling? Perhaps with the argument that the guy didn’t seem like the guy who should review that game?

Craig: It isn’t defending as such, but I think it’s in a games journalist’s forte to be versatile. If he likes the genre, then that can sometimes lead to bias reviews. Likewise, if he hates the genre, that can lead to bias. A reviewer should be able to just see each game individually.

Pat: I think you want some subjectivity to reviews. If a reviewer is a fan of a genre then he probably has more in common with his readers (who will be interested in the genre of the game he is reviewing) than someone completely out of his depth. If the reviewer makes his prejudices known (as this guy did, to some degree) all the better. I find a review most useful if it is biased, but i know what the reviewers biases are

Chris: If a reviewer has a bias toward a particular subgenre (as in this case), then that will fail to appeal to a lot of readers. This particular guy liked some RPGs but was apparently really jaded with the D&D system, which makes it seem like he would not be the best choice.

Pat: You are correct. In a lot of ways it’s a tradeoff of expertise for subjectivity. I don’t see a solution to that besides letting robots review games.

Craig: Well, once again it’s down to the editorial team to sniff out who is the best for each review, but I still stand by the fact that a professional reviewer should be able to review anything.

Horatio: I think what this comes down to is that it was a poorly written article. What they should have done is let him rewrite it in a more compelling way so that he could get his point across. In some ways I think he did the best to admit his bias– he offered a separate score for people who are fans of the genre.

Isn’t it sort of reinforcing the stereotype to have an Asian guy running EGM?

Jay: I think this comes down to what we expect from reviewers. If people want to know if a movie is good, they check a random movie review. If people want to know what film experts thought they check Ebert’s review. This may not be entirely correct, but my point is that some critics are well known and are supposed to express their opinions and not the opinion of an objective reviewer explaining the movie/game for people.

Craig: People tend to get up in arms when reviewers don’t conform to the norm of the rest of the scores.

Christian: I’m going to defend Peckham’s stance here. Regardless of how it was written, his view is solid; its all great from a numbers and stats standpoint but he didn’t enjoy the story, the gameplay, the content we look for in a game. And to him, judging it as “a toolset” is bullshit, because he was looking for a game. I think this fiasco brings up just how scary the “hivemind” is in gaming culture. People complained that the review didn’t match their expectations, that he should have realized it’s most important as a great toolset. And 1up fans claimed they were glad the site was catering to the people who wanted the game. That scares the hell out of me. I don’t want a site catering to anyone. I want to know what they really think.

Jay: Do you think it was a clear cut of catering to fans?

Chris: It certainly looks that way.

Pat: Probably. Although it was somewhat poorly written, it wasn’t pull-worthy. If this were real journalism they would have been crucified for retracting that. 1up will give that game a seven when all is said and done.

Craig: Absolutely, this is the editorial team bowing out under pressure. There have been three reviews similar to this at Rewiredmind.com in the last couple of weeks like these and they’re still there.

Horatio: Random interjection, but I think that’s one of the benefits of most gaming magazines is they always panel review games. So you can have different viewpoints/subjectivities, and choose a reviewer you know represents a similar stance on games to you.

Christian: I think the reasons people are decrying Peckham are as asinine as anything he might have said. But look who won. Also, if the new review is significantly cheerier, and gets a much higher score will you guys be more inclined to suspect foul play?

Chris: Maybe it all comes down to whether an editor will defend a writer or cave in to the site’s fans. Definitely.

Jay: I don’t understand how they can possibly justify doing anything but reposting the same review after cleaning it up a little writing wise.

Craig: There’s another factor at play here too that we’ve neglected – the all seeing PR people. Do you think that that has affected 1 UP’s decision to pull too?

Jay: Yes. Advertising. Atari must have gone nuts.

Horatio: So I think they should repost the review (but better written). But it might also be that they should have someone who is a fan of the genre review the game. Because there are definitely niche genres (think puzzle games) that should be avg. reviewed so non-fans of the genre know to buy them, but also a review so people who love the genre know what to do about the game.

Another guy’s head, just to go along with the theme.

Pat: That’s a good point but also that’s what Metacritic is for.

Horatio: Well I agree, but also a normal game website should recognize that there are many criteria to review a game and if one review can’t encompass them all because of inherent bias, then multiple reviews may be necessary.

Pat: Agreed.

Jay: I don’t necessarily agree on the multiple review thing. It takes a lot to have two reviews of everything.

Craig: That also removes the whole purpose of a review.

Jay: I think we need to expect more from readers. They need to understand that people have different opinions and learn to excise important information from reviews and weigh it in their own minds

Pat: That’s like expecting more from voters. Good luck.

Horatio: Well multiple reviews aren’t a requirement, I’d say. But if a site wants to give a good review to all its readers then it either needs omniscient reviewers who can take multiple stances, which is often possible or it needs multiple reviewers.

Craig: Or editors willing to back their opinions.

Jay: Ha. Before we finish does anyone have anything final to add?

Craig: Bottom line: this isn’t about what is expected of a reviewer; it’s about what people expect a review to be.

Pat: These places make their decisions when they hire a reviewer. They choose the person because they respect their ability to critique games. As such they should edit (as that’s their job) but stand behind the review.

Christian: Peckham is a freelancer. Just wanted to clarify that.

Pat: I’ll admit I did not know that, but the point’s the same. They paid him to review a game, they didn’t pay him to give the game a good review fans would be happy with.

Christian: Not to sound melodramatic, but I think this is a huge loss for indie sites, for true fans, for a lot of us. 1up has proven to me in the past that they don’t always care about the amateurs, the New games Journalists and they know that their fans see them as gods in the industry. They’ll cater to those fans, and at the same time, their clout as editors will sway the minds and opinions of a lot of readers. Read any list of 1up blog comments and look at how damn homogenous it is. It’s scary. And in turn, 1up can do whatever the fuck they’ll please, and the fans will gobble it up. That makes it hard to get a fair shake in a review, and for guys like us to get any sort of strong voice

Craig: A scary thought and a scary position of power.

Jay: OK, last call. Anyone? Anyone?

Craig: BOYCOTT 1UP!

5 Comments

  1. Matt said on November 8, 2006:

    Good roundtable, guys. All of you brought up some great points. But here’s what I think: I guess it all depends on what truly happened. Does the guy hate the genre? Did the editor bow down to the pressure? Did Atari call foul? We may never know what really happened (the editor in question seems to think the guy was too biased, but why publish it in the first place?). I just can’t believe that ONE reviewer hated this game for the right reasons. All others on the web loved it, so what’s his deal? Granted the parts of the game he hated seemed to be presentation stuff, not the real essence of the game, but to grade it that poorly? Being biased seems to be the most logical explanation.  I’m glad the review got pulled, because if it was still up there, the site would be looked upon very poorly. Like, not being a video game site anymore type of poorly. People would probably think 1up is made of complete idiots and never go there. Granted most of their reviews do suck (Luke Smith is actually pretty good, though), but not as bad as this one turned out, politically. Also, reviewers should be as versatile as they can be, but it’s hard to enjoy every genre. The reviewer should have stated his opinions, but not reduce the score on those alone. I like when people say, "if you like this, then go for it." That shows he has an opinion, but knows people may not agree (IGN is pretty good with this). Basing it on the editor’s eye witness accounts, the review never should have gone live. He said it seemed to be a more "I hate this genre to begin with" story than anything. And think about that: would you want a reviewer to rate Zelda when he clearly hates 3D adventure/RPG games? Everything that you like about Zelda will probably be all the reasons why the guy rated it so poorly, so the editor should have just chose another writer. That’s the crux of the problem. The editor should have known better. But still, I don’t think this is as bad as some people made it out to be. There really aren’t any set rules in the video game journalism space, so we have to make mistakes before we can grow. It’s just growing pains, that’s all.

  2. jay said on November 8, 2006:

    I honestly think part of the problem is that a lot of people review a game based on what they think it should get scorewise, not what they think it should get scorewise. Meaning if they can tell a game seems like people would enjoy it and seems very well produced, then they will score it high whether or not they personally have fun with it. I gave Disgaea a bad review despite enjoying a lot of the time I spent with it. I guess the difference is my review was focused on the industry and not the player. Plenty of gamers love the title, but I gave it a bad review as a way of saying, "Stop making these overly bloated SRPGs that are nearly identical to one another."

  3. Christian said on November 8, 2006:

    This is a good post Matt, one that looks at things differently than we did in the roundtable.  Personally, I just can’t take this standpoint because of my opinion of 1up.  I’ve just seen too many mistakes or brazen comments from their editors, and I just can’t shake the feeling that they’re doing their best to try and glorify their position and their ethics so that they basically become bulletproof when any sort of criticism comes their way.  Yes, they often apologize or say this and that, but actions speak louder than words, and so far they haven’t redeemed themselves in my eyes. 
    Plus, as was mentioned in the roundtable, a lot of us are unsure about what the new review is going to be.  If instead the review was just as poorly written, lacking just as much when it comes to justifying the writer’s position, but praised the game and gave it a 9.5, I don’t think people would be asking for a pulled review.  The actions taken here seem to have stemmed from the score given, which is a problem if they’re not going to be universal about things.
     
    Like you said, a lot of it may be growing pains.  I  just hope they learn from their mistakes.

  4. Matt said on November 9, 2006:

    Yeah, it’s definitely going to be interesting to see the NEW AND IMPROVED review of NWN 2. I know the editor is going to look through that one with a fine-toothed comb. People will say it’s just appealing to the masses, though, even if it is an actually well-devised review. And that’s why the Internet is filled with complete idiots. You’re damned if you do, and damned if you don’t. 1UP has lost some credibility here, but I actually never went there for their reviews anyway. They’re usually too short and uninformative. And if we replaced the original score, but kept the writing part of it, it probably wouldn’t have been pulled, but that’s why I pointed to the "I didn’t like it, but some would" mentality. It wouldn’t have been that bad if the score was different because we could see that the game is actually deserving of that score but the reviewer had a bad time with it personally (I would like to get a look at the review to see if he completely shitted on it or not, because this may change my feelings on the issue). But this immediately points out that the reviewer was the worst possible person to review this game. The editor should get a spank on the toushy for this one.

  5. Matt said on November 9, 2006:

    Here’s their retraction article: http://www.1up.com/do/newsStory?cId=3154916

Leave a Reply