Wikinews:Water cooler/policy/archives/2010/February

How to peer-review

I'm not aware of substantive guidance on the site about on how to go about peer-reviewing articles. What I mean is that (unless I've missed something, but if I have then there's a decent chance I'm not alone in that) a newly promoted editor has nothing that tells them how much of what kind of work they ought to be putting into the task. Much of their prior experience of the peer review process is that it looks easy; they only see the tip of the iceberg — maybe some copyedits, and a little edit that replaces {{review}} with {{publish}} and puts a template on the talk page saying it's been certified acceptable on several criteria. One might expect to find the primary description of the process in Wikinews:Reviewing articles, but that page doesn't give a real sense of the part of the iceberg below the surface.

As a first step toward figuring what we should tell prospective peer-reviewers about the process — and authors, too, who might find the process easier to appreciate if they had a clearer picture of it — I'd like to ask how other editors go about peer-reviewing an article. I too have only seen the effects of other people's reviews; and I'm not really satisfied that I know how to do some aspects of reviewing properly.

As a starting point, I'll open myself up to ridicule by describing how I go about peer review.

After reading through the article to get a sense of what is there, I open up a second browser tab, and use it to set up a scratchpad copy of the entire body of the article in Special:ExpandTemplates, which lets me edit the scratchpad without any danger of accidentally saving it. Then I scan through each of the sources from beginning to end, and as I find things in the sources, I use <s>...</s> to strike out those facts in my scratchpad copy of the article body; when I'm all done, everything in the scratchpad copy should have been struck out. This also serves several other purposes:

• Scrutinizing the relationship between article and sources should also detect copyright problems of the "block copy" sort (if such were to occur).
• Scrutinizing the article piece-by-piece should detect a large percentage of style problems (assuming that the reviewer is likely to recognize style problems when their nose is rubbed in them, this will do the nose-rubbing).
• Scrutinizing the sources is likely to detect if they're really from the same newsfeed rather than being independent sources.

When I've done all that, which concentrates on the details, I try to step back and look at the whole, a bit (I admit I'm still struggling with this). Like, are all the really important aspects of the story included in the article, is appropriate weight being given to the different aspects, what about categories, picture(s), infobox.

I'd also be very interested in comments on which aspects of an article are most important to get right before publication, versus which aspects aren't such a big deal to make minor fixes to later. --Pi zero (talk) 14:42, 2 February 2010 (UTC)

This is something I haven't really thought about, but now that it's been brought to my attention, yes, I do notice that we haven't much in the way of help in how to publish articles for new editors. Perhaps we could create Wikinews:Tips on reviewing articles or something like that? You're quite right in that most reviewers are left to their own devices when publishing. I don't think we should have a firm policy on *how* exactly to review, but rather just suggestions what to look for.

Anyway, I really like your strategy for verifying articles, i think it's quite efficient but, at the same time, covers all important steps. My method for reviewing is less methodical: I go through the sources, make sure they're not from the same agency, then try to judge their accuracy and reliability (i.e., I'd take info from the Daily Mail or tabloid with a grain of salt, and perhaps try to find a more reputable source to back up the information taken from them). Then I proceed to the article itself: I read the first paragraph, find where in the sources it's verified, make sure it conforms to policy (style, NPOV), then repeat the process with next paragraphs, making incremental copyedits as I see needed. When I'm all done with that, I do a quick read-through to make sure appropriate balance is given to all viewpoints, see if categories and formatting is correct, and then publish. Tempodivalse [talk] 14:59, 2 February 2010 (UTC)

I don't think we want to specify how to do a review. For instance, I read the article, then read each of the sources and can tell if it's been copyviod or some things aren't mentioned. I just copyedit as I go. So no methodical method, but I'm able to do it like that and everyone is different. We could maybe add a bit more in tips, but we want to avoid specific methods.    Tris   15:48, 2 February 2010 (UTC)

---

My approach seems to differ significantly from those given above. And, I'm the one that spent a couple of years or more fighting to get the process in place.

First and foremost, I copyedit. Who, What, Where, When, Why, and How. Does the article tell a story? Does it do so in a way that flows? Is there actually any point to someone reading what, mostly, are our synthesis versions of main news events reported anywhere else?

Once that's done, and often as a part of that process, I'll correct Wikilinks to be local - eg, United States instead of w:United States. To a lesser extent I'll look for wikilinks that go to redirect pages on enwp; however, I would generally consider this sufficiently low-priority that the only ones to really worry about are where someone's linking to a disambig page on enwp.

You will note, I have zero mention of reading any of the sources yet!

Once I'm reasonably happy the article reads well, I will look at the visual appearance. Are pull-quotes really pull-quotes? Or, omitted from the article? Is there no infobox, and an appropriate one that could be used? Are image captions holding part of the story rather than admitting to use of a file photo? Are images credited appropriately? (On this point, many images on Commons need no attribution but I feel that the people who give these away with no expectation of recognition deserve such. I think it is setting a good example for others utilising Commons and there's less worry about if it is required to attribute reuse. Just do it anyway and be done with it.)

Then, and only then, will I move onto the sources, related news, and external links. Are related news items really related? (Eg, don't just slap the last 3 articles on a country on the article, are they really a continuation of the same story, or a source used in the article you're currently reviewing? If not, cut 'em; this can be a POV-by-association issue if you pick the wrong related news items.

Then, into the sources; sort newest->oldest, fix the perennial failure to cite the BBC as "BBC News Online" (my pet hate), check authors are credited, and agencies where appropriate, try (based on five years on-project) to have publisher names likely to wikilink to the correct enWP article, and curse people who do not use the standardised Monthname Dayno, Year dating standard.

Ensure that there's nothing wishy-washy, generic, undated in the sources; if needed move to external links and cite appropriately for there.

Then, it's time to move on to reading the sources. And, believe you me, I've had some entertaining experiences there. If it's a "publication" from Bumfuck Blogistan I'm not amused. I really dislike dealing with such sources unless I can quickly establish they have some credibility and aren't just cut-and-pasted from a more obscure or behind-paywall source. Other risks you have are stories that originate with one of the wire services and go global in the media echo-chamber. It is really, really easy for someone who isn't appropriately suspicious to pick three sources with vast geographical separation, and even poles-apart politically, to all be regurgitating the self-same Associated Press or Reuters report. I've seen it, and probably most who follow the news a lot have too, a global story, in the first 15 minutes of every bulletin on TV news in a dozen countries, and the thing is actually single-source to AP having a syndication deal with a tiny obscure newspaper in Nowheresville. What can be worse is where all the used sources are different tentacles of the same news conglomerate like News International. There's a right-of-centre to right-of-Attila-The-Hun bias to all their outlets; that can badly bias a story.

You'll note I've not yet mentioned copyvio. I do, sometimes, find this amusing. I've done my copyedit first, sources later, on articles only to find that I've changed the Wikinews story to a closer copyvio than the original author. This, to me, is a little ironic; the author's efforts to avoid accusations of copyright violation are superficial, and they're paraphrasing sentence-for-sentence from a source without realising a critical copyedit will turn it back into what the source they plagiarised wrote. While Wikinews remains a small community you quickly pick up on suspected copyvio authors; others, you learn, can be trusted to put in the effort to render checks for this unneeded.

Naturally, alongside the copyvio item above, you're fact-checking the article. You may find you need to go back and slightly revise the Wikinews article because some point has been over-simplified, or somewhat misrepresented.

Categories, to me, are less important than a lot of things. I believe you should never rely on a category being added via an infobox. An article could be expanded, additional images or pull-quotes added, within the 12-18 hours post-publish and render the infobox inappropriate. As far as I'm concerned the critical category items are at least one geographical category, and one major topic category such as Category:Politics and conflicts. When we do have the sitemap extension sorted, tested, and installed, then the category information will feed straight into Google News. If you don't fix the categorisation on an article before publication it may be unlocatable via GNews.

Virtually all that's needed is in the Wikinews policies - which seem assumed more aimed at authors than reviewers; the critical points I would emphasise are, active voice and near-memorisation of Strunk's. The latter will tell you to rip out every single nonessential word. Almost all copyedits I do need words like "that" dropped because they're just noise. Certain points get slashed back to brutally simple subject-object-verb sentences. Run-on sentences get condemned as "crimes against the English language"; and – both as a writer, or reviewer – the aim is to have a finished piece that someone with college-level English will, if interested by the lede, read from start to finish without feeling taxed, and, from which they walk away feeling more informed.

As you might suspect, I set standards so high even I have trouble reaching them in my writing. To-date Mike Halterman is, seemingly, the only Wikinewsie who appears to have gone from contributor here to paid journalist. I subscribe to the view, "Take the job seriously, but never yourself". And, for aspiring writers, I would commend Stephen King's advice in his book "On Writing" – spend at least one hour each every day reading and writing. Even if you do not intend to pursue a career in journalism or writing books, this is a skill that will serve you incredibly well in any career. Use Wikinews as a place to hone your writing skills and critical analysis of the writings of others; along the way you can help make this a high-quality news source that builds its reputation and proves that writing is a craft, not a "taught" skill. --Brian McNeil / ^talk 18:56, 2 February 2010 (UTC)

Btw, in regards to Pi Zero's method. There's a trick you can do with js. Go to the page your about to review, and go to the url javascript:document.getElementById('bodyContent').contentEditable=true;void%200 And then you can delete arbitrary text from the page. Bawolff ☺☻ 18:02, 16 February 2010 (UTC)

Is this browser-specific? (I'm wondering what it does nothing when I try it. Turning on javascript, I thought of. :-) --Pi zero (talk) 16:58, 17 February 2010 (UTC)

Shouldn't be. Part of html5 (and it originally came from IE, so i think its supported more or less everywhere). Tested on firefox (You'll need firefox 3 or later. doesn't work with version 2), opera and IE. (won't work on konquor, as it doesn't support javascript urls). Should just go to some page (viewing some page, not editing), type that line into url bar, hit enter, highlight some text on the page contents, hit backspace and it should dissapear. Bawolff ☺☻ 19:29, 17 February 2010 (UTC)

Eureka! Thanks. --Pi zero (talk) 20:32, 17 February 2010 (UTC)

I was playing with this idea a little more (aka i was looking for an excuse to do something with contentEditable that seems to be all the rage these days). I created a gadet called On Screen Edit, that adds a link to the toolbar at the bottom of the sidebar to switch the page into "on screen edit mode". Maybe it might be useful. At the very least its somewhat fun to play with. Bawolff ☺☻ 21:25, 17 February 2010 (UTC)

Yup, neat gadget. Compared for the purpose with Special:ExpandTemplates, it has the advantage of a much simpler user interface — no wiki/html markup and submission via an "OK" button to deal with, just directly modify what you see. The balancing limitation, as best I can figure, is that along with the complexity of wiki/html markup, it's also lost the capabilities of wiki/html markup. In my own review technique, I strike out passages using <s>...</s>, so the struck passages are still visible, as is the whole structure of the article. (I don't want the struck passages out-of-sight-out-of-mind, because I'm still thinking about the whole article throughout the process — doing multiple subtasks in parallel.) It depends, evidently, on just what the individual reviewer is looking for. Ah, trade-offs. --Pi zero (talk) 14:11, 18 February 2010 (UTC)

I tried adding a toolbar. You can now hilight stuff, and press a button and it will strike stuff (or do other things depending on which button you press). Bawolff ☺☻ 02:28, 19 February 2010 (UTC)

That looks like it ought to have it all over Special:ExpandTemplates for the purpose. Cool. --Pi zero (talk) 15:27, 19 February 2010 (UTC)