Wikinews:Tips on reviewing articles

This is an essay on using and contributing to Wikinews. The contents are strongly recommended practices, but may not be enforced official policy; however, they may supplement, comment upon, or offer guidance to interpreting official policy. Links to official policies can be found in the Wikinews introduction.

This is an essay, recommended rather than official policy; it may be useful to assist in the practical application of policy.

Reviewing an article for publication is a demanding task. This page gives you some tips to help you through it.

Only a small part of the review process is visible on the wiki — some edits that you make during your review to improve details of the article, nudging it closer to publishable quality; and of course the climactic edit where you submit your review, passing or not-readying the current draft of the article. The machinery on the wiki that coordinates that visible process, like the {{review}} template and the "Easy Peer Review" gadget, is explained at Wikinews:Reviewing articles.

This page is about the larger, behind-the-scenes part of the reviewing task. About a third of it is a classified list of things to remember to check.

How much to do

Peer review serves two vital purposes — one absolutely vital in the immediate event (right now!), and the other crucial for the future (which starts now): quality control, and teaching.

Note: A handy rule of thumb is, if it should ever happen that nothing else came up during review, always try to find some useful copyedit or other to make before submitting the review: looking for a useful copyedit can help keep you focused, and seeing the visible evidence of the copyedit can help keep authors focused. A passing peer review with no preceding edits by the reviewer should be a rarity, and a situation that submitting contributors should strive towards.

Quality control

Peer review makes sure no article gets published until and unless it is up to the standards of the project. That starts with legal concerns —the author and reviewer are both potential targets for lawsuits if plagiarism or libel is published— and continues through fundamentals like accuracy and neutrality with immediate potential to damage our reputation to an ever-widening range of stuff that still cumulatively could corrode the reputation of the project (even little things add up to significant corrosion, with time). That range of stuff is the subject of the checklist, below.

Of course reviewers aren't perfect, and little things may get through (alas), but big problems shouldn't (note: the legal stuff, copyvio and libel, are big problems). A misspelling might get through, but if there are a lot of them, you should catch that; and so on. By the time an article is published, it should be good enough that, even if no later improvements were made, it would be a credit —rather than an embarrassment, or even a black mark— to Wikinews.

Quality control isn't just a safeguard against inexperienced authors (or malicious ones) doing things they shouldn't; just as you the reviewer aren't perfect, experienced authors aren't perfect either, and in reviewing an experienced author's work, you are their safety net against goofs and blunders. Don't relax your reviewing standards just because you expect the author to know what they're doing; it probably will take you less time to review experienced authors' work, but that's just because there probably will be fewer problems for you to deal with.

Teaching

You aren't just accepting or rejecting the article, you are pointing out to the author(s) what should be done differently, what can be done better, and how to do those things. You are teaching, and the success of Wikinews going forward depends on everyone continuing always to learn to do better next time (including you, of course). Newcomers are starting along a steep learning curve, and even Wikinewsies who have been here for years and written hundreds of articles aren't past learning (and yes, you can be the vehicle for that even though you may have far less experience they they do — keeping in mind that you too may learn).

So writing informative comments when not-readying, and writing helpful suggestions when passing (on any given criterion, as well as overall), and not giving things a semi-apathetic pass when they aren't acceptable enough to be repeated — are all vital to help Wikiniewsies grow, which is where the future of the project comes from.

Changes too large to remain uninvolved

As reviewer, you are well placed to see changes that would make the article better, but are too substantial for you to make during review without disqualifying yourself from publishing.

If the article is already acceptable for publication, here are three options:

Occasionally, if you are in sufficiently real-time communication with the author (such as by IRC), you might choose to hold up publishing while you negotiate with the author to make improvements. Be careful not to become too involved simply by virtue of your suggestions, even though the author is the one who actually edits the article.
Publish, and in doing so, mention possible improvements in the review comments. This may or may not result in improvements to the current article, but either way it allows the author to consider your suggestions for the future.
After publication, edit the article, submitting improvements for another reviewer to review. Be careful not to let the publication blur in your mind with your edits: if the article isn't acceptable without your changes, it doesn't pass review in the first place. Also note, even if your edits would be acceptable to another reviewer, they may fall through if there isn't another reviewer currently available to review them.

If an article is not acceptable without changes that would disqualify you from publication, and there is another reviewer currently available, you may choose to give the article a not-ready review and then undertake the edits yourself, disqualifying yourself from further review. This doesn't work if you can't arrange for another reviewer to take over. Also, be sure to consider how the author is likely to feel about it.

What order to do it in

Avoiding review collisions

Usually, when you start to review an article you should put the {{under review}} template at the top of the article. In addition to warning reporters not to edit the article but instead to leave remarks on the talk page, it also lets other reviewers know that you're doing this one.

Sometimes, if you're "just" doing a "quick" not-ready, it's tempting to not bother with the {{under review}} tag. However, that can lead, surprisingly often, to minor accidents where two reviewers not-ready the article simultaneously. The current (venerable) review gadget doesn't prevent that from happening: when submitting a not-ready review, the gadget does not detect whether somebody else has edited the article since the revision you are looking at as you review. It does detect this for a passing review; but not for a not-ready.

Very rarely, Wikinewsies have collaborated to get an article about a breaking situation out the door extremely fast, with a reviewer starting on one part of the article while a reporter is still writing another part elsewhere on the same page. Naturally, we don't use {{under review}} in that case. Things can get quite hectic; there are no standard tools to help with such a situation, nor likely can there be since such situations are by definition exceptional. Expect to muddle through.

Scheduling review

Volunteer donations of labor to news production tend to come in large discrete lumps under time constraint. Somewhat true of news writing, this is even more true of review.

Effective review calls for intense, immersive concentration by a sapient mind — able to recognize what's important, how the parts of the story fit together, etc. The deeper the immersion the better, so it generally has to be done all in one block of time. Even if someone has the expertise and mindset for review, large blocks of time like that don't present themselves often (hence, we expect that a review-assistance device that significantly speeds up the process —without compromising the human element— could greatly increase an expert reviewer's ability to deliver reviews). Volunteer or not, a person can generally only deliver about four hours per day of intense concentration ([1]), which review has to share with the rest of your life; keep that in mind as you budget your time.

A review also has to be done on somebody else's timetable, whenever a reporter submits an article rather than when the reviewer happens to have time free and the mood takes them. Rates of submission can vary widely; in modern times we've had some long dry spells often followed by massive pile-ups on the review queue, far exceeding what the active reviewers could handle. The inevitable result has been some articles being lost because they went stale waiting for review. Experience has taught us that this happens sometimes —articles going stale waiting for review— even in times when we have large numbers of active reviewers; it's been suggested that however much review labor we provide, demand will still expand to exceed it. Sooner or later a reviewer will likely have to choose which submitted article to review first, knowing that some of the ones they don't choose will go stale as a result.

Historically, there was a tendency to review the oldest submitted article first; even now (as of this writing), Category:Review is often informally called "the review queue". This choice of review-order makes sense if one expects all the articles on the queue to be reviewed promptly, because the oldest one on the queue is presumably in most immediate danger of losing freshness while some other article is being reviewed. However, if there really isn't going to be time to review them all before some go stale, reviewing the oldest first has some unfortunate consequences. It maximizes how old our articles will be when published; and, because the articles that get reviewed are likely to be almost stale at review time, it minimizes the change that any not-ready review can be responded to before the article's clock runs out — which lowers the absolute publication rate since that not-ready review has consumed scarce review labor on an article that doesn't produce a publication. Other scheduling strategies you may wish to consider:

Review of original reporting may sometimes be deferred a bit (even if you consider yourself up to that — it's admittedly a demanding sort of review that novice reviewers likely shouldn't attempt), since OR may be a bit more staleness-resistant (depending on the nature of the OR). Don't overdo it, and beware of OR that has hidden weaknesses that will present more of a problem if not flagged out promptly by a reviewer.
Individual stories are sometimes especially desirable to review promptly because their content is especially valuable if published quickly, or because they're especially vulnerable to losing freshness quickly.
Articles likely to be easy to review might be done first. This can apply to articles that are obviously unacceptable, but sometimes those can be quite difficult to review because while the judgement of not-ready is obvious, writing helpful review comments may be quite laborious. It is an awkward irony of review that inexperienced reporters, whom we are particularly keen to support, generate articles that occupy a disproportionate share of available review. Skilled veteran Wikinewsies may be good at writing medium-to-short articles that are easy to review and pass (though of course it depends on the individual, at which point accumulated reputation comes into play). Triage may be especially important at moments when we're overwhelmed by a large number of poor submissions (which, yes, has sometimes happened, for one reason or another), since the few articles that would be viable if reviewed in time might starve for attention unless done out-of-order.
Articles that have been not-ready'd, and revised and resubmitted, may be desirable to prioritize so that if they have, in fact, been successfully fixed, the effort to do so doesn't go to waste.
If all articles are more-or-less equal in nature, and there are likely far more of them than can be reviewed in a timely fashion, it may make sense to review the most recent first. If the same number will end up published anyway, this strategy maximizes how fresh they are when published; and, if combined with prioritizing re-reviews, it may also give some not-ready articles time to be revised and resubmitted.

Within a given review

Our advice on the subtasks of a review: don't let anyone else tell you what order to do things in. (The order of the checklist below is a way of arranging the list, not a recommendation for arranging the review process.) Any order that works best for some reviewers is sure to fail miserably for some others. Try to find an arrangement that plays to your strengths.

Some reviewers don't even do the subtasks of review in a sequential order at all. Some do, perhaps breaking up the review into a large number of subtasks taken one-at-a-time. Others may do a large clump of subtasks in parallel, and then another large clump — for example, all the subtasks that only require looking at the article itself, then all the subtasks that involve comparing it to the sources (or vice versa: all the comparison subtasks and then all the article-only subtasks). Some reviewers, though, can and will simply read through the article and the sources and remember to do all the things that need doing, effectively doing all the subtasks in parallel.

There are a couple of devices that might be helpful in taking temporary notes during your review process (depending of course on what your review process is), without risk of accidentally saving changes to the wiki. You might, for example, set up a scratch copy of the article and highlight portions to indicate verified-by-source, possible-copyvio, or whatever else you want to note. The devices differ in how conveniently things can be marked up, how flexible the interface is, and how robust they are against losing the contents of the scratchpad.

The more robust device is the breadboard, into which you can write any wiki text at all —such as copy-and-pasting the contents of the article— and then edit or mark it up. It is moderately robust, in that while it won't survive closing the browser tab, if you navigate away and back (using the "back" button on your browser), your edits should still be there. You can highlight portions of your text using {{background color}}.
A lighter-weight device is the On Screen Edit gadget. It allows you to mark up a scratchpad copy of the article in various ways, directly editing the appearance of the scratchpad copy (in a WYSIWYG fashion) rather than working with the underlying wiki text. The scratchpad is fragile, in that if you ever navigate your browser away from the page for any reason, your edits are instantly permanently lost.

Checklist

This is a quick-reference list of some rules and guidelines that, as a reviewer, you might like to be reminded of. Besides reviewing an article for publication, there are a few items that only matter for changes to an article after publication; and the list might just be useful when writing an article, too.

Don't follow the list blindly. Some items are hard policy, others are guidelines, and some guidelines are softer than others. As a thinking reviewer, you can judge which are most important, when the spirit is more important than the letter, and when an article has a problem even though the problem isn't mentioned here (nor, for that matter, in any of the policies, guidelines, and help pages); hang on to that thinking advantage!

The size of the list is a compromise. As the list is meant to serve as a quick reference, it has been kept short. Too short, though, and some details under a general item might be overlooked despite the general reminder. (Compare the five-item checklist on the {{peer reviewed}} template.) Ideally, each item listed would be just specific enough to remind you of any further details omitted under it.

If there are other things you think should be added, by all means, suggest them!

The news story

Is it News? — News is

Specific.

Relevant (probably to more than a few hundred people).

Fresh (for synthesis: event within a week, new details have come to light within 2–3 days; OR may extend shelf life, sometimes by a lot).

After publication, significant edits should be made only within the first 24 hours and should not be self-sighted (they require peer review).

Headline (article page name) — See here.

Prose — Clear, concise, unambiguous. (Details here, especially here and here; also, here.)

Prefer active voice and active language, and promote the human aspect of the story (using quotes, etc.).

Use simple, plain English. No cliches or metaphors.

Attribute every factual claim to its source, within the story text and without interrupting its flow.

Attribute to non-news source when possible. Don't attribute uncontroversial non-exclusive information to a news source.

Don't direct-quote a source unless (very rarely) the source is part of the story.

Tense. Generally, use past, or present perfect. For timelines, present. For future events, say they're "scheduled", "planned", or the like.

Concentrate on the new facts.

Lede — Typically, the first paragraph succinctly answers the six basic questions:

Who What Where When Why and How.

Typically 50–80 words.

Oughtn't have too much detail; and once read, newsworthiness should be clear.

Body

At least another couple of paragraphs after the lede.

Avoid single-sentence paragraphs. Paragraphs of 50–80 words are usually about right.

Each paragraph should cover a single topic.

Article content should get gradually more detailed.

Balance

Is it presented in a neutral way?

Does it cover all the most relevant/interesting points of the story?

Are parts of the story given undue weight? Is there excess, encyclopedic content?

Is the relevance explained for an international audience?

Are controversial claims attributed? Is exclusive information attributed?

Be on the lookout for disagreements between sources; resolve the discrepancy, back off to a weaker claim agreed on, or acknowledge the discrepancy.

Other elements

Article templates

date/dateline — don't use {{dateline}} unless the reporter was actually there

lifecycle control — {{review}}, {{breaking review}}, {{under review}}, {{tasks}}, {{editing}}, {{develop}}

Citations sections

Sources — bulleted; {{source}}; note Citing syndicated (wire agency) content.

Related news — before Sources; bulleted; only articles published before this one.

External links — after Sources; don't use at all without good reason.

Other — Strongly discouraged as encyclopedic: References. Never: See also.

Categories

At least one geographical category — something from this hierarchy:

News articles by region

At least one major topic category — one of these.

Infobox

If any appropriate infobox can be found, use the most appropriate one.

The list of all infoboxes is here.

Don't trust the infobox to add appropriate categories; the categories should stay even if the infobox is later changed.

Peripherals

Picture(s) — caption, {{image source}}. See Image use policy.

Displayed quotations.

Have your say.

Miscellaneous — {{Translated quote}}, {{WikimediaMention}}, {{PhoneContact}}, etc.

Wikilinks should be internal to Wikinews whenever possible. Use mainspace redirects.

Check all non-Wikinews links to make sure they work. (Those links —including Wikipedia links— don't turn red when broken.)

Article links should go in Related news, {{Wikipedia}} in External links.

Sources

Acceptability — Each source should be reputable, verifiable, and must not be pay-to-access.

Two or more sources are required for a synthesis article.

List the sources from newest to oldest.

Only list sources actually used.

Freshness — New details of the story must have come to light within 2–3 days.
- There are rare exceptions for synthesis in both directions (early and late staleness); OR postpones staleness, in extreme cases by weeks.

The sources must verify all facts in the article (except the obvious).

No copyvios (copyright violations).

Paraphrasing a copyrighted source sentence-by-sentence is still a copyvio.

Gratuitous paraphrasing often leads to needlessly poor prose.

Copyvio

A lot of copyvio consists of passages copied from source and then "scuffed up". Sometimes a lengthy sentence or even several sentences will be copied this way; there may be only a few places where more than three consecutive words or so are identical to source, yet the whole lengthy passage may have the same grammatical structure and the same content words in exactly the same order, adding up to the same long passage as in the source — to someone who can read and understand the text, as a computer cannot.

Copyvio detector tools

For the verbatim-copied passages, there's a duplicate-detector tool, at toollabs:dupdet. With some work, the tool may also be useful for spotting copied-and-scuffed-up passages, by studying the context surrounding verbatim passages.
See also Earwig's Copyvio Detector, at toollabs:copyvios. This can analyze via external links present in the article itself, search engines for other websites, or both at the same time. It offers whole-document analysis rather than addressing individual passages, and the percentages offered should be interpreted accordingly.

The On Screen Edit gadget

Select the gadget on your user preferences. With Javascript turned on (you'll need that for the Easy Peer Review gadget too, of course), "On screen edit" will appear as an option on the toolkit on the left side of each page.

Open a tab viewing the article (not editing it), and click the gadget. You're now in on-screen edit mode. A toolbar appears at the top of that copy of the article.

To shade part of the text to some color, or to strike it out with a horizontal line (~~like this~~), highlight that part of the text and click the appropriate button on the toolbar. You can also delete the highlighted text entirely with the delete key; and, once you've clicked anywhere in the article body, you can even add text. All without having any effect on the saved version of the article.