User:Michael.C.Wright/sandbox/The problem

One of the hallmarks of an institution in crisis is that, far from preparing for the future, it is barely capable of managing the present.

– Ashley Rindsberg

For years various editors have complained about the lack of active reviewers at en.Wikinews. I am one of them. Below are my notes trying to make sense of the problem.

The problem defined

I don't believe one can truly evaluate a system from within the system (John Boyd). When you're part of the system, your view is restricted by the system’s rules, constraints, assumptions, and inherent biases. This internal perspective makes it challenging to identify structural problems, since your thinking and actions are shaped by the very elements that may need to be questioned or changed.

I do not consider myself fully part of the system yet, as a new reviewer with only a handful of reviews under my belt. Therefore I consider my observations of the system to at least be largely unrestricted by the system’s rules, constraints, assumptions, and inherent biases.

Per Boyd's understanding, outside information and fresh perspectives are needed to reinterpret circumstances, break from patterns, and make better decisions. Boyd believed that stepping outside the system enables a clearer view of its limitations, inefficiencies, and even unintended consequences, which are often invisible from within due to cognitive and operational biases.

The overall problem is our rate of publication is low and sporadic. This is possibly the most glaring sign of a problem. A news organization that publishes no news articles for several weeks straight is not a reliable source for news.

In the following analyses, I try to find trends and correlations that may help us prioritize changes in an attempt to increase our rate of publication.

Short term analysis (12 months)

Note: A key metric missing is the number of active reviewers each month. I'm still trying to figure out how to determine this historical data and better track it moving forward.

The first data set is a count of the number of articles published each month along with the number of days in the month with zero articles published.

Monthly publication rate and number of days with zero articles published

Month and year	# of articles published	# of days with 0 articles published
Category:January 2024	2	29
Category:February 2024	0	28
Category:March 2024	10	24
Category:April 2024	9	23
Category:May 2024	5	28
Category:June 2024	17	20
Category:July 2024	1	30
Category:August 2024	1	30
Category:September 2024	10	21
Category:October 2024	15	22
Category:November 2024	7	24
Category:December 2024	16	20

Statistical Insights on Article Publication (Updated)

Metric	Value
Average Articles Published per Month	7.75
Median Articles Published per Month	8.0
Standard Deviation of Published Articles	6.11
Total Articles Published in 2024	93
Total Days Without Articles	299
Month with Most Articles Published	June 2024
Month with Fewest Articles Published	Feb 2024

2024 English Wikinews publication rate, per month

Analysis of Q4, an above-average quarter

The period of October through December, 2024 had an above-average number articles published.

In September we also added one new reviewer. For the months of September, October, November, and Decemeber, the new reviewer published 20%, 15%, 6%, and 43% of all articles, respectively.

The difference in publication rate compared to our monthly average rate:

• September (10 articles): -2.91% below average
• October (32 articles): +210.68% above average
• November (18 articles): +74.76% above average
• December (16 articles): +55.34% above average

Therefore the increase in articles published by the new reviewer does not account for all of the increase in articles published. Q4 also had an above average number of active editors, which is further explored below.

Boom and bust periods

Below is an analysis of the data by ChatGPT:

• The months with low article publication (5 or fewer articles) were: Jan 2024, Feb 2024, May 2024, Jul 2024, Aug 2024.
• February 2024 had zero published articles.
• July and August 2024 saw only one article each.
• These months also had the highest number of zero-article days (28–30 days).

Recovery in Activity

• September 2024 marked the beginning of an increase (10 articles).
• October 2024 saw a major spike in publication (32 articles), the highest in the dataset.
• November and December 2024 continued the trend with above-average publication rates (18 and 16 articles, respectively).

Strongest Months

• October 2024 had the highest output (32 articles) and the lowest number of zero-article days (13).
• November and December maintained a relatively strong presence but with a small dip.

General Trend

• First half of the year (Jan–Jun): Low publication rate, averaging 7.7 articles per month.
• Mid-year slump (Jul–Aug): Very little activity, with only one article per month.
• Second half of the year (Jul–Dec): Higher activity, averaging 13.0 articles per month.
• End-of-year surge (Oct–Dec): The busiest publishing period, peaking in October.

Pages edited vs articles published

In addition to the publish rate dataset used above, the following, remaining sections of the Short Term Analysis use data from the following Wikimedia stats:

• Active editors [1]
• Net bytes difference[2]
• Edited pages[3]
• User edits[4]

The following is a comparison of pages edited vs articles published. It compares two data sets; the first is the number of articles published each month (the table at the top of this page). The second data set is the number of pages edited each month with the following metrics filtered out: anonymous, group bot, and name bot accounts and non content, or generally content not in main space. This gives us close to a like-for-like comparison of draft articles (pages edited) vs published articles.

Monthly article publication rate vs the monthly pages edited (filtered out anonymous, group bot, and name bot accounts and non content) on English Wikinews in 2024

I asked ChatGPT to evaluate any correlation between the two and this is the response:

The Pearson correlation coefficient between the number of articles published and the number of pages edited is 0.29, which indicates a weak positive correlation. The p-value is 0.36, suggesting that this correlation is not statistically significant at conventional levels (e.g., p < 0.05).

One area of likely "corruption" or skewing of data would be vandalism, which will increase the number of pages edited if the vandalism includes creating new pages. I asked ChatGPT to exclude the three outlier months that each include over 80 pages edited to see if that changes the correlation:

After excluding the outlier months where more than 80 pages were edited, the correlation coefficient increases to 0.69, indicating a moderate to strong positive correlation between the number of articles published and the number of pages edited.

The p-value is 0.039, which is statistically significant at the p < 0.05 level. This suggests that when the extreme outliers are removed, there is a meaningful relationship between the number of pages edited and the number of articles published.

This means that higher editing activity (within the normal range) is associated with more articles being published, and efforts to increase productive editing could help drive publication rates.

Pages edited vs active users

The last quarter of 2024 had an above-average number of active editors.

According to ChatGPT:

The correlation coefficient between edited pages and active editors is approximately 0.445. This indicates a moderate positive correlation, meaning that as the number of active editors increases, the number of edited pages also tends to increase, but the relationship is not particularly strong.

Correlation analysis

I asked ChatGPT to evaluate the correlation between the following metrics: articles published, pages edited, net bytes difference, and active editors. It produced the following correlation matrix:

Correlation matrix generated by ChatGPT. Values closer to 1.0 (colored red) are more correlated with each other.

I then asked it to identify any trends in the same data used to evaluate correlation. It returned the following:

Positive Correlations Between Metrics

Articles Published and Pages Edited: A moderate to strong positive correlation indicates that when more pages are edited, there is typically an increase in the number of articles published. This suggests that higher editing activity may contribute to publishing more articles, possibly due to active engagement in content creation or review.

Active Editors and Articles Published: A strong positive correlation shows that an increase in the number of active editors often corresponds to more articles being published. This highlights the importance of maintaining and growing an active user base.

Net Byte Difference and Pages Edited: This strong correlation is expected, as editing activity (e.g., adding or removing content) directly contributes to changes in the net byte difference.

Active Editors Drive Content Activity

The number of active editors correlates positively with all other metrics, including: Articles Published, Pages Edited, Net Byte Difference

This indicates that active editors are likely the primary contributors to editing, publishing, and overall activity on the platform.

Net Byte Difference as an Indicator of Content Depth

The correlation between Net Byte Difference and Articles Published suggests that articles published might also involve significant edits, whether in content expansion or reworking existing drafts. Large byte changes may be associated with substantial article contributions.

Insights for Actionable Strategies

Boost Active Editors: Since active editors have a direct impact on multiple metrics, strategies to recruit, retain, and support editors could lead to increased overall activity and publication rates.

Encourage Editing Engagement: Focused campaigns to increase the number of pages edited, such as collaborative events or themed editing days, could directly boost article publication rates.

Track Editing Quality: Monitoring net byte changes alongside other metrics can provide insights into the depth of contributions and their impact on publication.

Long term analysis (all time)

I provided ChatGPT the following data:

• Edited pages, all time (Nov. 2004 - Jan. 2025), filter to show only; User, Content[5]

It returned the following:

General Trend

• First half of the year (Jan–Jun): Lower editing activity, averaging 338.8 pages edited per month.
• Second half of the year (Jul–Dec): Increased editing activity, averaging 366.0 pages edited per month.
• End-of-year peak: The highest editing activity is observed in the latter months, possibly due to coordinated efforts or major events.

The number of pages edited (excluding redirects and non content pages) by Users (excluding Anonymous, group bot, and name bot) on English Wikinews in 2024.

How Does 2024 Compare to Previous Years?

• The average pages edited per month in 2024 is 29.2.
• Historically, the highest yearly average was 1120.2 pages edited per month.
• The lowest yearly average was 29.2 pages edited per month.
• The historical overall average (all years) is 354.5 pages edited per month.

Are We Trending Up, Down, or Neutral?

• 2024 is the lowest on record.
• The trend appears to be declining compared to previous years.
• Editing activity in 2024 is 8.2% of the historical average.

This suggests a downward trend in user-only content page edits.

Too few reviewers or too few draft articles

We need to know the number of articles that are marked stale while in the review queue to more accurately identify the source of our problem as too few articles developed or too few reviewers. To do this, maybe we should do something like the following test for 90+ days:

1. When articles go stale in the review queue, use {{stale}}
2. If the article is abandoned, use {{aband}}
3. After two days of abandonment, instead of deleting the article, we do the following:
   1. blank the content (it remains available in history but not readable otherwise)
   2. add a new template + category, something like {{Stale without review}}

In this way we can better track the number of articles that go stale in the review queue without ever being reviewed.

See also: Gryllida's Task 33

Possible areas of attention

Based on the data and analysis above and the historical conversations or statements found below, we could focus our attention on the following areas:

• Reduce the amount of time and effort required to publish an article
  • This could include review and developing markup below
  • This could include revisiting and changing the review process itself
• Increase the number of active reviewers
  • This could include asking inactive reviewers to re-engage
  • This could include actively seeking new reviewers
• Increase the number of active authors
  • This could include reaching out to journalism departments of various colleges/universities (has been done in the past)
  • This could include collaborative events or themed editing days (we've had writing contests in the past)

Proposed actions

Review and developing markup

Part of the delay in publication comes from the back-and-forth between reviewer and author(s).

Reviewers can't expand articles or add sources and remain uninvolved. If an unsourced statement is found, the reviewer could simply remove it but that might negatively impact the flow of the article or its coherence.

To help alleviate this, I think we should develop a set of templates specifically to be used before publication to allow authors and reviewers to quickly, clearly, and accurately communicate between each other.

I have developed {{Verify}} and {{PhraseReview}} as examples.

Other useful templates may be:

• {{Note}}
• {{Needs attribution}}

• Question how to prevent edit conflicts. maybe better to write questions on article talk and ping the author(s)? Gryllida (talk) 10:48, 26 October 2024 (UTC)

If by edit conflict you mean where two editors click 'save changes' to an article at the same time? If so, I'm not worried about it. I don't see how using in-line markup in the article would increase the risk of edit conflicts. Reviewers should use {{under review}} to prevent other edits to the article and editors can use {{editing}}.

I look forward to the day when we are so busy, we have to worry about edit conflicts. ツ —Michael.C.Wright (^Talk/_Published) 13:26, 26 October 2024 (UTC)

I'm kind of worried as I would prefer talk page format to notes in draft. It means I have a place where to leave a message while keeping draft nice and clean. It also means I can reply to each concern separately using threaded messages on talk page. For my own drafts I would prefer to keep it that way. I believe consensus or a consent from draft author would be needed to change it to the 'verify' templates. Gryllida (talk) 03:14, 27 October 2024 (UTC)

I don't understand your hesitancy here.

• It is already common to recommend that authors use hidden, in-line citations to aid the review process. {{verify}} can be the same thing, but not hidden.
• I believe we could more effectively communicate between author and reviewer using markup in addition to the talk page.
• Eventually requiring authors to clearly cite information will speed up the review process by improving the quality of early article drafts.
• How does it benefit the project to prevent us from using {{verify}} due to lack of consensus or consent?
• What else do we require an author's consent before doing?

I believe consensus or a consent from draft author would be needed to change it to the 'verify' templates. Why would you like to see consensus for this change, which directly supports core policy, but not feel you need consensus to unilaterally disregard core policy in your own, current, 5W-only review process? —Michael.C.Wright (^Talk/_Published) 14:04, 27 October 2024 (UTC)

Past conversations

One aspect of en.WN culture is the notion of institutional knowledge. I am not a fan, as it represents hidden or exclusive knowledge.

Organizations spend a lot of time and resources developing knowledge and capability. While some of it gets translated into procedures and policies, most of it resides in the heads, hands, and hearts of individual managers and functional experts. Over time, much of this institutional knowledge moves away as people take on new jobs, relocate, or retire.

– Ron Ashkenas, Harvard Business Review

Below are past conversations on the topic of too few active reviewers. The list should not be considered exhaustive.

Also keep in mind the list is inherently biased—I am looking for conversations where someone is complaining about the review system.

2018

'Wait just a bit longer...'

In my opinion, in at least the last six years, the 'just wait' crowd has not yet been proven right.

and I hope you forgive me for feeling unenthusiastic about the prospect of waiting just a bit longer, since that's essentially the argument we WN people have made since the days I was a bureaucrat on this wiki. IlyaHaykinson

I draw two conclusions from that observed pattern: (1) if we could do things that significantly and permanently increased our review capacity per reviewer (or to put it differently, per unit of review labor), we could sustain a significantly increased level of output. Pi zero (in direct response to above comment)

The following conversation echos many still being had.
The best way to prove the bad reputation of this site wrong is an increase of relevant, informative news stories, not useless denial of the obvious misery (a la Baghdad Bob). Any animosity or snobbishness towards the bigger sister is totally unproductive, in a situation where Wikinews is dying from a scarcity of fresh content. Gray62

Proposed solution by Pi zero: tweaking existing processes using 'semi-automated' tools.
The key to tweaking the parameters, as I see it (and as I must have mentioned before), is semi-automated assistance to streamline the various on-wiki tasks, including (though not limited to) both writing and reviewing. It's essential to follow the semi-automated route, where the essential human component is carefully preserved and nurtured, because that human component is part of both the quality, the robustness, and the ideals of the project. Pi zero

2013

It seems to me that the long-term viability of this project would be substantially improved just by attracting a larger readership. CMBJ

There is (averaged over time) as much demand for what we do as our reviewers can supply: increase our review capacity and, with a time lag, demand for review goes up until it saturates the available supply. Pi zero (in direct response to above comment)

I interpret what Pi zero is saying here is that with more active reviewers we'll get more articles written and conversely; with fewer reviewers fewer articles. We've known for years that reviewer capacity is a key to solving The Problem.

2010

Review waits are dismal. Many of our articles are coming from people as-yet too inexperienced to receive reviewer, or who do not have English as a native language. A supply of people regularly writing has dried up. Blood Red Sandman

'Bust' of boom and bust blamed on the lack of AGF
We have a long-term problem here. A (sort of) boom and bust cycle, playing out on a scale of multiple years. Our community dynamic is unstable, resulting in the boom that peaked early this year, and the bust we're in now, because we (necessarily) don't have AGF. No AGF means that unpleasantness doesn't get to hide behind superficial forms. In good times, our authenticity makes this a bright place (quite refreshing compared to WP), leading to more new contributors and more good times. When things go sour, the reverse happens. Boom and bust. Pi zero

quality energizes the project
"If the choice is between ..." There is no such choice. There is no choice between quality control and anything else. Quality control is a sine qua non; without it we'd be an inferior sort of community blog, with no good reason for existing. We want to improve by getting things out the door faster. We want to improve by growing. Quality control is an inherent feature of the landscape where we want to do those things. Put it this way: without quality control, the project would not grow, it would wither and die. Pi zero

Your contention is that the sine qua non is our quality control. My response to the current "crisis" is to increase the size of the community, while yours is to defend the standard of quality. Neither is necessarily better than the other, but I think mine can lead to yours, while I do not see how yours can lead to mine. Amgine (in direct response to above)

2008

Near the inception of the 'modern review' system

Since the changes to the reviewing system in late July/early August, (which led to compulsory peer reviews) the number of new articles being published appears to have fallen dramatically. It also takes much longer to get an article published than it used to, due to the need to involve a third party. GW_Simulations

And some random banal thoughts to boot

• I love all of this and it's very smart (someone send ChatGPT a few cookies, please!)-- I have seen the boom-bust-boom-cycle play out here for years, and even to this day: I don't know what the answer is. I don't feel as though we have EVER had a coherent discussion about working video into our flow here more readily(which might help a lot!). Good writers, good articles and good turn-around are the answer -- and yes, there is still TOO MUCH back-and-forth on articles. And, to this day: I just don't know who our 'customers' are. It aggravates my innards, that -- honestly: WP is (at least somewhat) better at handling 'news' than we are -- like it or not, that's the way it is. It shouldn't be that way.--Bddpaux (talk) 19:19, 22 January 2025 (UTC)

Bddpaux, I agree that WP is more effective at getting content out there than we are. I think looking at WP vs WN can help us understand The Problem. They don't have peer review and they already have a large author base. I believe our issue lies in those differences. I believe our review process and lack of editors (and the related things that go along with both conditions) are the source of our issues.

That's why I was hopeful when you tried to shake things up by rapidly adding a new reviewer. Looking back at old conversations, such as those had, or attempted to be had, by yourself, Gray62, and others, I see we're stuck in a rut and I believe many reviewers and admin fail to see (or refuse to admit) we have a perennial problem.

I'm not prepared to, or proposing that we scrap the review process but it does need an overhaul. Simply adding reviewers can help but is not the answer. I think that was illustrated when I was added as a reviewer. My additional contribution only accounts for a small portion of what gets published[6]. —Michael.C.Wright (^Talk/_Reviewer) 15:05, 23 January 2025 (UTC)

Some thoughts

I don't love ChatGPT but I do love quantifiable stuff! I was just thinking that you'd need to look back a lot further than one year to see meaningful patterns here on Wikinews and then you incorporate anecdotal evidence to do just that. A good start. I was very active here during what I'll call the Pi zero era (whether you thought his contribution was good or bad, there is no denying it was big). There were times when half the articles on the front page were articles I'd drafted. So most of what I have to say concerns that stretch from 2016 on.

Wikinews needs to stop driving capable article drafters away.

It's not failure to attract them. It's that they show up and then they are driven away. There are two things that drive drafters away.

1. Frustration with the review process. They draft an article that looks fine to them and Wikinews just throws it away. That's going to bother anyone. What we're doing with Melledelle right now, making sure that first article is in shape and has a decent shot, we've got to do that with everyone who shows up willing to put in a decent effort. We also need to foster acceptance from day one: Even perfect articles sometimes age out. If you can't handle that, then Wikinews isn't for you.
2. Toxic Wikinews culture closely associated with fetishization of reviewer status. I could get into that, but I'd be too tempted to rant. One thing we could do to fix it would be to make sure that reviewers continue to draft articles. Drafting X number of articles per unit time could be a requirement for maintaining reviewer status. This would do a lot. a) It would keep the pool of article drafters from shrinking too much. b) It would keep reviewers from losing perspective. c) It would provide proof of continued perspective to drafters who merely thought said reviewers had lost perspective. Another? Formal review could be required to draw overt, explicit distinctions between "must have" changes and "would be nice" changes. Everything required for publication (here defined as having a written, extant policy, guideline, or non-subjective precedent to link to) would be clearly marked as such. Everything that the reviewer thinks would make the article better but is not necessary for publication would clearly marked as such. Article gets published as soon as "must have" is handled. If reviewers think "But we don't have a written guideline/policy for this thing that I believe all articles need," then the community gets together and decides whether or not to write one. I've got more. Darkfrog24 (talk) 00:46, 30 January 2025 (UTC)

I tried to implement your "must-have" vs "would be nice." Let me know if this is along the lines of your suggestion: Talk:Hankede Bridge opens amid delays. I think part of this is to revisit how we prioritize policies vs guidelines (PaGs). The argument has been made to me recently that there is no hierarchical difference between them, that they are all required and I disagree. I think if we are to reform the review process, that could be one avenue; by rethinking the hierarchy into policies being 'must have' and guidelines being 'would be nice.' We'd need to review all PaG's as part of that.

I see a fervent protection of 'reviewer status' by some and I think in some cases it is warranted. I think potentially-new reviewers should, at a minimum, have already published several articles and also have demonstrated the desire to help others get articles published—even if just by improving existing drafts. Reviewers must understand our PaGs as written as well as how they have historically been interpreted. I'm also not saying we should robotically continue previous interpretations.

Regarding an on-going requirement of reviewers writing articles; I personally backed off writing due to an extreme lack of active reviewers plus having several articles go stale in the review queue as a result. I believe we've now had two months of maybe two active reviewers. Every article written by a reviewer requires at least two reviewers to ultimately get published. In times of reviewer scarcity, this is sub-optimal. One could argue that if there are zero articles in the review queue, then a reviewer should write an article, and I might agree with that. However, there are many other tasks a reviewer could do; such as sighting changes to published articles, managing abandoned articles, updating documentation, etc. Wikinews is in a increasingly-rapid state of decay. There is a lot of routine maintenance and upkeep either not happening or happening too slowly. —Michael.C.Wright (^Talk/_Reviewer) 18:12, 1 February 2025 (UTC)

The many-article requirement and other standards for becoming a reviewer are not the part of reviewer status that I consider fetishization or toxic.

Drafting articles and watching them go stale? Yes, reviewers should have to put up with that to keep their reviewer status. That's part of perspective. Reviewers need to keep in touch with both the emotional and practical sides of drafting: How long does it take to run down an additional source? How much work is it to do X or Y? And they must be able to point to their work as proof that they know these things. In fine, reviewers must not order drafters to do things that they haven't done themselves in [duration] years and have forgotten about.

I would have a reviewer tell me to make a change to the ariticle (usually one that I did not think good or necessary) and I would think "Does the reviewer know that this would take me hours, if the source even exists at all? Maybe not." Whether I was right or I was wrong in any given case, this solves the problem by giving reviewers a credential or even a specific case to point to. Darkfrog24 (talk) 13:48, 5 February 2025 (UTC)

I looked at your review of the Addu City Council article, and yes, that's what I was talking about with must have vs would be good. Darkfrog24 (talk) 16:52, 5 February 2025 (UTC)

Project guidelines

Hi, I think I saw something mentioned earlier about possibly working on the project guideline pages to make them more easily accessible and to make them more useful for first-time editors (refute me if that is some random dream I had at some point). I wrote my company's training program and I also did technical writing projects in college. I've published something approaching 15 articles here through trial and error, and I'm getting more fluent with the site's policies and guidelines with every article I publish I think. I think I could be an asset. Let me know if you want my input if those guidelines pages are to get worked on. Have a good day! Lofi Gurl (talk) 20:52, 10 February 2025 (UTC)

Lofi Gurl, refreshing our policies and guidelines should be a central part of this 2025 project, in my opinion. Further to this discussion on my talk page; maybe it would be good for you to jump into the proposed Wikinews:Correction policy and start suggesting actions to finalize it and move forward making it official. As a very new addition to Wikinews and an active contributor, despite your self-professed semi-retirement ツ you still have an outsider's perspective, which in this case I see as a good thing. Because of your newness here, I would not recommend you jump in and start making edits directly. Instead, start by proposing changes and actions in talk space and get a discussion started. Maybe research best practices for corrections and retractions in journalism. [7], [8], [9], [10]

Think wholistically. For example, if we finalize our corrections policy, should it include a correctionswn-reporters.org email address for readers to share corrections with us? Should that be included in {{correction}}? Are there more creative ways we could receive these communications from registered and unregistered accounts alike?

There are discussions scattered about that you could search for, locate, and try to centralize.[11]

While we're talking about feedback, I welcome your feedback—and that of others—on your experience with the review process as an author. If you have specific thoughts on any of my reviews especially, please share them. We intentionally do not follow an WN:AGF policy and I’m quite open to and appreciative of constructive criticism. —Michael.C.Wright (^Talk/_Reviewer) 17:45, 11 February 2025 (UTC)