Research:AfC processes and productivity

Created

15:42, 5 April 2014 (UTC)

Contact

Aaron Halfaker (EpochFail)

Wikimedia Foundation

Collaborators

Jodi Schneider

INRIA

Bluma Gelley

New York University

Duration:  2014-04 – 2014-04

  Open access
via jodischneider.com

---

Research:Projects

This page documents a completed research project.

Previous research suggests that the en:WP:Articles for Creation (AfC) is having a negative effect on productivity. In this project, we examined AfC's process more closely to better understand the phenomenon with the goal of producing concrete design recommendations for AfC and similar strategies.

Much to our surprise, we found that AfC's process does not suffer from significant process delays. It takes less than a week for more submissions to be reviewed. However, we did find evidence that AfC's draft space is hidden from other potential contributors and that this hiddenness results decreased productivity. We also observed many process breakdowns due to a lack of literacy in Wiki process and templates on the part of newcomers.

Background

Article creation workflows

Article creation process (enwiki). Article creation process workflow for English Wikipedia is presented as a directed graph and split into "draft space" and "main space"

History of AfC

The sudden decline in newcomer success. The number of surviving articles created per page creating newcomer is plotted with a LOESS smoothed line fit.

Rising newcomer use of AfC. The proportion of newcomer created articles/drafts that started in the AfC workflow is plotted over time.

December 2005 -- Brion Vibber posts on en:Village pump (technical)

"Jimmy's had me disable creation of new pages by anonymous users on en.wikipedia as an experiment. [...]"

January 2006 -- Articles for Creation is created

"The easiest way to create a new article is to create an account then log in - no personal information is necessary. If you prefer not to create an account, you can post requests for article creation here. Registered users who see these requests can create articles for you. [...]"

February 2007 -- Articles for Creation expands to reviewing newcomer articles

"This page allows unregistered and new users to create new articles with the assistance of experienced Wikipedians."

November 2007 -- Failed RfC to turn anon page creation back on

"I can see no harm in turning on anon page creation for a period of 30 days, as long as after this period has finished, we can take a long hard look at the whole process and evaluate whether or not we want this as a permanent feature." -- Ryan Postlethwaite

"Sticking your finger into an electric shredder is not a sensible experiment, we know exactly what the result will be and confirming the prediction will be painful." -- TimVickers

August 2009 -- The Article Wizard is enabled

"Welcome to the Wikipedia Article Wizard! This wizard will help you through the process of submitting a new article to Wikipedia."

October 2011 -- The AfC is added to the Article Wizard as the recommended workflow

(AfC draft description) "If you would like feedback on your article, or if you don't want to register an account, your article will be created in project space and will be reviewed by another editor before being created."

(Direct to main description) "Place your article into Wikipedia straight away. Warning: if your article isn't ready to go live, you may find it quickly deleted! If this happens, see Wikipedia:Why was my page deleted?."

December 2011 -- The option to create and article "Direct to Main" is removed from the Wizard

"I propose that we remove completely the option of the Article Wizard that a page can be created directly out of the Wizard. Every experienced user knows how to do that or should or otherwise should use the wizard! [...]" --Mabdul

July 2013 -- The option to directly create articles is added back

"Just added a button to allow users to bypass AfC if they so wish to do so, since per policy they do not HAVE to use AfC" -- Charmlet

August 2013 -- AfC is "strongly encouraged"

"[...] you may create your article directly in the article namespace. But we strongly recommend that you use Articles for Creation instead."

Hypotheses

One potential problem is the delay that a pre-publication review enforces on draft creators. A long wait period between when an editor submits a draft for review and the completion of the review process may be discouraging. Such a long wait period would suggest that many good draft articles languish in a backlog which would would explain the decreased level of productivity. If AfC's processes are so slow, that could explain the observed productivity decline.

• Hypothesis 1: AfC's review process is too slow.

Another potential problem is that, the lack of visibility of drafts may reduce the capacity for efficient, stigmergic collaboration to take place. The process of loose coordination around articles in Wikipedia bears a striking similarity to open source software projects, which seem to operate under Eric Raymond's version of Linus's Law--“given enough eyeballs, all bugs are shallow”--since attracting ‘eyeballs’ seems to correlate with success. If this phenomenon were also present in Wikipedia, then hiding drafts from the view of potential collaborators in the project namespace would result in missed opportunities for collaboration – therefore reducing productivity.

• Hypothesis 2: Draft space is hidden from potential collaborators

Results & discussion

Bytes changed by others. The geometric mean bytes changed by non-creating users is plotted for 4 weeks after a draft/new article was created.

Revisions by others. The geometric mean revisions by non-creating users is plotted for 4 weeks after a draft/new article was created.

Unique other editors (anon). The geometric mean unique anonymous users editing the page is plotted for 4 weeks after a draft/new article was created.

Unique other editors (registered). The geometric mean unique registered users editing the page is plotted for 4 weeks after a draft/new article was created.

See also

• OpenSym paper
• slides from OpenSym