Research:Section edit modification

This is a potential growth team project at the Wikimedia Foundation.

Background

The current position of the section-level edit button on Wikipedia may not be ideal from a usability perspective, and may possibly be obscuring the editable nature of the page to readers and potential new editors. Preliminary research done for the Vector extension has shown that moving the button from the far right corner of the article to the section header space increases the number of clickthroughs to the editing screen. However, the number of conversions (clickthroughs plus successfully saving changes) was still low for unregistered editors, which suggests that complex markup may have been a deterrent to editing.

Testing the section edit UX change again on a diverse sample of articles should show whether this modification can provide better conversion rates on some articles (e.g., stub and start class articles without complex markup) than others.

Justification

Impact:
Community:
Workload:

Methods

Timeline

Analysis requirements

A/B comparison of clickthrough rate of all edit links by user type (anon vs. registered) with section edit mods vs without (control).
A/B comparison of edit conversion rates by user type (anon vs. registered) with section edit mods vs without (control).
A/B comparison of clickthrough rates of all edit links and conversion rates on different types of articles (long, high-quality articles with complex markup versus short, low-quality articles with basic markup) with section edit mods vs without (control).
A/B comparison of clickthrough rates of edit conversion rates on different types of articles (long, high-quality articles with complex markup versus short, low-quality articles with basic markup) with section edit mods vs without (control).
A/B comparison of editor retention rates for editors who have access to section edit links vs no section edit links (control).

Overview analysis from previous test

(from Dario)

Yes – we do have a consistently higher number of clicks in the test group compared to the control group (+117.0%) and this higher volume of clicks does produce a higher number of actual edits (+8.6%). Conversions (edits per click) tend to be consistently lower in the test group (-29.0%) but we don't care as long as the change produces a higher absolute volume of edits.
No – there is no magical effect of changing the appearance of section edit links on the number of clicks or edits people do via the edit tab. The previous findings were flawed because of a single spike in the number of edit tab clicks occurring on March 21 and actually accounting for most of the observed change. I don't know what caused this spike but it looks like either a glitch in the data or something that really happened to registered editors and I don't have an explanation for.
The change in the appearance of SEL affects to a much larger extent anonymous users (for whom the increase in clicks and edits is very large and consistent over time) than registered users (for whom it's not clear without further testing whether the change makes any actual difference)