Research:WikiGrok
 | This page in a nutshell: This page describes a series of tests of mobile microcontributions run by the Mobile Team using the WikiGrok interface |
About
[edit]WikiGrok is an experimental MediaWiki feature designed by the Wikimedia Foundation's Mobile team, allowing readers of a Wikipedia article to extract and fill in structured data from this article. This feature is inspired by Magnus Manske's Wikidata game.[1] (read more). As of May 2015, the project is no longer in active development, however, community and involved developers are currently evaluating the experiment thus far, do define lessons learned and possible next steps.
Research questions
[edit]The high-level research questions that we want to answer are the following:
- What are the aggregate conversion and drop-off rates for each version of the widget?
- What's the relation between article traffic and conversions?
- Does the position of the WikiGrok widget affect conversions?
- What's the distribution of observations per statement obtained with each version of the widget on the Beta site?
- How much do distinct WikiGrok contributors agree on individual statements?
- Does the version of the WikiGrok widget affect the quality of contributions?
- Does logged-in status or edit count affect the quality of contributions?
Tests
[edit]The goal of the initial set of experiments is to dry-run the test with a subpopulation of users opted into the Beta mode and validate that the instrumentation and experimental design is sound, as a pre-condition for running the test on a larger population of editors and readers. All times are UTC.
| Test name | Start | End | Target | Notes |
|---|---|---|---|---|
| Test 1 | 2014-11-08 01:00:00 | 2014-12-02 01:00:00 | all users, alpha+beta mobile site, enwiki | first dry run, testing the instrumentation |
| Test 2 | 2014-12-12 00:41:00 | 2014-12-18 00:00:00 | logged in users only, stable mobile site, enwiki | small release on stable on logged in users only, testing patches to instrumentation and measuring conversions |
| Test 3 | 2015-01-13 01:00:00 | 2015-01-? | 16% of all users, stable mobile site, enwiki | first release on the stable mobile site with a sample of readers |
Data
[edit]Data for these tests is collected via the following logs:
- Schema:MobileWebWikiGrok — logs user interaction with the WikiGrok widget
- Schema:MobileWebWikiGrokResponse — stores the response submitted by the user via the WikiGrok widget
- Schema:MobileWebWikiGrokError — error log for the WikiGrok widget
Links to qualitative data reports the Design Research team gathered via guerrilla usability testing:
- Guerrilla Testing Wikigrok Version A
- Guerrilla Testing, November 2014: Wikigrok Version B
- Guerrilla Testing, December 2014: Wikigrok Version B v.2
Gallery
[edit]-
WikiGrok version A: binary question -
WikiGrok version B: tagging
