Grants:IdeaLab/Gamified Microcontributions on Wikipedia

Idealab

Gamified Microcontributions on Wikipedia

Quick, easy, fun tasks that enable people to contribute to Wikipedia in under 30 seconds.

idea creator• Ocaasi

developer• Rdds

advisor• Tuntunaung

this project needs...

volunteer

developer

advisor

project manager

community organizer

designer

researcher

created on14:18, 9 August 2014 (UTC)

Project idea

What is the problem you're trying to solve?

Knowing where to get started on Wikipedia is difficult. You don't always have time or knowledge for a big task, but you have time or energy for a small task. Editing can be mentally taxing and we want to create ways to reduce the mental friction to contribute.

What is your solution?

Inspired by The Wikidata Game, we want to create micro-tasks for improving Wikipedia. They should require a minimal amount of human decision-making or action and be able to be completed very quickly, in rapid succession, any time you have a few free minutes. It's important that the tasks need human intelligence without exhausting the participant.

Ideas for games

• Identify articles without images
• Pick an image from commons to add to an article without an image
• Gamify pages listed at https://en.wikipedia.org/wiki/Special:SpecialPages
• Spellcheck game (bot finds unusual words, human decides if it's a typo or legitimate)
• Fixing a broken link by finding where the url moved to
• Pronounce various words from wiktionary (or proper names in wikipedia) using some in-browser recording widget
• add image based on w:Category:No_local_image_but_image_on_Wikidata
• add reference using visual editor reference generator based on #1lib1ref campaign

Get Involved

Participants

• Advisor Find solutions to this idea. Tuntunaung (talk) 14:28, 9 August 2014 (UTC)
• Developer The idea looks very interesting for me. I am a software developer with some experience in game programming and artificial intelligence. I have experience with several languages that includes C++, python, java, R, web dev (HMTL, JS, CSS). I have also experience with collaborative tools such as cvs, subversion, git. Rdds (talk) 17:09, 5 December 2015 (UTC)

Endorsements

• More than ever, Wikipedia has a difficult time finding simple tasks for newcomers to get started with. This seems like it could go a long way toward fixing that. WWB (talk) 14:23, 9 August 2014 (UTC)
• This has potential. Taw (talk) 14:29, 9 August 2014 (UTC)
• Ayup. Will add more later. Protonk (talk) 15:45, 10 August 2014 (UTC)
• This sounds like a great new approach to onboarding. Let me know if I can help! Jtmorgan (talk) 01:03, 2 September 2014 (UTC)
• Slowking4 (talk)

Gamify the tools we have

What is the problem you're trying to solve?

Wikipedia has an editor problem. In comparison to the first half of its history, Wikipedia (especially though not exclusively the English Wikipedia) has gained fewer editors and new editors have faced a higher threshold to get their contributions to "stick" on articles. This could be caused by internal policies and practices, simple maturation of the site, competition for attention with social media or blogs, or a combination of the above. It does not, however, have an edit problem. Editors on Wikipedia commit edits to the site at a relatively constant rate over the past few years. Many of these edits comprise what we might consider "low hanging fruit" for new editors; fixing typos or references, reverting vandalism or making similarly minor changes. Among those edits, a large portion are performed via semi-automated tools. These tools are almost exclusively kept from new editors for a variety of reasons, namely their potential to allow for rapid edits across a large number of pages. Their very existence is also a secret to many readers and editors, not out of any attempt to hide the tools themselves but because of the significant knowledge of the project and affiliated technical resources required to use some of them.

That's a shame because the tools themselves and the edits they afford are among the most "game like" elements of the Wikipedia editing experience, allowing for editors to get in the "zone" and experience flow, offering structured goals and manageable milestones and potentially setting up opportunities for immediate feedback. And although we don't have a complete theory on what makes elements of Wikipedia game-like or fun, I feel the popularity of these tools among long term editors is a fairly strong sign that something about them is fun and rewarding.

What is your solution?

desired goals

In order to explore the game-like elements of these tools, we need to do a few things:

• Decouple use of the tools from intimate project/technical knowledge. As a practical matter, Wikipedia has probably already reached saturation among potential editors who have the time and technical expertise to operate these tools independently. AWB seems like fun, but it is insufficient to tell new editors (even if it were technically possible) to try it out and see for themselves.
• Break down the decisions required for use of these tools into smaller chunks. Tools like Huggle, HotCat or AWB offer a baffling array of editing options, many of which are (probably) not of interest to a new or marginally attached editor or which require deep knowledge of the project not immediately apparent to readers.
• Make the use of a subset of these tools productive for editors who want an ephemeral experience and safe/useful for long term editors watching over a number of articles. Semi-automated tools are restricted to certain editors because computers are bad at many things and humans are bad at judging the output of a computer. :) Further, a tool to allow for new editors to commit semi-automated changes would require at least some buy-in from the community and we would first have to convince them it would do no harm.
• Give editors an objective function that isn't merely "minimize the time to make a decision". Ideally editors should be nudged toward getting the "right" answer, but that's not always a solved problem.

how to get there

To accomplish the above I propose a system with three largely separable components:

1. A generator or generators for streams of potential edits. Tools like Huggle consume the recent changes feed (among other sources) for edits which may be vandalism. STiki uses algorithms to produce streams of potential vandalism by observing meta data. Similar streams could be built using non-vandalism related sources, e.g. categorization suggestions for recent articles, typo correction suggestions for bot selected articles or a number of other sources built by either running semi-automated tools on a labs server or building equivalent tools to search for potential edits.
2. A front end which consumes one or more potential streams of suggested edits and offers a highly simplified interaction to end users. Actions could be different for different types of streams or all streams could be written to allow for a small set of actions. For example, we could have 4 basic actions: approve, decline, edit and view. The data required by the front end should be likewise standardized, perhaps requiring a parsed snippet of the result, the article title and or a diff between revisions.
3. A system for actually committing the changes and monitoring the performance of editors and streams. Because the error rate for suggested edits from any tool is non-negligible, we need some method for judging the acceptability of an edit before and after it is committed. This may be as simple as requiring more than one editor to recommend the same change before it is committed to the encyclopedia. Tracking agreement on specific edits could extend to streams or editors. If a particular stream produces low quality results (much disagreement over identical edits) it can be tracked or removed. Similarly an editor who consistently disagrees with identical edits made by different editors can be given lower weight when tracking agreement.

The three components are separated not just by function but by likely expertise needed for developers and designers. The front end application should be as simple as possible and could be worked on by editors with no particular experience with semi-automated tooling on wikipedia or machine learning. Generators for streams could be built by tool creators or developers looking to learn how semi-automated tooling works on Wikipedia. The same goes for the back end system for agreement and monitoring.

agreement

As I mentioned above, the tools I am hoping to exploit are by and large deliberately withheld from new editors, often with good reason. Even experienced editors make mistakes using semi-automated tools and tools which preload a preferred outcome are particularly suceptible to this. Requiring that multiple editors agree on the same edit should reduce this, although it won't eliminate it.

Requiring agreement isn't merely defensive! It can help improve the tools used to generate streams if a particular stream causes a lot of disagreement. It can also provide a transparent solution to the negative reputation problem that Wikipedia has. If an editor makes an edit which is reverted (via a tool or not), they're often reverted and their talk page is templated with a note about the edit. Inside the UI of tools like Huggle, that prior revert has a strong reputational signal--editors with multiple reverted edits appear differently in the view of a diff showing potential vandalism. On wikipedia these templates are used informally by editors to see who might have caused problems in the past. Some policies require warning before blocking and the templates themselves serve both as a warning to the editor and a signal to an admin that blocking may be acceptable. In general this sucks because as an editor develops a negative reputation they either start a new account or leave the service rather than choosing to "straighten up and fly right". If instead we find that an editor using our tool doesn't make the "right" decision, their input can be silently modified to reflect this. That doesn't stop us from providing info to the user (in fact, that part is crucial) but it doesn't come at a reputational cost.

Agreement can also be an important part of the objective function. We don't just want editors to blaze through suggested edits, we want them to shoot for the "right" decision.

steps

1. Agree on a spec for streams of edits and individual suggested edits covering largely what data each stream must provide
2. Port a single tool from a run on demand service to a stream.
3. Build a front end tool to handle the output of a stream

1. Ideally a command line tool in node or python which can be dogfooded for testing
2. A modern web interface built for mobile first
3. Perhaps a single native application if that's easy enough. I'd like to try making a Swift app but if someone really likes android development we can do that as well.

1. Build a back end service for managing distribution of streams, recording of agreement between editors across edits and within streams across editors and scheduling committing of edits

what's cool about this

We can give editors on mobile a potentially fun way to make changes to wikipedia with very limited committment and time investment. The very simple streams will allow us to (potentially) let editors make changes with a gesture and get immediate feedback. By constraining the space for action we can tweak components of the UI or the underlying challenges to try to make them more fun or more effective.

By splitting off the generation of edit streams from the app itself we can leverage a lot of existing work in semi-automated tools (some of which have been in development for a long time and are quite robust) and by recording agreement--possibly also noting where changes have been reverted--we can provide real feedback to tool authors on the effectivness of individual tool components.

Constraints on actions also gives us a considerable degree of flexibility with respect to the UI. We can experiment with how to present the content and how to suggest actions as well as how an editor sees feedback. If we're smart enough, we can also adjust how an editor sees feedback what kind of materials they see without updating an app. Imagine starting with typo fixes and "leveling up" to approving more complex edits without an update to the app or changes to the basic interactions.

A testing system allows us to remain honest with ourselves and to test specific edits differently in a way that is totally transparent to the end user. For instance, a stream generates a suggested edit and we could present that to the editor as approval for the change or reverse it (without letting them know) and suggest an edit we suspect is wrong. An editor swiping right for "yes" on everything will quickly fail that test and we can use that to update their likely accuracy or prevent confirmation bias from swamping our system of agreement.

problems

Problems with technical solutions:

• Generating a stream of suggested edits from existing tools has an obvious problem. Those same tools can be run on demand by other editors or other editors could edit the articles between when a chunk of edits are suggested and when they are eventually committed. Resolving those conflicts has to be transparent (or preferably invisible) to the end user and not too computationally intensive (read: no repeated polling of articles).
• Tracking distributed changes is hard. It's a solved problem in general, so it's not a mystery, but it isn't trivial or easy by any stretch of the imagination.
• Porting tools designed to be run on demand over a single article into a generator that produces single edit suggestions suitable for quick consumption and action is also hard. For some tools it may be very difficult. For example, fixing all the hypenation errors in an article in one edit is probably useful. Fixing one is likely less fun and definitely less useful.
• Tracking editors over time may be hard. If the front end is a native mobile application it gets easier but if the front end is a web interface (as I'm hoping will be possible) we may not be able to perfectly match humans to actions. We could allow editors to sign in with OAuth but I'm hoping the audience for this will largely be people without accounts so we can't rely on that.

Problems without technical solutions:

• I don't know yet who gets the "credit" for making a suggested change. We cannot group edits from multiple people into a single stream, as that runs afoul of the foundation's policy on shared accounts. We also probably don't want to take away credit from an editor who made a change if we don't have to.
• Agreement may not be a very informative measure of an edit's worth. Semi-automated edits could be independently viewed as valuable but seen as problematic depending on factors which are well outside the scope of a tool like this. For example, if we had a stream which converted British English to American English, editors could disagree on even identical edits and edits which had agreement could be unacceptable to the community if the article in question is not one that should be converted to one variation of english over another.
• Saying "yes" or "no" may not be fun or interesting for editors.

summary

I think this is a promising but challenging route to get more editors involved with wikipedia. It may not result in users of the app signing up to wikipedia and making the kinds of edits we normally associate with long term editors, but that's ok! What we're looking to do is making minor contributions meaningful and fun for people. If someone uses the app on the train or in the bathroom and never learns about the Wikipedia community, that's still a success story. They'll still understand that wikipedia is written by regular people (dispelling a common misconception about who edits the encyclopedia) and they'll still be contributing to human knowledge. The sole purpose is to make meaningful edits fun for people who wouldn't otherwise know to contribute or wish to do so. Protonk (talk) 18:08, 28 August 2014 (UTC)