Research:Wikipedia Education Program evaluation

Contact

Mani Pande

Wikimedia Foundation

---

Research:Projects

This page documents a planned research project.
Information may be incomplete and change before the project starts.

Key Personnel

• Mani Pande
• Ayush Khanna

Project Summary

This project will track key metrics of the ongoing and upcoming Wikipedia Education Program activities in the United States, Egypt, Brazil and Canada, and conduct pre- and post-surveys with students, professors, and community members.

Methods

We will be using user data from students enrolled in the Wikipedia Education Program to measure success of the programs in different geographies. We will also be collecting data through surveys to measure satisfaction of the participants in the program.

Methodology for WEP comparative metrics (Fall 2011)

This research compares the contributions of students of the U.S. Wikipedia Education Program in the Fall 2011 term with a random sample of new users who created an account on English Wikipedia in September 2011. The month of September was chosen to coincide with the start of the Fall 2011 academic term in the United States.

1. First, we scraped usernames from the course pages for courses that participated in the U.S. Wikipedia Education Program in Fall 2011 (920 students in total).
2. Next, we generated a random sample of 1,000 users who created an account in the month of September.
3. For each group, we calculated a set of metrics, including their edit counts, percentage of edits in article space, and edit activity in the last month. These were collected via the Wikipedia API.
4. In addition, we wrote a script to calculate the amount of content added by each user, as well as the amount that survived on Wikipedia (as of the day the script was run). (Note: This calculation was done in "significant bytes" - i.e., bytes that did not include white spaces. Typically, white space accounts for 15–20 percent of total bytes.)
5. Next, we calculated some aggregate metrics to compare these two groups.

Dissemination

All findings will be publicly available on a WMF wiki.

Benefits for the Wikimedia community

Community and foundation will be able to better gauge effectiveness of the education program and improve it.

Timeline

(Ongoing)