Purpose of the report

This form is for organizations receiving Annual Plan Grants to report on their results to date. For progress reports, the time period for this report will the first 6 months of each grant (e.g. 1 January - 30 June of the current year). For impact reports, the time period for this report will be the full 12 months of this grant, including the period already reported on in the progress report (e.g. 1 January - 31 December of the current year). This form includes four sections, addressing global metrics, program stories, financial information, and compliance. Please contact APG/FDC staff if you have questions about this form, or concerns submitting it by the deadline. After submitting the form, organizations will also meet with APG staff to discuss their progress.

Global metrics overview - all programs

We are trying to understand the overall outcomes of the work being funded across our grantees' programs. Please use the table below to let us know how your programs contributed to the Global Metrics. We understand not all Global Metrics will be relevant for all programs, so feel free to put "0" where necessary. For each program include the following table and

Next to each required metric, list the outcome achieved for all of your programs included in your proposal.
Where necessary, explain the context behind your outcome.
In addition to the Global Metrics as measures of success for your programs, there is another table format in which you may report on any OTHER relevant measures of your programs success

For more information and a sample, see Global Metrics.

Overall

Metric	Achieved outcome	Explanation
1. # of total participants	16,887	We exceeded our goal of 16,035.
2. # of newly registered users	15,699	We exceeded our goal of 15,520.
3. # of content pages created or improved, across all Wikimedia projects	18,028	Due to the Student Program change noted below, we fell short of our goal of 19,960.
4. # of words added to the article namespace	14.2 million	We exceeded our goal of 12 million words added.
5. # number of articles that have at least a 10-point improvement ORES-based quality prediction score	4,935	We fell about 500 short of our goal of 5,475. See below for an explanation of our learnings about this metric.

Telling your program stories - all programs

Introduction

Calendar year 2018 was one of big change for Wiki Education — and our most impactful year to date. Through our strategy, fundraising, and programmatic work, we made great strides and have set ourselves up for an even more successful 2019 — with lots of learnings along the way.

Strategy

Our board met in January to confirm our strategic direction for 2018–21, with a focus on three pillars: Equity, Quality, and Reach. In June, the board approved the strategic plan emerging from that direction. Our new three-year strategy, includes the following goals: (1) Increase knowledge equity by focusing on content and communities that are under-represented on Wikipedia and Wikidata; (2) Provide people who seek knowledge online with accurate information in topic areas that are underdeveloped; and (3) Reach large audiences with free knowledge by making Wikipedia and Wikidata more complete.

This strategy complements our existing strengths, while more directly aligning our work with the overall Wikimedia Movement strategic direction in both the "knowledge as service" and "knowledge equity" pillars. Our support of the Program & Events Dashboard is key to both pillars. As an important piece of infrastructure used by 2,006 program leaders globally to support programs on 101 different wikis, the P&E Dashboard is a tool that serves the global Wikimedia community as we interact with GLAM institutions, universities, and other knowledge repositories to collect and use our knowledge. P&E Dashboard usage more than doubled in 2018, from 1,418 to 3,240 courses or events tracked. Because it works across all languages and all projects, global Wikimedians working on knowledge equity issues have found it to be a key piece of software; it's been used by program leaders doing projects on the Acehnese Wikipedia (ace.wikipedia.org) to the Taiwan Chinese Wikipedia (zh-tw.wikipeida.org) and everything in between, including the Arabic Wikibooks, the Hebrew Wiktionary, the Indonesian Wikisource, the Italian Wikinews, the Polish Wikivoyage, the Portuguese Wikiversity, and the Spanish Wikiquote. Many of the courses and events tracked in the P&E Dashboard are run by program leaders specifically interested in improving knowledge equity.

Our own work in knowledge equity has been confirmed as a priority in our new strategy, in terms of both content and contributors. While we are unaware of any Wikipedia editing community research that has identified race of contributors, our survey results from fall 2018 showed 37% of our student editors identify as non-white; we believe this is significantly higher than the general editing population. Through our targeted outreach work to faculty who teach in equity content areas, we've also improved thousands of articles related to race, gender, and sexuality.

Fundraising

In 2018, we hit a major milestone: We brought in our first money from our new fee-for-service model. Thanks in part to a push from the FDC and an organizational development grant from the Hewlett Foundation, we worked with a business development consultant to explore possible business models for our work. The investigation revealed we'd have a hard time making a fee-for-service model work with our Classroom Program. However, we discovered demand for a version of our Wikipedia Fellows program aimed at providing professional development credits — and institutions have professional development budgets to pay for this. As such, we re-organized our staffing to create a new Advancement Department tasked with enacting the new fee-for-service model, rebranded our programs into the Student Program (formerly Classroom) and Wiki Scholars & Scientists (formerly Wikipedia Fellows), and set off to bring in money in this new model. In our first two courses, we partnered with the National Archives and Records Administration (NARA) to encourage experts to improve articles related to the 100th anniversary of the 19th Amendment to the U.S. Constitution, which granted women the right to vote. NARA is planning an exhibit for 2019 about women's suffrage, so we were able to sell two courses of professional development work to improve articles related to women's suffrage using NARA resources.

In addition to our FDC grant, the match of the FDC grant from the Stanton Foundation, and our January donation from the Pineapple Fund, we also received major support for our Communicating Science initiative from the Heising-Simons Foundation and the Moore Foundation. We also received a renewal and increase on our grant from the Hewlett Foundation.

Programs

2018 was our most successful year ever in terms of impact to Wikipedia. Our programs added 14.2 million words to Wikipedia — 2 million more words than we added in 2017. In early 2018, we also reached a major organizational milestone: Contributions to English Wikipedia by participants in Wiki Education's programs have now eclipsed the final print edition of Encyclopædia Britannica. The entire last print edition of Britannica contained 44 million words in 32 volumes; from 2010 to 2018, student editors in our Student Program have added more than 53 million words to Wikipedia.

With nearly 17,000 participants in our programs, we're successfully supporting large numbers of new Wikipedians at scale. And we're making a difference in the global numbers; according to the Wikimedia Foundation's Audiences team, we bring in 19% of all new editors on the English Wikipedia — that's 9% of all new editors in all projects globally each year joining the Wikimedia movement from Wiki Education's programs.

As the slide from the WMF Audiences team's deck pictured at right demonstrates, we're making such a big impact on the new editors on the English Wikipedia that in early 2018, we actually brought in more new editors than every other source of new editors to the English Wikipedia combined. For years, a major focus of the Wikimedia Foundation and its affiliates has been on bringing in new editors; Wiki Education takes particular pride in our many-years effort to demonstrate impact on the new editor numbers at scale, which several staff members have been steadily working toward since 2010. We look forward to continuing this trend in 2019.

Student Program

Note: In late 2018, as part of the organizational reorganization mentioned in the Fundraising section above, we renamed our programs to better distinguish the target audiences. In our proposal, this program was named the "Classroom Program" but is now known as the "Student Program"; we are using its current name throughout this section.

Quantitative targets

Measure of success	Type	Goal	Actual	Explanation
New discipline-specific handouts developed	Input	4	4	We published new discipline-specific handouts on science communication, LGBT+ studies, cultural anthropology, and art history.
New academic association partnership agreements signed	Input	2	0	We did sign a partnership with the Deep Carbon Observatory focused on another of our programs.
Academic association conferences attended to recruit instructors	Input	15	17	We attended two more than anticipated.
Number of students	Output (participants)	16,000	16,696	We anticipated supporting 16,000 students, and came in right above target.
Number of newly-registered students	Output (participants)	15,500	15,555	We set 15,500 new participants as our goal, and met it.
Percentage of students who identify as women	Output (participants)	60%	57%	The slight drop in the percentage of women compared with 2017 is likely a result of our shift away from supporting large lecture courses. In 2017 there were several large course with very high percentages of women students.
Number of articles edited	Output (direct product)	19,200	16,394	Our goal was 19,200, which we missed by about 2,800 articles. We attribute this to a change in programmatic strategy where we stopped supporting new "small projects" as we determined these did not add high enough quality content to Wikipedia to justify our staff time. That meant we pushed participants into spending more time working on one article in depth instead of small contributions to multiple articles.
Amount of content added	Impact (short term)	11.2 million words	13.27 million	Due to the shift away from small projects, our contributors added more content to fewer articles, meaning we significantly exceeded our 11.2 million goal for words added.
Number of articles edited with at least 10 ORES point improvement	Impact (short term)	5,000	4,497	We missed our goal of 5,000 by about 500. See below for a discussion of how we adjusted this metric for next year.
Instructor satisfaction rate (percentage who indicate interest in teaching with Wikipedia again)	Impact (short term)	98%	97%	While we missed our goal by 1%, we still feel this survey result speaks well of our work.
New student learning outcome study completed	Impact (long term)	1	0	We eliminated this goal due to the decrease in funding from the FDC.

What we did

In 2018, we developed a strong focus on the quality of courses we are bringing in to the Student Program. Several terms ago, we developed a formula to assign a numerical "course rating" to each course that participates in our program. The formula takes both quantitative as well as qualitative factors into account. How much content students contribute as well as the quality of that content and the amount of staff time spent on the course all factor into a course's end of term rating. We separate our returning instructors into different buckets, and focused our biggest retention efforts on the courses with the highest ratings. We no longer do active outreach to instructors with low-rated courses. If an instructor with a previously low-rated course wishes to run another Wikipedia assignment, staff discuss whether the course is viable and then facilitate a discussion with the instructor to improve the course's contributions to Wikipedia. This year, we further refined our metrics around course ratings, and we are beginning to see positive results. As part of this quality effort, we also discontinued supporting new courses that made only minor edits to Wikipedia, and we discouraged large multi-section courses from participating at all.

To enable us to more easily make updates to content delivered to instructors and students, we undertook a complex revamp of our course page and training content on our Dashboard software. The outcome is that if we need to make a text change (for example, if we realize some way we've described how to do something is confusing to students), we can make one edit and have it populate to all students now. We also added four new discipline-specific handouts to provide better support to student editors who are editing in topic areas related to science communication, LGBT+ studies, cultural anthropology, and art history.

To better support students and instructors, we also debuted a new Article Finder tool that enables people to locate existing articles within topic areas that might be ripe for improvement, using a combination of Wikipedia's category system, ORES rating, and page views. The tool was developed by Google Summer of Code intern Pratyush Singhal, an indication of how beneficial our open source software interns have been to our programmatic efforts.

As part of our ongoing efforts to create a better program, we also conducted a focus group of instructors at WikiConference North America. Ten of our instructors were attending WikiConference North America, and the organizers were kind enough to find space for us to engage with these faculty and learn from their experiences. Feedback from them and from our end-of-term instructor surveys has helped guide our plans for changes we'd like to make in 2019 to better support our instructors.

In 2018, we also started laying the groundwork for a forthcoming Wikidata program we plan to launch in 2019. As part of our new strategy, we recognize that people are increasingly getting knowledge from personal digital assistants like Alexa or Siri and not from reading long-form encyclopedia articles. Thus, to ensure that our mission of getting representative, accurate, and complete information to people, we are also pursuing programmatic efforts to improve Wikidata content as well. We are exploring a partnership with Wayne State University's School of Information Science to incorporate Wikidata assignments into their library students' curriculum; as part of this work, we led Wiki Education's first Wikidata workshop at Wayne State's campus in December.

What worked well

In 2018, we had our largest impact to date: We supported 811 courses at 366 different universities in the United States and Canada. All told, more than 16,000 student editors added more than 13 million words to Wikipedia — up from our baseline of 11 million words in 2017. The sheer scale that we're operating on is unparalleled in the Wikimedia movement: The fact that students in this program represent 9% of all new editors to all Wikimedia projects in all languages speaks to how important this program is to our movement. We are unaware of any other Wikimedia movement program that brings in the same numbers of new contributors.

This achievement has been many years in the making. We've built up our visibility over time, and in 2018 we reached the milestone of having brought in more new classes from our general visibility work than we did through our targeted outreach. While our targeted outreach is still an important part of our strategy — it enables us to specifically target content areas related to our Communicating Science and Equity initiatives — we don't need to do as much general recruitment in topic areas that aren't of particular strategic importance. That means we were able to reduce the number of academic conferences we attended this year, while still achieving our goals. We count visibility as people who join our program from a recommendation of a colleague, media or blog posts that inspire them, or Googling and finding us. We anticipate attending even fewer conferences in 2019 as the impact from our ongoing visibility work grows.

A key to scalability has been building up our Dashboard software system to provide participants everything they need without interacting with our staff unless they run into a problem. We continued making progress on this in 2018, with an increasing number of instructors finding our website, creating a course page, assigning their students to edit Wikipedia, and empowering the students to add high quality contributions to Wikipedia, without ever interacting one-on-one with our staff. We are scaling the impact of the program without significantly adding staffing resources to it, and succeeding in adding high-quality content to Wikipedia.

What didn't work well

As mentioned in our midterm report, we had to make two programmatic cuts based on lower-than-anticipated revenue: (1) the student learning outcomes research study, and (2) the Guided Editing project. While we still believe both are important projects and we hope to get them funded at some point in the future, we've been left to conclude now is not the right timing to pitch them to funders, who seem to have other funding priorities.

Additionally, we fell short of our goal for the number of articles improved by 10 points on the ORES scale. As we've had a full year of using this metric now, we still believe it has value in capturing, in a rough way, the improvement people make to articles. However, one challenge is that different edits are rated differently. Simply creating a new stub article only big enough to pass through the New Page Patrol gauntlet provides about 20 points; for an example, see the article on Ethiopian poet Nebiy Mikonnen, written by one of our student editors in fall 2018, which was a 21-point improvement. Another student editor in the same course added the same number of words by making multiple minor improvements to the article about Ugandan entertainer and politician Robert Kyagulanyi Ssentamu, but despite the similarities in amount of content added, the second student's work didn't even merit an increase in ORES score on the article. Thus, it's an imperfect measurement — but we still think it's better than any other currently existing measure of article quality in the Wikimedia world, so we are planning to continue using it in 2019.

Connecting it to our strategy

Our strategy has three main pillars: Equity, Quality, and Reach. Our programmatic work in the Student Program connects to each pillar in our strategy.

Equity

Our strategic goal for Equity states: "Increase knowledge equity by focusing on content and communities that are underrepresented on Wikipedia and Wikidata".

In our Student Program, we've addressed the content axis by specifically targeting higher education faculty that teach in underrepresented topic areas, such as race, gender, and sexuality. Articles in underrepresented content areas improved by our students during this year include:

Karankawa people, an expansion of the article on an extinct Native American group written by a student editor in a University of Southern California course on "Borderlands in a Global Context"
Police use of deadly force in the United States, an expansion of an issue of timely interest in the context of racial equity in the United States, written by a student in an Emory University course on "Gender, Race, Inequality - Evidence"
Chanda Prescod-Weinstein, an article on a woman scientist expanded and shepherded through the Good Article process by a student editor from Mount Allison University's "Gender and Science" course
Ann Brewster, a new article on a female cartoonist and illustrator written by a student in a University of Florida course on "American Women in Comics"

All told, student editors improved nearly 2,000 articles in equity content areas in 2018, adding an impressive 1.6 million words to those articles.

Student editors in our program are also more diverse than the general Wikipedia editing population in areas we have demographic data for. While the 2018 Community Engagement Insights Report found 90% of contributors to Wikimedia projects are male, and the median age range of all contributors is 35–44, among our student editor population:

57% identify as women
37% identify as non-white
For 13%, English is their second language
5% are adult non-traditional students; the rest are of typical college age (late teens/early 20s)

Quality

Our strategic goal for Quality states: "Provide people who seek knowledge online with accurate information in topic areas that are underdeveloped".

Quality is one of the hardest elements to measure in the Wikimedia universe. While we are eager to pursue a better method for assessing Wikipedia content quality on a regular basis, we are currently limited in terms of our metrics for quality of articles. Nevertheless, as part of our course close-out process, our Wikipedia Experts on staff review student work that comes from this program and provide a general assessment of quality of content that emerged from each course. We prioritize encouraging high-quality content-producing courses to teach with Wikipedia again. We also use the ORES measure of structural completeness as a stand-in for quality, with the caveats mentioned above. In past years, we have undertaken manual quality assessments and found the article improvements made by the students be significant; while we did not undertake these studies in 2018, we have anecdotal evidence that quality has either stayed the same or increased from past years. We are confident that as our onboarding and monitoring systems continue to get better, we are getting better at bringing high quality content to Wikipedia through this program at scale.

Reach

Our strategic goal for Reach states: "Reach large audiences with free knowledge by making Wikipedia and Wikidata more complete".

In terms of Reach, there are three elements this program tackles. The first is developing our infrastructure through tools like the Article Finder mentioned above, that enable us to scale the impact of our programmatic work. The second is page views; the sheer size this program is operating on means articles improved by Student Program participants get millions of page views each term. While the page views each participant brings on average is only 32,000 (compared to 130,000 per participant in our Wiki Scholars & Scientists Program), the sheer volume of participants makes this program the biggest driver of page views in our programmatic stable. Finally, we have laid the groundwork this year to start a Wikidata program in 2019, with the goal of improving Wikidata content used by personal digital assistants like Alexa or Siri.