Grants talk:APG/Proposals/2012-2013 round1/Wikimedia Foundation/Impact report form

Thank you for submitting your complete impact report! We look forward to it. Due to the timing of the FDC funding cycle, it will take staff a little longer than usual to offer feedback about this report and post clarifying questions. We appreciate your patience with this process, and welcome any urgent questions or concerns that you may want to address before our comments are ready. Thank you for your attention to the reporting process and best regards, Winifred Olliff (Grants Administrator) talk 02:12, 9 November 2013 (UTC)

Authorship of summary document

This summary summary is written by FDC Staff and is intended for the FDC, FDC grantees, and the Wikimedia movement at large.

Timing and Purpose of WMF’s Impact Report

The Wikimedia Foundation (WMF) submitted its Impact Report to the FDC on time, on October 30, 2013. We appreciate the time and effort WMF devoted to creating this report for the FDC, and thank WMF for submitting a timely and complete report.

The purpose of this Impact Report is to give the FDC and the larger Wikimedia community an idea of how WMF progressed toward its programmatic and financial goals throughout the term of the funds allocation. This report is intended to track progress and document learning for WMF, as well as the wider movement.

Although WMF applied for and received an allocation in 2012-2013 Round 1, its funding period is 1 July 2012 - 30 June 2013; therefore, its funding period ended before the terms of most grants in that funding cycle.

Financial summary

During the funding period, the Wikimedia Foundation spent US$37,216,407, or 88%, of its planned US$42,069,750 budget for the fiscal year 2012-2013. WMF reports they had higher expenditures in capital expenditures, contracted services and travel expenses, as well as unanticipated property taxes on data center equipment. These higher costs were offset by lower staff costs (since several positions went unfilled in 2012-2013), lower costs for internet hosting, and less funding granted through the FDC process than planned.

Highlights

• During the 2012-2013 fiscal year, WMF reports that the projects averaged 80,600 active editors per month, compared to 79,620 in the previous fiscal year. The number of Wikipedia articles increased as well during this period, from 22.63 million to 27.49 million, and added 34 million bytes were added to Wikipedia articles as a direct result of the Global Education Program. Congratulations to WMF and the Wikimedia movement for attracting more than 500 million monthly users.
• The Global Education Program continues to produce content, much of which is rated as quality content. We also note with interest how the Global Education Program may be affecting gender balance on Arabic Wikipedia by targeting female contributors and may be achieving higher than expected levels of retention on Arabic Wikipedia.
• In general, we appreciated that this report includes details about how each of the programs reported on were furthering WMF’s activities related to the gender gap, both through its own programs and its work funding chapters like Wikimedia India and organizations like the Centre for Internet and Society.
• We appreciate the insightful lessons learned about using data and finding ways to make WMF’s programs more scalable. We look forward to the results of these strategies as they develop. We also hope this is something the movement at large may learn from.

We would like to learn more

• WMF reports a surplus in 2012-2013. Is WMF planning differently to achieve a more accurate spending plan for its current or upcoming fiscal years?

Thank you for all of your questions. We are happy to provide the following answers. To successfully support our intended initiatives, strategies and goals, the Wikimedia Foundation prepares the annual spending plan based on various variables and assumptions. We strive to manage the spending as close as possible to the approved spending plan without over-spending. There will always be unanticipated factors that result in under-spending, such as attrition and/or delay in hiring due to an extremely competitive market for engineers. Also, there will be times that the Wikimedia Foundation needs to increase its cash reserve to what it would consider a healthy level (which is currently to cover 12 months spending), as it’s critical in the event of unplanned expenses, emergencies and/or revenue shortfall. So the surplus is intended and built into the plan. JonathanCinSF (talk) 18:33, 24 March 2014 (UTC)

• WMF reports that neither WMF nor the community are satisfied with community engagement, and also that steps have been taken to “improve on how to collaborate with our community of volunteers in 2012-2013.” Would you please describe this progress in more detail, and also provide some insight into future strategy in this area?

The Wikimedia Foundation, beginning with the VisualEditor, tasked a team of employees specifically to supporting the deployment and the necessary community change. These employees, initially short-term contractors, have now been with the Foundation for roughly nine months and the Product team has committed fully to this model, with the creation of a role for the Director of Community Engagement (Product) and subsidiary team. Best practices are emerging and documentation is being assembled to support learning in this area, and to assure institutional memory and knowledge transfer. In addition, the Foundation has begun to assemble lists of editors who can help with similar efforts for the future. We view our progress in this area as exactly that -- progress -- and believe that future efforts will support incremental growth in this area as we learn from each deployment. JonathanCinSF (talk) 18:33, 24 March 2014 (UTC)

• What impact has the Visual Editor had on attracting and retaining new editors so far?

VisualEditor is a new editing system that lets contributors easily format Wikipedia articles while bypassing the intricate mark-up coding that had been the sole editing system since Wikipedia’s beginning in 2001.

We introduced VisualEditor in a prototype form in 2012, and more widely from July 2013 onwards. We believe VisualEditor has already had a positive impact on attracting and retaining new editors, but we have yet to collect overall statistics about this in a sufficiently rigorous way, as the editor is subject to significant and rapid change of function, usability and performance.

That said, based on editing counts that monitor how often VisualEditor is used, we can say that hundreds of edits are made every day using VisualEditor, with uptake varying across wikis. The edits are publicly tagged on wiki (e.g., for the English Wikipedia), and show that VisualEditor is being used to edit the widest possible range of articles, whether it’s removing unsourced information (see here for an example), adding links (see here), resizing images (see here), expanding article sections (see here), adding categories to articles (see here), adding citations (see here), expanding information boxes (see here) — in short, almost everything that contributors have come to expect to do on the movement’s projects.

We continue to work extensively on VisualEditor, expand capabilities to give users more editing options, increase usability and add clarity to existing functions, improve performance, and fix bugs. We are working to bring VisualEditor to mobile editing, so that those who seek the convenience of VisualEditor are rewarded whether they go to the mobile site or the desktop one. We release weekly and monthly progress updates for the community on VisualEditor here. JonathanCinSF (talk) 18:33, 24 March 2014 (UTC)

• Among three objectives for the grants program in 2012-2013, WMF lists funding to the Global South and funding gender gap work. WMF reports that 8% of grants went to the Global South in 2012-2013. We would like to know if this number is satisfactory to WMF, and if not, what WMF has learned from this and what strategies WMF is employing now to increase this number in the current and upcoming fiscal years?

No -- this number is not satisfactory, which is why we increased our Global South spending goal (and spending) for 2013-2014! Progress against these goals is tracked quarterly on meta.

That said, we acknowledge that there is a lot of work to be done in generating communities in the Global South, which is a predecessor to equitable spending in the area. For this reason, we are also supporting the development of Global South communities in nonfinancial ways: for example, through materials, mentorship, and some programs (e.g., Arabic Education Program). We realize that money is simply one resource in developing communities and ideas: while we want to be proactive about supporting communities and expertise in target areas through funding, we also want to be careful not to infuse these communities with inappropriate amounts of money. JonathanCinSF (talk) 18:33, 24 March 2014 (UTC)

• We appreciate the important findings of the E3 experiments. How will these findings impact Wikipedia and its sister projects in the future? Are the results of these experiments yet influencing WMF’s approach to increasing the numbers of editors on these projects, and can we expect to learn more about this in upcoming reports?

This team (now known as the Growth team, formerly E3 experiments) adheres to the following general cycle of product development: during an initial experimentation phase, we test a variety of different hypotheses about how to attract and retain new editors. After this testing and experimentation phase, the team then moves to a "productization" phase, where proven hypotheses are turned in to permanent new features on Wikipedia and Wikimedia sister projects.

The Growth team's projects have included everything from redesigning our account creation forms to improve usability, to creating interactive guided tours that point new editors to articles and show these new Wikipedians how to edit. These features have already become permanent, integral additions to the new user experience on Wikipedia. Today, for example, 80% of all newly registered users on Wikipedia's top 10 language communities by pageviews are delivered editing suggestions and help by Growth team software. (Learn more about the roll-out of these features.)

In the 2013-14 fiscal year, the Growth team reoriented its approach away from measuring success by velocity of experimentation. Now that the team has matured, we are instead focusing on measuring our success by impact alone. Based on projections from previous experiments, we aim to acquire 2,400 additional active editors (five or more edits) every month, adjusting for seasonality. This target is ambitious but not impossible. If achieved, it will represent a significant step toward growth in our total number of active editors, across Wikimedia projects.

The Growth team plans to spend the remainder of its time this fiscal year experimenting in three new areas of the user experience: first, with finding ways to encourage anonymous editors to considering registering; second, on improving the way new users create articles on Wikipedia; and last, by experimenting with personalized task recommendations for active editors that will encourage them to keep contributing. Expect to hear more about the results of experimentation in these areas in future reports. JonathanCinSF (talk) 18:33, 24 March 2014 (UTC)

• WMF reports that Catalyst programs run by Wikimedia Foundation consultants in Brazil and India have been transitioned to local organizations funded through grantmaking activities, and repeatedly emphasizes that 2012-2013 has been a year of transition for the Catalyst Programs. Does WMF have any learning to share with the wider movement about these program transitions?

There are significant learnings around the transitions, and there are many more to come, we are certain. Here are a few:

1. Local, established organizations can be well situated to influence a given region (country, city, etc.): for example, we saw CIS immediately start multiple conversations with partners in the area given their pre-existing networks. Moreover, they are able to leverage their existing blog distribution network for the blog posts about Wikimedia partners. In addition, they were able to use issue experts (like their advisor on Indian languages and educational institutions) in partnership with Wikimedia community members in order to develop language strategies designed specifically (and differently) for each language community.

2. Transparency is important in a transition (particularly in our movement): the Brazil Catalyst’s transition into Ação Educativa was marked by lots of long, open discussions on the Wikimedia Brasil mailing list and wiki, as well as on Meta. As a result of the openness and careful consideration, the ultimate decision was not a surprise and was accepted by the engaged community. We learned how to do this better from our experience with the India catalyst process, where our communities felt they were not being effectively informed or consulted.

3. Outside organizations can augment different areas of expertise: one thing that happened right away in the partnership with CIS was the engagement of different areas of expertise by the now-enlarged broader organization. For example, CIS had technical expertise and thought of quick, efficient ways to build software to visualize the complexity of the Indian language Wikipedias. CIS has also had its own expertise augmented by its engagement with us: they report value in learning from the Wikimedia program implementation in improving their work on policy advocacy for access to knowledge. JonathanCinSF (talk) 18:33, 24 March 2014 (UTC)

Suggestions for future reports

• We encourage WMF to make reporting more rich in content other than text, such as images and videos.
• We would like to see WMF report more about its organizational setbacks and challenges as well as successes. We are encouraging everyone participating in the FDC process to share setbacks and challenges openly so that the entire movement can learn from the challenges of each organization and so the FDC has greater insight into how each organization is considering its challenge.

Audited statement received

We've received your audited financial statement. Thank you for posting it publicly here. Your grant is now complete. Cheers, Winifred Olliff (FDC Support Team) talk 22:03, 16 April 2014 (UTC)

Inaccuracies in India Catalyst program narrative

I quote from Catalyst program section. "During the fiscal year, for example, readership in India increased from 21.9 million to 28.6 million – an increase of 30 percent, according to comScore data, which we use to document our readership numbers. On its own, the Wikimedia Foundation measures readership engagement by other statistics such as pageviews. On Portuguese Wikipedia, pageviews in the fiscal year went from 404 million to 509 million and on Indic-language Wikipedias, pageviews were generally up – as in Hindi, where pageviews went from 6.6 million to 7.3 million, and Malayalam, where pageviews went from 2.7 million to 4.6 million. On Portuguese, active editors averaged 1,598 during the year – a decrease from the 1,606 monthly average from the previous year. Hindi Wikipedia, meanwhile, numbered 49 monthly editors on average, a slight decrease from 54 monthly editors during the previous year, while Malayalam Wikipedia jumped from 80 average editors in the 2011–2012 fiscal year to 104 this past year, and Bengali Wikipedia went from a monthly average of 52 to 59."

The above narrative with respect to Indic languages does not provide a true picture. The Catalyst program was focused on English language contributions and to a small extent on Assamese. Subsequent CIS-A2K program focused mainly on Telugu, Kannada and Odia languages. The above narrative quotes numbers from Hindi, Malayalam, which are not appropriate. It is also important to note that Unique visitors count reduced to 24.92 Million (Feb 2014) for India subsequent to the period of report, despite CIS-A2K's work on the program.--Arjunaraoc (talk) 05:58, 28 April 2014 (UTC)