Grants:Project/Rapid/Anthere/Mass message impact on WLA
anthere.org
Please see the sample Editathon/Training application before drafting your application.
Project Goal
[edit]Choose one or more of the following goals. You can add or delete goals as needed.
- Invite former contributors to Wiki Loves Africa 2014-2015-2016-2017 to contribute to Wiki Loves Africa 2019 using mass messaging.
- Measure the impact of mass messages through a research testing different experiments
- Draw conclusions to feed future uses of mass messages for Wiki Loves Africa contest and hopefully Wiki Loves X
Project Plan
[edit]Rationale
[edit]Wiki Loves Africa has been successful at engaging new and experienced editors in submitting thousands of new photographic material to Wikimedia Commons, but that is not our only aspiration. We have two additional goals: that new users engaged through Wiki Loves Africa continue to be active editors/uploaders and that those collected images be integrated into Wikimedia’s many languages Wikipedias.
Towards those ends, last year I initiated a mass message campaign to encourage early participants in WLA to add more images and all participants to integrate their photos (and others’) into their language Wikipedias. Two messages were sent, one during the contest to early (2017) participants and one post-contest to all 2017 participants. This campaign born from a strong request from participants to WikiIndaba Ghana, who supported more on-wiki communication with the uploaders during and after the contest and suggested using mass-messaging. Hence... this was implemented and has been documented on Talk:Wiki Loves Africa 2017/Communications elements.
Problem statement...
We do not, however, know to what degree those messages mass-posted on uploaders talk pages were effective at further engaging WLA participants. We also have not tested that strategy on former years WLA participants.
In comparison, 2-3 years ago, Joseph helped us craft a more effective site notice, based on the knowledge collected by the fundraising team. In comparison, doing email campaign using Mailchimp provides valuable information in most effective messages and allows testing. In comparison, last year, we modified the main page of the contest to reduce the number of clicks to upload images, and make it more accessible on mobiles and could see the impact on visits. Etc. But our current use of mass-message is pretty much a shoot in the dark...
Proposition of resolution
This year, I plan to initiate a partnership with CivilServant, a research nonprofit incubated by Global Voices, to test the effectiveness of a new mass-message campaign. CivilServant collaborates with online communities to test tools that make those communities flourish. They are currently working on studies with five language Wikipedias - Arabic, French, German, Persian and Polish - as part of a project developed in consultation with WMF researcher, Aaron Halfaker, and board members Dariusz Jemielniak and Maria Sefidari. In past research, CivilServant has worked (and continues to work) with Twitter and multiple Reddit communities.
Working with CivilServant, we will conduct a randomized control trial to test the effectiveness of our mass message on engagement. CivilServant’s experience combines research design and software engineering. We will collaboratively design the study with their researchers. As part of that collaborative process we will determine the outcomes that matter most, but we anticipate testing the message effect on the overall activation or re-activation of those contacted.
Through this collaboration we hope to learn that our mass messages are effective, and to identify most effective factors. If, however, we learn that it has no or limited impact on participant engagement, we will be able to use that knowledge to focus our efforts on more effective means of engagement in 2020.
Activities for the entire research project
[edit]Tell us how you'll carry out your project. Be sure to answer the following questions:
1. What do you want to tell us about this project?
- The research will be organized with two experiments.
- * a first experiment will test impact on image contribution during the contest by former participants to WLA. We may test different messages content, linguistic approach based on countries etc.
- * a second post-contest experiment, in April, will test the impact on editing contributions during a post-contest activity, Call to action.
- The research will be implemented by myself (Anthere) and Nathan Matias, with undergraduate and graduate students at Princeton University led by J. Nathan Matias where students learn the ethics and craft of of field experiments.
- More information is available on the page documenting this research CivilServant's Wikimedia studies/Wiki Loves Africa Recruitment 2019
2. How will you let your community know about the project? Please paste links below to where relevant communities have been notified of your proposal, and to any other relevant community discussions.
- I will inform the research community (mailing list) and the people on the Wiki Loves Africa Facebook group.
3. Do you have experienced Wikimedia editors to lead the event?
- This is not an event. This is a research project and I am an experienced wikimedia editor.
4. How will you engage participants after the event(s)?
- This is not an event. However, I do plan to involve other people in particular to provide input on the messages sent to contributors and to provide translations.
5. Calendar
- December 2018:
- refine the hypotheses, outcome variables, and plans for the intervention
- submit rough study designs to the Princeton University ethics review board
- January 2019:
- prepare materials for class assignments
- prepare materials for the experiment that recruits previous contributors to participate again
- pre-register the full design of the first study on OSF for transparency purposes
- February 2019 (class begins):
- Feb 1: begin the first study by sending recruitment messages to previous contributors
- connect a student team with this project
- students work with Wiki Loves Africa and the CivilServant team to refine the metadata study design
- March 2019 (Wiki Loves Africa competition ends):
- send debriefing materials to first study message recipients, informing them about the first study, and how public data from Wikipedia is being used
- the debriefing will include a link to public, open access information with full study results, including anonymized data
- students finalize second study design and pre-register the full design of the study on OSF for transparency purposes
- send debriefing materials to first study message recipients, informing them about the first study, and how public data from Wikipedia is being used
- April 2019
- messages for the second study are sent to competition participants
- May 2019
- collect early stage data from the second study for student analysis, if possible
- July 2019
- complete final data collection
- send debriefing materials to second study message recipients informing them about the first study, and how public data from Wikipedia is being used
- the debriefing will include a link to public, open access information with full study results, including anonymized data
- November 2019
- if possible, conduct a followup analysis to see if increases in the creation of metadata contributed to increases in image use across Wikipedia
Impact
[edit]How will you know if the project is successful and you've met your goals? Please include the following targets:
- Written report with results submitted by CivilServant
- Written report to the community by me, with summarized outcome, lessons and recommandations for next contest
- Possibly research article published
Resources
[edit]What resources do you have? Include information on who is organizing the project, what they will do, and if you will receive support from anywhere else (in-kind donations or additional funding).
- The project will involve J. Nathan Matias (User:CS_natematias) CivilServant, Princeton University, User:Anthere, Julia Kamin (user:Juliakamin) CivilServant, Max Klein CivilServant and students from Soc412.
- Most of the participants will get direct support from Princeton University and CivilServant.
- The current grant request is meant to provide individual support to Anthere for her involvement in this endeavor.
What resources do you need? For your funding request, list bullet points for each expense and include a total amount.
- December 2018: 300 US$
- refine the hypotheses, outcome variables, and plans for the intervention
- refine the call to action
- January 2019: 300 US$
- prepare materials for the experiment that recruits previous contributors to participate again
- February 2019: 300 US$
- begin the first study by sending recruitment messages to previous contributors. Work on messages and their translations.
- support Civil Servant in collecting data from the first study
- work with students and the CivilServant team to refine the metadata study design
- March 2019: 300 US$
- send debriefing materials to first study message recipients, informing them about the first study, and how public data from Wikipedia is being used
- work on second study design with the students
- April 2019 300 US$
- messages for the second study are written, translated, and sent to competition participants
- May 2019 200 US$
- support the students and Civil Servant in collecting data from the second study
- July 2019 300 US$
- send debriefing materials to second study message recipients informing them about the first study, and how public data from Wikipedia is being used
- Community report on outcomes and sharing results
Endorsements
[edit]Community members are encouraged to endorse your project request here!