Grants talk:Project/nschwitter/The Role of Offline Ties of Wikipedians

Eligibility provisionally confirmed, Round 2 2021 - Research and Software proposal

This Project Grants proposal is under review!

We've provisionally confirmed your proposal is eligible for review in Round 2 2021 for Research and Software projects, contingent upon:

• confirmation that the project will not depend on staff from the Wikimedia Foundation for code review, integration or other technical support during or after the project, unless those staff are part of the Project Team.
• compliance with our COVID-19 guidelines.

Schedule delay

Please note that due to unexpected delays in the review process, committee scoring will take place from April 17 through May 2, instead of April 9-24, as originally planned.

• Please watch your talkpage, which will be the primary method of communication about your proposal. We appreciate your timely response to questions and comments posted there.
• Please refrain from making changes to your proposal during the scoring period, so that all committee members score the same version of your proposal.
• After the scoring period ends, you are welcome to make further changes to your proposal in response to committee comments.

COVID-19 planning for travel and/or offline events

Proposals that include travel and/or offline events must ensure that all of the following are true:

• You must review and can comply with the guidelines linked above.
• If necessary because of COVID-19 safety risks, you must be able to complete the core components of your proposed work plan _without_ offline events or travel.
• You must be able to postpone any planned offline events or travel until the Wikimedia Foundation’s guidelines allow for them, without significant harm to the goals of your project.
• You must include a COVID-19 planning section in your activities plan. In this section, you should provide a brief summary of how your project plan will meet COVID-19 guidelines, and how it would impact your project if travel and offline events prove unfeasible throughout the entire life of your project.

Community engagement

We encourage you to make sure that stakeholders, volunteers, and/or communities impacted by your proposed project are aware of your proposal and invite them to give feedback on your talkpage. This is a great way to make sure that you are meeting the needs of the people you plan to work with and it can help you improve your project.

• If you are applying for funds in a region where there is a Wikimedia Affiliate working, we encourage you to let them know about your project, too.
• If you are a Wikimedia Affiliate applying for a Project Grant: A special reminder that our guidelines and criteria require you to announce your Project Grant requests on your official user group page on Meta and a local language forum that is recognized by your group, to allow adequate space for objections and support to be voiced).

We look forward to engaging with you in this Round!

Questions? Contact us at projectgrants wikimedia  · org.

Marti (WMF) (talk) 05:37, 17 April 2021 (UTC)

Aggregated feedback from the committee for The Role of Offline Ties of Wikipedians

Scoring rubric	Score
(A) Impact potential Does it have the potential to increase gender diversity in Wikimedia projects, either in terms of content, contributors, or both? Does it have the potential for online impact? Can it be sustained, scaled, or adapted elsewhere after the grant ends?	6.8
(B) Community engagement Does it have a specific target community and plan to engage it often? Does it have community support?	7.2
(C) Ability to execute Can the scope be accomplished in the proposed timeframe? Is the budget realistic/efficient ? Do the participants have the necessary skills/experience?	6.0
(D) Measures of success Are there both quantitative and qualitative measures of success? Are they realistic? Can they be measured?	5.4
Additional comments from the Committee: Interesting research. The general question is interesting, as it connects to the social infrastructure of knowledge production, thus bearing directly to our priority pertaining to Wikimedia as a service infrastructure. Research is part of an ongoing graduate study, thus has an appropriate setting. The projects fits wikimedia strategic goal. The research can be scaled, sustained or adopted elsewhere because it looks at how offline activities have effect on online events. It fits with Wikimedia's strategic priorities and can be sustained. The effects of offline and online meetups should be an area we should pay attention to in these difficult times. We need research ideas like this. This proposal has realistic measures of success and clear targets for evaluating impact and capturing learning, as it is rooted on a set of well defined hypotheses. The idea and hypothesis are interesting and I have seen such research before though the subject is still important: we pay much attention to offline events but do they really matter? The project takes innovative approach to solving key problems. The potential impact of the project is greater than risk. The project has an evaluation plan that measures the outcomes of the project This research is innovative, and its potential impact will be great. The plan to execute this project is well explained. This will definitely create a very long impact on the German Wikipedia community. not an active wiki-member and budget seems excessive for me: there was no information about hours required, hourly rates. Comparison to stipends given seem not very relevant to me, I compare it more to other projects where such an amount provides like 20hours/week of intensive work for a extended period of time Tthe project can be completed in 12 months. The budget is realistic enough based on activities it would be used for. The participants has the skill sets and also the participants would involve experience wikimedians to support in the projects. This project can be completed in 12 months, but $29300 research sponsorship is a bit too high. I will suggest the budget be reduced to $26,000 considering the project outline and duration. The project proposer is very experienced and can deliver. The project has a specific target community (the German community) of which the project lead plans to engage. It has community support with some experienced wikipedians. The project involves diverse groups like students, wikipedia and some lectures. This project is good for the German Wikipedia and I believe it will help other Wikipedias as well. There seems to be a good community engagement as some students would be involved. Interesting research. This is an interesting project, based on an ongoing research track in Sociology from the proponent. I do have a couple of questions: Could you please provide more information on the network theory you are relying on? Is this coming from a Granovetter-like analysis, a social-capital network analysis? Will there be a control group of Wikimedians that have not been part of meetups? If so, how will it be selected? If no, why not? Are you relying on any typology of meetups? It might be useful to identify --for specific hypotheses-- relevant characteristics to improve the explanatory variable. (* More information on the methodology is needed, as I would assume any inference would be dependent on the number of cases, and this might be a rather small n. How will you control for variations that are related to a possible small n? Could you provide more details on how you will proactively connect with the community to discuss/share your research results? I would have supported when: the grantee is an experienced wiki-member, not just an outsider The project has WMF community organising team or at least WMDE - we don't need just a research, we need to implement conclusions afterwards. If it is understood that events should be changed in some way, we need to do it (and we need a big chapter to be able to test it and share results), otherwise it won't benefit us a movement and would be just one more research I would award the grant to this project and I believe funding every component of the project would be of essence. The reason is that the project is feasible and this would help the community understand how offline events have affected online activities of which possible solution would be provided to make engagement better. By doing the community would be much informed on the kind of event they organize offline and online. I will support this project, but the budget should probably be reduced to $26,000. Apart from that, everything looks pretty clear, and the proposer seems to be very experienced.

This proposal has been recommended for due diligence review.

The Project Grants Committee has conducted a preliminary assessment of your proposal and recommended it for due diligence review. This means that a majority of the committee reviewers favorably assessed this proposal and have requested further investigation by Wikimedia Foundation staff.

Next steps:

1. Aggregated committee comments from the committee are posted above. Note that these comments may vary, or even contradict each other, since they reflect the conclusions of multiple individual committee members who independently reviewed this proposal. We recommend that you review all the feedback and post any responses, clarifications or questions on this talk page.
2. Following due diligence review, a final funding decision will be announced on Thursday, May 27, 2021.

Questions? Contact us at projectgrants wikimedia  · org.

Marti (WMF) (talk) 05:33, 5 May 2021 (UTC)

Response to the feedback

Dear Project Grants Committee,

I want to thank you all for your favourable feedback. I appreciate you taking the time to read my proposal and to comment on it. Many valuable points were raised in your comments. I want to make some responses and clarifications to the questions raised.

• Experience with Wikipedia: While I have to agree that I am not a highly experienced editor of Wikipedia, I would also not consider myself an outsider. After taking part myself in an offline editathon organised by my University’s library, I have finally registered an account (after making minor edits as an IP before) and started to also write some articles. I also think it might come as an advantage that I am not deeply embedded into the world of Wikipedians as it assures a neutral and objective point of view. In any case, I fully support the movement of open, free and accessible knowledge and collaboration. While I am not a heavy editor of Wikipedia, I have contributed to other open-source projects (e.g. written software packages for R).
• Not being affiliated with WMF/WMDE: I absolutely agree with the statement that conclusions should be implemented afterwards and that this is something I cannot provide. However, I think my project is a valuable first step. Before changes to e.g. the organisation of meetups are implemented, it is necessary to analysed whether meetups are a relevant factor in the first place. This is the gap my research project will address. At the end, I will be happy to work on and discuss concrete recommendations on the basis of my project and would be more than willing to collaborate with WMF/WMDE.
• Engagement with the community: As outlined in the proposal, I plan to publish the results in the Wikimedia community forum Der Kurier. I’d also be very happy to discuss the results at an actual offline meetup (once they resume) for a more local discussion. Besides this, I have attended the WikiWorkshops 2020 and 2021 as a participant and will aim to submit an article myself in future years. Besides directly connecting with the community, I also want to highlight that I have presented and will present my research in other contexts (at conferences, in seminars to students, in research seminars) to connect it to wider audiences.
• Budgeting: My budgeting is based on the ESRC studentship as I found it difficult to assign a clear workload in hours and calculate an hourly rate. I will work fulltime on this project for the next year. The ESRC studentships covers fees and living expenses as a PhD student at an English university while assuming fulltime dedication to the PhD. As the ESRC also assigns $1700 money to academic/personal development, I can agree with a reduction of the budget.

I have also received some more methodological questions which I would like to answer. Thanks for these!

• Network theory: I will combine different network theoretical approaches, and test the usefulness of different concepts. Fundamentally, I define the network as a social structure made up of nodes and being in relation with dyadic ties. In my view, e.g. Granovetter’s hypothesis on the strength of ties can be well expressed in terms of social capital, in that strong and weak ties hold different forms of social capital. This relates well to the distinction of bridging and bonding social capital. Overall, it will be an empirical question to test which hypotheses hold true.
• Control group: Making claims about the effects of meetups requires a comparison with users that have not attended any. I will work with the complete collection of all Wikipedians, so this includes both users that attended meetups and those that did not. I currently plan to work with the full list of users and to statistically control for tenure and activity in analyses (so that I can e.g. reliably say that it is the attendance at a meetup that makes people contribute more and not for example only the number of previous edits). Another option would be to use a matching approach. With a matching approach, one would match each user that attended a meetup with a user that did not attend one but is comparable in other regards. For example, imagine user A and B which registered on the same date and had a comparable level of activity in their first month. In their second month, user A attended a meetup while user B did not. User A and B could now be matched and compared. This is the very basic idea – in my case, it would be scaled up to all users that attended a meetup and not only one and I would also use a more advanced version than this 1:1-matching. At this moment in time, I do not yet know which strategy of analysis is ideal, but these are the ideas I am currently following up upon.
• Typology of meetups: I currently distinguish between “work meetings” and “social meetings” with work meetings covering any meetups which were clearly designed to explicitly work on Wikipedia (e.g. editing events). I also separate all meetups that have taken place or were organised by community spaces (which are “official” spaces and directly supported by the WMDE such as the WikiBär in Berlin). I have not yet thought of any further typology of meetups. However, this is an interesting point that I will keep in mind, thanks!
• “Small n”: With the German Wikipedia being one of the largest language versions and featuring a very rich meetup culture, I have never faced the problem of a small n in any of my preliminary analysis. I am able to analyse over 4000 meetups (with 1-300 attendees per meetup). Only if I focus on specific subgroups, for example specific regions, I might end up with a smaller number of observations where it is not possible to draw inferences. In these cases, I am still convinced something can be learnt from rich descriptive analysis which do not require large numbers to be insightful but can instead highlight local peculiarities.

I think this should cover all of the methodological remarks. If there are any more specific questions (methodological or otherwise), I am happy to answer and outline my thoughts.

I want to thank you again for considering my proposal and for your time. --ASociologist (talk) 09:11, 7 May 2021 (UTC)

Response to expert feedback

After a very helpful conversation with Mercedes Caso, I have made some changes to my proposal to address both the feedback of the grant committee and experts. In short:

• Budget: I have reduced the budget to $26000 and provided a more detailed timeline of my activities for the year which also makes the allocation of the money more transparent.
• Community engagement: I have extended the section and outlined some more ways to actively engage with the community.
• Scalability: I have extended my project goals to include the write-up and documentation of the data collection and the analyses conducted in the fashion of hands-on guidelines.

I want to add a few further remarks on the topic of scalability. It definitely is possible to extend the analysis beyond the German Wikipedia but it is out of scope for this project. Just the collection of all meetups in the German Wikipedia was a fulltime job for over 6 months and also required the language skills to read and understand the organisational pages. I absolutely agree that comparing the meetup culture of different language versions would be extremely valuable (and a potential follow-up project or I would need research assistance in the present project ). As a step in this direction, I now plan to write up the process of data collection and analysis to allow other researchers to apply and reproduce the analyses in other contexts. I have previous experience with writing guidelines for data collection and analysis projects as I have written a guideline on Twitter data collection for Oxfam which was received very well.

Other more methodological questions have also been raised which I want to reply to here. I have discussed how and that I plan to include all editors in the analysis to gain a full picture in the previous response. Another question was raised about the quality of the data. This is a good question. During data collection, I have aimed at collecting every offline meetup that has taken place in the German Wikipedia. To achieve that, I have checked all regional portals as they are listed on the German meetup page (the current version as well as different versions from the past that I accessed through the version history), I have covered all portals and “Redaktionen” to search for meetings, I checked the meeting calendar and any other sort of list that mentioned offline meetups. To the best of my knowledge, I have covered all documented German meetups.

Also, I have used the best available information, but I cannot fully guarantee its accuracy. However, as most meetups provide minutes about what was done/discussed at a meetup or provide pictures of the meeting, and considering the fact that Wikipedians tend to report back and stick to the facts, I think the data is reliable. Yet, when preprocessing the data, there is no way to assess whether I have missed any meetings or whether the information that was provided is inaccurate: Even if there were any peculiar patterns in the data (like a sudden stop of meetings, no meetings happening in a certain month, etc.), these peculiar patterns often just happened. As long as any missing data is missing at random and not in a systematic way, results will not be biased.

The project does not involve the collection of any socio-demographics, at least not on a large or quantitative scale. The reason is that these data are not available (e.g. not all users provide full names which would allow to create a gender variable); a survey would be needed to collect them reliably and this is beyond the scope of this project. However, during data collection, I have realised how exclusive meetups tend to be as the reluctance for certain groups to join is very clearly expressed on the organisational pages. A more in-depth analysis of those texts as well as the pictures from the meetups would allow an assessment of the skewed gender distribution at meetups (in the minutes of meetings, the skewed gender distribution is often mentioned directly). The quantitative research will be informative to assess the importance of offline meetups generally which will allow to draw conclusions on the consequences of those skewed meetups. Besides gender, the role of meetings for newcomers can be more directly assessed as the date of the first edit is more readily available than information on gender.

Once again, I want to thank you all for considering my proposal for funding and for your time and your valuable input, it is extremely interesting to see your points of view and I appreciate it greatly.

--ASociologist (talk) 10:01, 13 May 2021 (UTC)

Round 2 2021 decision

Congratulations! Your proposal has been selected for a Project Grant.

The committee has recommended this proposal and WMF has approved funding for the full amount of your request, $26,000

Comments regarding this decision:
The committee is pleased to support this research that promises to provide helpful insight into the efficacy of meetups, a standard offline organizing tool in the Wikimedia movement, including how important it is to editor retention and online engagement. They also found a value in the creation of a hands-on set of guidelines, as one of the project's deliverables, that will allow to transfer the learnings to movement organizers to help them focus on evidence-based strategies in how they design their meetups.

NOTE: Funding of any offline activities (e.g. travel and in-person events) is contingent upon compliance with the Wikimedia Foundation's COVID-19 guidelines. We require that you complete the Risk Assessment Tool:

• 14 days before any travel and/or gathering event
• 24 hours before any travel and/or gathering event

Offline events may only proceed if the tool results continue to be green or yellow.

Next steps:

1. You will be contacted to sign a grant agreement and setup a monthly check-in schedule.
2. Review the information for grantees.
3. Use the new buttons on your original proposal to create your project pages.
4. Start work on your project!

Upcoming changes to Wikimedia Foundation Grants

Over the last year, the Wikimedia Foundation has been undergoing a community consultation process to launch a new grants strategy. Our proposed programs are posted on Meta here: Grants Strategy Relaunch 2020-2021. If you have suggestions about how we can improve our programs in the future, you can find information about how to give feedback here: Get involved. We are also currently seeking candidates to serve on regional grants committees and we'd appreciate it if you could help us spread the word to strong candidates--you can find out more here. We will launch our new programs in July 2021. If you are interested in submitting future proposals for funding, stay tuned to learn more about our future programs.

Marti (WMF) (talk) 04:52, 28 May 2021 (UTC)

Access to the thesis

Hi @ASociologist. Thanks for doing this fantasting work. I'm Aafi, founder of the Deoband Community Wikimedia, a recognised user-group affiliate. We are determining methods for the strategic development of the affiliate. The proposal is available here. Since your thesis is relatable to the proposal and can help us learn from the learnings that you learnt throughout the course of this work, could you please share the thesis over? The best place to send the thesis would be dcwwikimedia.org. Thanks, ─ The Aafī ^(talk) 09:06, 28 January 2023 (UTC)

Hi @TheAafi, absolutely! I sent you the thesis. Feel free to get in touch if you have any further questions! ASociologist (talk) 10:05, 30 January 2023 (UTC)

@ASociologist: Received. Thanks and regards. ─ The Aafī ^(talk) 12:07, 30 January 2023 (UTC)