Research:Slow Editing Towards Equity
This project's data is available for download and reuse.
Supported by Wikimedia Research Fund
This page documents a research project in progress.
Information may be incomplete and change as the project progresses.
Please contact the project lead before formally citing or reusing results from this page.
Project Outline
[edit]Researching Wikipedian policy development and knowledge equity. This project aims to identify the skills needed by Wikipedians to move equity initiatives through the policy development process. The grantees will evaluate the paths of equity initiatives through the policy development process in five Wikipedia language editions (Arabic, Dutch, English, French, and Spanish). We will collect and analyze both successful and failed proposals that sought to increase knowledge equity. This work has the potential to increase understanding of Wikipedia’s policy landscape and how existing practices influence the likelihood of a proposal’s success. The project will produce both academic publications as well as a field guide to policy change.
Introduction
[edit]Goals & Intended Impacts
[edit]The intended goal of this research is to provide a set of best practices, skills, and role descriptions that Wikipedians can mobilize in order to enact policy changes. This will be achieved by using mixed methods (content analysis, qualitative textual analysis, and interface analysis) as part of two research strategies.
- Research strategy 1: Analyze the moments when Wikipedians successfully and unsuccessfully garnered the broader acceptance of the community; such as by moving an essay or project page into a guideline or official policy within different language editions.
- Research strategy 2: Collect and analyze both successful and failed policy proposals that were designed to increase knowledge equity.
The major impact of the work will be in increasing the knowledge of how Wikipedia’s policy environment is developed and how these practices have shaped the success and failure of equity-based policy proposals. In doing so, policy reform initiatives will benefit by learning from this history and can plan their activities and mentorship accordingly. Furthermore, this research will be applicable to multiple languages of Wikipedia and therefore adds to the capacity of the re- search to address the diverse needs of different Wikipedian communities. As such, it is expected that policy workshops and taskforces will use this information to train Wikipedians and new users on how to develop and maintain equitably policies.
Initial Research Questions
[edit]- RQ1. What kinds of skills and roles have Wikipedians used to move a rule from an initial project page (or essay) to established policy?
- RQ2. What are the markers of successful policy proposals and the barriers for unsuccessful policy proposals dedicated to increasing knowledge equity?
Literature Review
[edit]Over the past decade, there has been a significant and concerted effort to address both Wikipedia’s knowledge equity and the relative stagnation in increasing and retaining editors. This has manifested in article creation initiatives like Art+Feminism and Whose Knowledge?. However, knowledge equity is not just a question of content — it is also about process. As Amanda Menking and Jon Rosenberg argued,[1] Wikipedia’s policy environment reinforces problematic norms about knowledge processes. Similar sentiments have been levelled against specific guidelines and policies dealing with reliable sources,[2] and notability.[3][4] As such there is a need to reconsider the foundational assumptions about knowledge that are being manifested in these community-defining documents. Given this situation, the research project proposes that in order for the 2030 Wikimedia Strategic Direction[5] on knowledge equity to succeed, these concerns about Wikipedia’s policies must be addressed.
To understand the methods chosen for this research, it is necessary to first understand how the research object will be theorized. In this particular instance, policies and policy-making will be conceptualized according to legal scholar Cornelia Vismann’s position that legal media exist as “cultural techniques”.[6][7] This means that the “making” of policy is not simply the translation of a wiki-based discussion into a document. Instead, policy development will be understood as the combination of actions and tools that connect logics (how it is argued into being on the talk page), inscription and storage (the changing composition of the policy page), and its circulation (practices of hyperlinking to policy documents). This theoretical approach falls in line with previous research about Wikipedian governance that is sensitive to the entwined relationship between social practices and technical operations.[8][9] Consequently, this socio-technical approach requires a unique set of mixed methods. In particular, one that combines the quantitative benefits of content analysis, the interpretative value of textual analysis, and the media-sensitivity of interface analysis, methods that have been used in other research projects.[10][11][12] The synthesis of these methods will provide the means for answering the following research questions.
Methods & Data (RQ1)
[edit]Any answer to RQ1 requires conceptualizing what it means for creating policies that are accepted by the community. From a theoretical perspective, this means analyzing how groups establish identities through meaning. Based on this goal, we used the method of discourse analysis which incorporates qualitative content analysis. Data was collected from five of Wikipedia's fifteen largest language editions (Arabic, Dutch, English, French, and Spanish) and then a purposive sampling (according to five dimensions) of three policies from each edition. The edits histories of these pages were collected using the Digital Methods Initiative's Wikipedia Edit Scraper. In total, we collected 8040 edits that ranged from February 2005 to April 2023.
We then proceeded through several passes of the edit history of each rule and identified the moments when the status of a rule was identified by using a hatnote template or hatnote HTML. We then built a timeline of these edits and assessed which technical skills was employed (templating, editing, or reverting) and categorized these according to theoretically defined categories of authority.
Sample of Rule Pages
[edit]| Rule language | Rule Name | English translation |
|---|---|---|
| Arabic | ar:ويكيبيديا:الاستشهاد بمصادر | Citing Sources |
| Arabic | ar:ويكيبيديا:دليل الأسلوب/تنظيم المقالة | Style Guide/Article Organization |
| Arabic | ar:ويكيبيديا:صفحات المستخدمين | Users’ Page |
| Dutch | nl:Wikipedia:Wees Duidelijk en Concreet | Be Clear and Concrete |
| Dutch | nl:Wikipedia:Balans | Balance |
| Dutch | nl:Wikipedia:Gebruikersnaam | Criteria for Usernames |
| English | en:Wikipedia:Reliable Sources | Reliable Sources |
| English | en:Wikipedia:Proposed Deletion | Proposed Deletion |
| English | en:Wikipedia:Disruptive Editing | Disruptive Editing |
| French | fr:Wikipédia:Commentez vos Modifications | Comment your Edits |
| French | fr:Wikipédia:Notoriété des Fanzines | Notability of Fanzines |
| French | fr:Wikipédia:Transcription_du_chinois | Transcription of Chinese |
| Spanish | es:Wikipedia:Conflicto de Interés | Conflict of Interest |
| Spanish | es:Wikipedia:Consultas de Borrado | Deletion Policy |
| Spanish | es:Wikipedia:Fuentes Fiables | Reliable Sources |
Results
[edit]Wikipedians across the five languages use a variety of different identifiers to denote the status of a rule.
[edit]This is of course not surprising. But what is particular is the fact that in the rules we read, each language focused on different kinds of status and they are used with different frequencies. This means that there are important distinctions between language editions in terms of how they view the meaning of rules that are "official," "policies," "guidelines" or "essays."
The status of the examined Wikipedian rules are established early.
[edit]This means two things. First, that Wikipedians tended to designate the long-term status of a rule within the first year, although often within the first three months of that year. Second, most of the statuses of these important rules were attached before 2011.
The status of rules are established largely by individual decisions and rarely through collective decision making.
[edit]Depending on the language edition, Wikipedians used a variety of practices such as edit warring, talk page discussion, surveys, in the process of establishing the final status of a rule. However, in only a few cases (such as in the Spanish language rules) was the decision an explicit representation of the agreement of the editors involved. Otherwise, most rules were designated as rules based on the authority of a single editor.
Conclusion
[edit]Of the fifteen rules studied, the status of most rules technically represented the decision of an individual and not a collective decision. The explanation for this was that the use of policy hatnotes imbued a technocratic authority that allowed individual decisions to appear as collective ones. This suggests that Wikipedians invested in policy-making engage in technically-defined acts of authority that shape what counts as community consensus. However, this research also points out a particular way for developing policy-making that can involve consensus, but that certain activities -- such as designating the authority of a rule -- can be produced through collective decision-making mechanisms so that it actually represents the general will of the community.
Project Activities
[edit]Publications & Conferences
[edit]- Jankowski, S., Bueno, C. C., Kemper, J., and Sabbah, O. (2024). From Idea to Consensus: A Cross-Linguistic Study of Wikipedian Policy Development. ICA 2024. Australia.
- Jankowski, S., Bueno, C. C., Kemper, J., and Sabbah, O. (2024). Changing Wikipedia Policy: Skills and Roles in a Multilingual Environment. In Wiki Workshop 2024.
- Jankowski, S. (2024, June 19). Agreeable data: How Wikipedian consensus is conceptualized by computer science researchers. Wikipedia is/as data, Gold Coast, Austrailia.
- Jankowski, S., Bueno, C. C., Kemper, J., and Sabbah, O. (2023). Wiki Workshop 2023: Global Platform Governance: Multilingual Policy Development on Wikipedia
Data sets
[edit]- Jankowski, S.; Kemper, J.; Celis Bueno, C.; Sabbah, O. (2024). Wikipedia Policy Status Moments in Five Language Editions. University of Amsterdam / Amsterdam University of Applied Sciences. Dataset. https://doi.org/10.21942/uva.27377457.v1
Monthly Update
[edit]- 2024-11: Added the DOI for the CC0 Dataset.
- 2024-09: Revised manuscript sent to New Media & Society.
- 2024-09: Received team revisions to manuscript.
- 2024-07: Initial revisions made to manuscript.
- 2024-07: Official end of research project.
- 2024-06: Revisions received for first paper.
- 2024-06: Presented research at the Wikihistories 2024 Conference in Brisbane, Australia.
- 2024-06: Presented research at the Wikiworkshop 2024 Conference.
- 2024-06: Presented research at the International Communications Association 2024, Gold Coast, Australia.
- 2024-04: Manuscript for first paper submitted to the journal New Media & Society.
- 2024-04: Research team meetings to discuss draft 2.
- 2024-03: Second Draft of first paper based on Research Strategy 1 complete.
- 2024-03: Research team meetings to discuss draft 1.
- 2024-03: Team revisions of paper completed.
- 2024-03: Abstract accepted to ICA pre-conference.
- 2024-02: First Draft of first paper based on Research Strategy 1 complete.
- 2024-02: Research team meetings to discuss draft 1.
- 2024-02: Abstract submitted to ICA pre-conference.
- 2024-01: Abstract accepted the annual International Communication Association conference.
- 2024-01: Began drafting first paper based on Research Strategy 1.
- 2023-11: Submitted abstract based on Research Strategy 1 to the annual International Communication Association (ICA) conference.
- 2023-05: Presented at * Wiki Workshop 2023 with the presentation Global Platform Governance: Multilingual Policy Development on * Wikipedia
- 2023-04: Abstract accepted and presentation prepared for Wiki Workshop 2023.
- 2023-04: Began initial analysis of data.
- 2023-03: Finalized the research project's logo.
- 2023-03: Research team meeting concerning the status of Research Strategy 1.
- 2023-03: Finished coding of the policy talk pages.
- 2023-02: February 2023
- 2023-02: Second pass of the Research Strategy 1 coding scheme for talk pages through cross-coding.
- 2023-01: Tagging of 3 policies from each of the five languages was completed.
- 2023-01: First pass of coding scheme for the talk pages analyzed under Research Strategy 1.
- 2022-12: Second pass of the Research Strategy 1 coding scheme of history pages through cross-coding.
- 2022-12: Tagging of 3 policies from each of the five languages being studied begun.
- 2022-11: Datasprint for data collection for the Arabic, English, Dutch, French, and Spanish versions. Each researcher began a * pilot analysis of three policies relevant to the research.
- 2022-11: First pass of the Research Strategy 1 coding scheme completed.
- 2022-11: Data collection for Research Strategy 2 (English) begun.
- 2022-10: Meeting with fiscal sponsor to coordinate budget
- 2022-10: Submitted Abstract for the International Communication Association Annual Conference (Rejected, January 2023)
- 2022-09: Two post-doctoral researchers and one research assistant formally hired
- 2022-09: First meeting with research team
- 2022-09: Data collection started for Research Strategy 1 (English)
- 2022-09: Development of social media assets for research communications
- 2022-07: Administrative meetings with fiscal sponsor (University of Amsterdam) and WMF to process funding.
- 2022-06: Slow editing towards equity research project proposal accepted.
See Also
[edit]References
[edit]- ↑ Menking, Amanda; Rosenberg, Jon (2020). "WP: NOT, WP: NPOV, and Other Stories Wikipedia Tells Us: A Feminist Critique of Wikipedia’s Epistemology.". Science, Technology, & Human Values: 1–25.
- ↑ Berson, Amber; Sengul-Jones, Monika; Tamani, Melissa (2013). "Unreliable Guidelines: Reliable Sources and Marginalized Communities in French, English, and Spanish Wikipedias". Art+Feminism. Retrieved 2022-09-26.
- ↑ Gauthier, Maude; Sawchuk, Kim (2017). "Not notable enough: feminism and expertise in Wikipedia". Communication and critical/cultural studies (14): 385––402.
- ↑ Tripodi, Francesca (2021). "Ms. Categorized: Gender, notability, and inequality on Wikipedia". New Media & Society: 1–21.
- ↑ Maher, Katherine (2017-08-10). "Wikimedia 2030: A draft strategic direction for our movement". Diff.Wikimedia. Retrieved 2022-09-26.
- ↑ Vismann, Cornelia (2008). Files: Law and media technology. Stanford University Press.
- ↑ Vismann, Cornelia (2013). "Cultural Techniques and Sovereignty". Theory, Culture & Society (30): 83–93.
- ↑ Geiger, Stuart (2017). "Beyond opening up the black box: Investigating the role of algorithmic systems in Wikipedian organizational culture.". Big Data & Society: 1–14.
- ↑ Ford, Heather; Wajcman, Judy (2017). "`Anyone can edit’, not everyone does: Wikipedia’s infrastructure and the gender gap". Social Studies of Science: 1–17.
- ↑ Jankowski, Steve (2022). "Making Consensus Sensible: The Transition of a Democratic Ideal into Wikipedia’s Interface". Journal of Peer Production (15). Retrieved September 26, 2022.
- ↑ Jankowski, Steve. "The Trouble with Knowing: Wikipedian consensus and the political design of encyclopedic media". York University. Retrieved September 26, 2022.
- ↑ Jankowski, Steve (2015). "No consensus on consensus: A paradox within Wikipedian governance and collective action". In Torres, C.; Mateus, S. From Multitude to Crowds- Collective Action and the Media. Peter Lang. pp. 177–196.