Recommendation # 2: Content Diversity Metrics and Guidelines

Q 1 What is your Recommendation?

That all communities and stakeholders be able to access and receive statistics, tools and guidelines on the current diversity of content.

This would guide and help them cover the unrepresented concepts and points of view, whether they are related to gender, countries, LGBT+, culture, historically marginalized communities, among others.

Q 2 What assumptions are you making about the future context that led you to make this Recommendation?

Prior to analyze content it is necessary to define diversity types. One comment from the community feedback said: “Before considering strategy of diversity, scope of diversity should be very well defined. It is not the case now.” There are numerous types of diversity that should be present in Wikimedia content. Some of them are: Gender, Geographical, LGTB+, Cultural, Indigenous people, Religion groups and Ethnic groups.

We assume it is possible to locate the concepts/points of view that represent them so that we can assess the gaps and coverage of these 1) at concept level, 2) at point of view level in all Wikimedia projects. We want to avoid marginalization in content from any specific groups by both detecting (1) the level of representation of specific concepts in each language edition and, when this knowledge is local to a language, 2) the level of sharing across the rest of language editions.

The complementation between events and tools that provide stats and guidelines is the best strategy. We are seeking content empowerment. This is related to the scoping question: “How do we increase awareness in low awareness regions, in order to ensure adequate representation, both in the level of volunteer participation and amount of content?”. What gets measured, gets improved.

This recommendation is in line with the project Cultural Diversity Observatory. Therefore it could rely on its framework and code architecture. This could expand the project and even take out the ‘cultural’ part to work on these other types of diversity.

The research part of the project is mainly dedicated to article gaps - by creating lists of articles and statistics. But other research projects like Omnipedia or Manypedia have been focused on showing the points of view in common between articles. Any project that highlight diversity in content should be incorporated to the diversity stats.

Q 3a What will change because of the Recommendation?

Hopefully the pursuit for more diversity in content will become more and more transversal to any activity once editors are aware of the gaps and provided with lists of articles for every underrepresented group and on any topic (e.g. GLAM, folk, monuments and buildings, etc.).
At the same time, this may increase in higher quality of content. Articles reflecting more points of view reflect better neutrality and Wikimedia platforms including more conceptual diversity better reflect the sum of human knowledge.
The measurements can be a reference point to turn to in case of deletion conflict when considering an article is not notable enough.
The classic notion of an encyclopaedia and ‘universal knowledge’ needs to be discarded. Having top priority content about any group of people, nation,... is in this direction. The idea of encyclopedic knowledge feels problematic. What is a “universal knowledge”? Who gets to decide what is “universal”? We need to focus on moving from a single center to multiple ones.

Q 3b How does Recommendation relate to the current structural reality? Does it keep something, change something, stop something, or add something new?

It presents a unified project to continue the task of raising awareness in content diversity started with gender gap and cultural diversity
It connects the different activities, research, and tools better.
It eliminates the western points of view on what is universal knowledge.
It really vindicates the idea that the sum of human knowledge must encompass the knowledge of all the human groups.

It relates to the three thematic area goals:

to map the current areas of diversity and the diversity gaps within the movement as well as projects that have been concentrating on bridging these gaps;
finding ways to increase awareness of privilege and to overcome related cultural, institutional, technological, and behavioral barriers to inclusion and to knowledge equity;
Finding ways to include missing voices and bridge gaps in content, reach, and users (in terms of both access and contributions).

Q4a. Could this Recommendation have a negative impact/change?

Some editors might not agree on the need of content diversity and continue deleting articles based on notability reasoning and tensions might emerge. While we cannot force volunteers to write on identified content gaps, we can use models to make people aware of where those gaps are.

Q4b. What could be done to mitigate this risk?

Suggesting articles or lists of articles has a limited effect. It may require other actions in order to work on diversity:

Create Discourse on diversity specific to the different areas in the Wikimedia movement.
Raise awareness with specific measurements and data visualizations.
Improve organization with tool development and content recommendations in order to address specific gaps.
Develop the strategy with goals and priorities to address.

Once metrics and visualizations are set up for all those diversity types we can start and trigger a virtuous cycle.

More communication by community leaders, WMF and utilization of existing working models like Women in Red’s redlists can make people aware of where gaps are and how their specific interests can be focused to fill those gaps.

Q5. Why this Recommendation? What assumptions are you making?

It can be supported by the maturity models theory. To get mature at one topic (e.g. diversity) you need to incorporate discourse, methods, organization and strategy. It has already been suggested (Miquel, 2019) how to get mature at cultural diversity using a maturity model approach.

Q6. How is this Recommendation connected to other WGs?

It relates to capacity building because those gaps we find along with the priorities and guidelines we propose, may need to be addressed by the different communities. Some of them may not have enough capacity to represent themselves. It is also connected with the Technology & Product WG as metrics, and it could be co-managed with the wikitech team.

Q7. Does this Recommendation connect or depend on another of your Recommendations?

Yes. It is necessary to revise the policies in order to protect content representing diversity.

Q8a. What is the timeframe of this Recommendation in terms of when it should be implemented? 2020, 2021, etc. Does it have an urgency or priority? Does this timeframe depend on other Recommendations being implemented before or after it?

The implementation, dissemination and improval must be continual. This is not a one-step implementation but a progressive and steady research and communication task.

Q8b. Who needs to make a decision on this Recommendation?

Considering the centrality of the proposal, it might fall between community engagement, research, data analytics and languages.

Q10. What type of Recommendation is it?

This recommendation is between simple and complicated in the sense that there is already a foundation but it needs planning and research in order to properly map the different types of diversity in a flexible and pragmatic way in order to measure the gaps, and ultimately, come up with relevant guidelines to bridge the most important ones.

Q12. What are the concerns, limiting beliefs, and challenges for implementing this Recommendation?

The main concern is to obtain visibility among the Wikimedia channels. Some projects are buried in meta or in toollabs without other people knowing them. Therefore, it is necessary that these metrics, visualizations and guidelines are introduced in other communication channels (mailing lists, Facebook groups, etc.) and their updates and news are being constantly disseminated in Wikimedia events.

QXX. How much money is needed to implement this recommendation?

The cost would be basically a researcher & developer able to map the different topics, retrieve, analyze and store the data in order to generate the statistics and visualizations.

Ideally there should be a program dedicated to diversity in the future in the same way there is one dedicated to libraries.