Talk:Global metrics/Review

Review of Global Metrics

Discussion

Thanks for coming to share your feedback on Global Metrics.

Narrative Feedback from Art+Feminism

The lead organizers of Art+Feminism have spent quite a bit of time thinking about ‘what counts’ when evaluating Wikipedia project through outcomes metrics. We don’t believe the primary focus on active editors meshes entirely with our mission, especially as it is defined at present. Specifically, we object to the exclusion of all non-article-space edits, and the expectation that one edit at least 5 articles every single month, or you don’t count at all as a productive success. And because we feel that one of our biggest goals is organizing a human-resources infrastructure, we feel that there isn’t any way to account for our community building work. The neoliberal drive to quantify and measure all human outputs means we value what we can easily measure. Those outputs, like community, which are less measurable become less valued precisely because we can’t quantify them.

We would like to reconsider the definition of a retained active editor. At present a retained active editor is defined as a user that has made at least 5 edits per month in article space, for a period of 6 or 12 months. All three of the lead organizers for Art+Feminism do not qualify as “retained active editors” over a 12 or 6 month period in its current definition. Think about that. We are metapedians who spend much/most of our time in meta, AfD, meetup and talk pages; we compose longer texts (like this) collaboratively in a word doc or make all our edits in our sandbox like good Wikipedians, then paste them into articles space and only get credit for one edit; we spend many hours a week organizing off-wiki; we go to Wikicon and give presentations that demonstrate leadership and which others learn from. None of this "counts." Furthermore the annual schedules of of academia and the NY art world means that two out of the three of us to take much of August off from as much responsibility as we can, Wikipedia included. It strikes us that this resembles a re-inscription of a traditional hierarchy of gendered labor. This facilitation is the invisible labor of “making of the home” -- we are enabling the legible work other people. This work is erased as legitimate labor. The historical campaign Wages for Housework, argued that housework was not understood as legitimate work or labor because it is not remunerated.

We also believe that this focus on metrics excludes recognition for one of our biggest successes: we catalyze others to organize in their own communities. Aside from the 30+ events others organized in 2014, 70+ in 2015, and 130+ events organized in 2016 as part of our annual March international editathon, we have had many other examples of catalyzing others: Wikimedia NYC’s public activity jumped dramatically after our 2014 editathon, from between 8 and 14 public events from 2008 to 2013, to 29 public events in 2014 and 60 in 2015 (Data Here). We trained User:Aliceba at one of our 2014 Train-the-Trainers sessions; Alice went on to start the AfroCROWD initiative. We have catalyzed other such standalone projects, or IEG research projects. Many of our node locations began holding monthly or quarterly meetups, and we have heard from others that the Wikipedia events in their cities are now stronger, and now squarely focused on intersectional issues of social justice. And we have seen our past participants move elsewhere and organize events, and node organizers relocate and start events in their new location, or help us at MoMA when they move to NYC. At best we can piece together some narratives of the events we know about. Community building and community organizing is important. It is a base from which to mobilize, and can be a bulwark against harassment. There is no extant tool to measure this, no obvious number to assign to it, no easy way to quantify it. And thus it fails the active editor litmus test.

Lastly, we believe that press is a relevant metric -- not as a goal itself, but as a means to an end. The reality is that press is one of the key drivers of traffic to our events. Pre-event press has a direct impact on event attendance, and post event press builds overall awareness. We could not have achieved anywhere near the level of participation we have achieved without using the press as a mechanism. Additionally, we want to emphasize that press, social media, and other forms of communication are hard work, especially if you want to do them well. --Theredproject (talk) 20:55, 18 March 2016 (UTC)

Thank you Theredproject for your insightful feedback. Your points around motivation, community building, catalyzing others really resonates with me, and makes it clear how much is missing in the discussion about outcomes today, especially those non-content, non-participation related outcomes. I really hope that whatever "Global Metrics" becomes starts to break down the "quantitative vs. qualitative" divide, and gives space for the things you mentioned. -- Shouston (WMF) (talk) 19:19, 25 March 2016 (UTC)

Than you @Shouston (WMF):. I too hope that these can be incorporated.--Theredproject (talk) 21:00, 25 March 2016 (UTC)

Hi @Theredproject: - I wanted to follow-up with you on your comments, especially those around community building. This update to Global Metrics has continued and one of the new areas we are interested in introducing as a "global measure" is community building, to replace some of the other metrics we currently have (i.e. Active Editors, New editors and the learning question on motivation) (You can find more information about the proposed updates to Global Metrics here.) The intention is that this "measure of community building" would not be a number; it would be an opportunity to capture qualitatively those many things you referred to before. However, the term "community building" is quite broad and means many things to many people; we are seeing this firsthand as we interview grantees about what "community building" is to them.

So, I'd love to get your thoughts on a few questions:

• What is "community building" to you? Could the term be broken down into components (e.g. motivation, community health, skill development, leadership development) to make the definition of the term more consistent (and thus more translatable to those who don't speak English)?
• Do you have any suggestions about how community building could incorporated to be a part of grant proposal and reporting requirements? (e.g. how could grantees set goals / measures of success around "community building" and then capture the outcomes of their efforts and report on them in a grant report?)

Any insights or suggestions you have, based on your own / the collective experiences of the Art+Feminism team would be greatly appreciated! We are excited to investigate this area, but we want to be cautious that we don't simply create a broad, open-ended question that is difficult to understand and difficult to answer. Thanks so much! -- Shouston (WMF) (talk) 19:24, 15 June 2016 (UTC)

Hi @Shouston (WMF): I did see the new conversation starting about your efforts to revise reporting and was very glad to see the community aspect. I don't have a simple answer because, as you note, these are not simple questions. I will think about, and discuss with @Failedprojects: and @Siankevans:. --Theredproject (talk) 18:56, 16 June 2016 (UTC)

Hi Theredproject - Wonderful! I look forward to your thoughts and discussion. Thanks in advance for your insight and feedback :) -- Shouston (WMF) (talk) 21:49, 16 June 2016 (UTC)

Hi Theredproject - I wanted to let you know that Global Metrics has finally been updated after almost 7 months of review and consultation. You can find (a lot) more information about the replacement for Global Metrics in the final report we just published. Thank you and the Art+Feminism team for all the feedback you provided! We could not have updated Global Metrics without that feedback, and I hope you find that your thoughts and suggestions are reflected in the changes that were made. On the specific topic of community building: Unfortunately, while there was a lot of positive responses about the usefulness and importance of community building, there wasn't convergence on how it could be defined, captured and communicated within the context of a grant report. However, we will continue to watch and learn how grantees include outcomes related to community building in their grant report / activities, and keep an eye out for how it develops! -- Shouston (WMF) (talk) 20:46, 2 August 2016 (UTC)