Chapters Dialogue/Insights/Measuring success
Measuring success.jpg) Impact is another term that was frequently used but where we did not see much consensus regarding its meaning and measurement. The discussion about impact and metrics is very prominent in the movement, and it revolves around the question: What data and metrics are needed that can show the success or failure of activities? Of course, it is difficult to define metrics if the essential and basic question is remaining without answers: “What are the right things to do?”. Consequently, even those activities and projects that have received approval (via grants, via consecutive or repetitive execution, or by being featured in the press) are not proven to have a definite impact. The WMF faces a lot of pressure from its stakeholders and needs to ensure responsible handling of donors’ money, worldwide. They need to “detect the black sheep” that could exploit the openness of the Wikimedia movement. Therefore, there need to be rules: metrics, measurements and application procedures. Due to the high complexity, not all of them can be tailored according to individual context needs. WMF intends to create processes that are fair, participatory and used as an advising and learning tool. Together with programme leaders from within the ranks of Chapters and the community, the WMF is developing evaluation tools and learning patterns. Yet, Chapters are having difficulties to to apply metrics and processes in order to justify their budget and prove the impact of their activities. As described in previously, the goals and values are influenced by the individual context of each Wikimedian or Wikimedia organisation. And with differing goals there are differing conceptions of success. In addition, most Wikimedians are driven by the volunteerism, by the great sense of doing something good. They often conduct all these activities in their free time, in addition to their full-time job or university studies. For some, being an active volunteer and engaging in different activities already is a success. In the eyes of many Chapters, this volunteer spirit should be celebrated, but instead – as many see it – attempts are made to try to measure it and to compare it to metrics and processes, to frameworks of efficiency and effectiveness. For many Wikimedians, it feels as if they need to “force themselves” into the established metrics of the WMF, in order to fit somehow. In many cases, Chapters are engaging in “on the ground” basic work in terms of Free Knowledge (exploring new territory). It is known that in some areas, like education, are reacting very slowly to any change and that activities in the present might show their effects only in ten or twenty years. Consequently, there are no immediate results of Chapters’ activities. For short-term activities, one could measure the outcome in Kilobyte text that has been created in the course of a project, but how about long-term impact work? Some activities are created from scratch, without a blueprint from other organisations, and it’s hard to tell how their impact can be measured. Many interviewees stated that they have difficulties to directly translate their activities into success measures. In this situation, Chapters fear that they might look useless and unsuccessful – and this feeds back to the existing insecurity. To sum up, the topics metrics, measurements and reports are a big source of frustration among Chapters. At the same time, no matter if it’s the WMF, the FDC or affiliates, the whole movement is going through the challenge of evaluating its work in an adequate way. The key question remains: How can the movement combine experimentation and exploration of new territory with solid metrics? |
Learn more about
Or go back to
|
