Learning patterns/Calculating global metrics

Calculating global metrics

problemYou need to measure your project's output according to the global metrics specified by the Wikimedia Foundation.

solutionBegin documenting your process and planning data collection before you begin your project or host your event. Use the resources made available by WMF, and reach out early if you have questions.

creator• EGalvez (WMF)

discuss

endorse

created on27 August, 2014

Global Metrics This is the overview for seven global metrics learning patterns. Learn what's behind Global metrics - read the overview page.

What problem does this solve?

This learning pattern describes how to successfully gather the 7 global metrics. Grant-funded projects and programs will need to include these metrics in their project reports beginning in late 2014.

Some project leaders have experience reporting these metrics, while others are encountering these reporting requirements for the first time. These metrics can be calculated accurately and relatively painlessly for all projects, as long as you understand the metrics and plan ahead.

What is the solution?

The steps below are intended to help make sure that you are able to report your project's outcomes in terms of the 7 global metrics. You should make sure that you know how you are going to collect and calculate the data you need to report before you start. If you have questions about global metrics, you can ask your grants officer, post to the global metrics talk page, or email eval@wikimedia.org.

Before you start your event, project, or program

Create event or project pages

Project or event pages can help you organize your project or event. They provide a public place for you to record the relevant information and it serves as a central location for participants to sign up, ask questions, and begin participating. Some things you may list in a project page include:

Project or event name
Project duration or event dates and times
Project or event details (description, focus, target audience)
Opt-out information for getting user data (more on this below)
Sign-up list for participants
Any survey information
Pages edited
Follow-up information about other upcoming events or who to go for questions.

Plan to collect usernames

Online e.g. Wikipedia Adventure, International Hackathon	In-person e.g. Editing workshop, Local Editathon
1. Obtain a list of usernames
For most online projects, simply create a project page on wiki. If you are running an event, make sure to create your event page at least two-weeks prior. Invite new users who are participating to create accounts beforehand. For some online experimental projects, like the Wikipedia Adventure or the Teahouse, you can gather usernames by building it into your code and/or using MySQL. If you need assistance with this, email `evalwikimedia`.org.	Set up an event page on wiki two weeks before, and invite new users to create accounts. If your event is in one building with one Wifi connection, only 6 accounts can be created that day. If you are unable or do not need to create an event page on wiki, have a sign-up sheet prepared at the event date with a column for username and a column for email address in case they do not have a username yet or their username is illegible.
2. Ask a user's permission to get data
Post an Opt-out paragraph, which is text that tells participants that they can request to not have their data collected long form medium form short form	Ask participants to Opt-in when you ask for their username, that is, We must ask their explicit permission to use their usernames to get data about them. long form medium form short form

For projects that involve images or media, create a category in Commons...

To be able to track which photos are uploaded for a specific event, project, or program, you must create a unique category tag that uploaders can use when they submit a photo to commons.

Step 1: Come up with a category name, (e.g. Wikimania 2014 Hackathon)
Step 2: Search categories in commons to make sure the one you want to use does not exist
Step 3: Create a category by creating a category page: https://commons.wikimedia.org/wiki/Category:Add your event name here

... and promote the USE of images on Wikimedia projects

If your program is based on images and media, be sure to tell participants to add images to Wikimedia projects!
Be sure to use the correct event name and spelling when publicizing your event.

Learning Question: Determine how you will gather this information

Use the Learning Question Learning pattern to review the different ways you can collect this information. How you collect this is up to you!! We hope this metric will give you interesting insights about your event, project, or program.

During or After Your Project or Event

Take or Review the Wikimetrics Learning Module

The Wikimetrics learning module will help guide you through the process of getting information from your list of usernames. Be sure to take the learning module well before your reports are due in order to give you time to ask questions. The Wikimetrics learning module can be found at Grants:Evaluation/Learning modules

Follow the learning patterns to report your Global Metrics

The learning patterns associated with the Global Metrics are not meant to be exhaustive. However, these resources are being developed to provide support to a wide range of projects, programs, and events. Consult the patterns below for guidance on how to collect and report global metrics for your project. Feel free to improve the pattern to make it more relevant to others to may perform similar activities in the future. Post questions or comments on the pattern talk page.

Not all programs will include all 7 global metrics - for some programs a metric may not be applicable. That is, while all programs have participants, not all programs focus to add or remove text or code (bytes added and/or removed) or to upload media files for instances. Just explain why it does apply and that is perfectly okay.
Global metrics are not the only measures - in fact, exploring and reporting other metrics beyond the 7 global metrics is encouraged.

Example report form

This is an example of how to report global metrics. Your data might change depending on your project. Some of these number might even be zero - this is perfectly ok. It is important to provide clear explanations for how you arrived your numbers in the explanation column, and any other notes or information that you feel is relevant.

Metric	Achieved outcome	Explanation
1. Number of active editors involved	15	All the users who had at least 5 or more edits from July 1 to July 30 (the event was July 31)
2. Number of new editors	3	These were editors who created usernames two weeks before the event, up to the end of the event.
3. Number of individuals involved	21	All new and active editors who attended, plus volunteers who organized the event.
4a. Number of new media added to Wikimedia articles.	7	7 images out of the 100 images uploaded were used in wikimedia projects.
4b. Number of new media uploaded to Wikimedia Commons. (Optional)	100	100 images uploaded were uploaded to Wikimedia Commons.
5. Number of articles added or improved on Wikimedia projects	3	1 new article + 2 stubs expanded
6. Absolute value of bytes added to or deleted from Wikimedia projects	1,200	(absolute value) {1,000 Bytes Added} + {200 Bytes Removed} = 1,200

Learning question: Did your work increase the motivation of contributors, and how do you know?

Yes, we sent out a survey after the event and asked "To what extent did this event increase your motivation to contribute?". 16 participants (76%) said "a lot" or "some".

You can use this template to create a table for reporting global metrics as above.

General considerations

While global metrics are required for all grantees, they may be collected differently depending on the project. For example, while many will use Wikimetrics to obtain data, some may need to use MySQL because their project requires different technical skills. Also, some metrics might be zero and that is ok. If you are a grantee, speak with your grants officer if you have questions. If you are a volunteer interested in collecting this data, but need additional help, email evalwikimediaorg.

When to use

Endorsements

EGalvez (WMF)
Wolliff (WMF)
This is helpful. Satdeep Gill (talk) 17:47, 14 December 2015 (UTC)[reply]
This learning pattern is really helpful, even when its the first time you use global metrics. Thank you for sharing this.^[1] --Margott (talk) 08:34, 23 December 2015 (UTC)[reply]
VERY enlightening! Saintfevrier (talk) 22:04, 20 February 2016 (UTC)[reply]