Jump to content

Events Team Portal/Community space/Event Evaluation Toolkit

From Meta, a Wikimedia project coordination wiki

Event Evaluation Toolkit

[edit]

Below is the list of basic question that the Wikimedia Foundation uses to evaluate large movement events like Wikimania and other regional and thematic events. These questions can also be used to evaluate Wikimedia movement events that the Foundation cannot evaluate directly. Feel free to use any part of these questionnaires to evaluate your own event. The questions are organized into an initial registration questionnaire, a post-event survey, and a follow-up survey for participants 3 months after the event has finished.

These survey questions have been designed to answer several evaluation questions about the impact, design, and equity of a movement event. You may be interested in only some of these questions for your event.

  • Diversity: How diverse and representative are event participants of the Wikimedia movement and the world?
  • Safety and security: Do all participants feel safe and comfortable participating?
  • Virtual environment: For virtual events, were participants able to access all event elements that they needed or wanted to? Were participants and hosts satisfied with the technical infrastructure of the event?
  • Inclusion and equity: How accessible and inclusive is the event? Are there any differences in conference experience, utility of information gained, or individual impact between demographic groups? For newcomers to the movement or the event?
  • Reaction: Did participants find event content to be relevant and of high quality? Did they find the format and environment sufficient for community building and collaborative work?
  • Learning: What gains in knowledge, skills, beliefs, confidence, and commitments do we see in participants?
  • Application: To what degree did participants apply what they learned at the event to their Wikimedia work after the event? To what degree did they share what they learned at the event with others in their community?
  • Results: What new ideas or project progress happened at the event?
  • Partnerships and Network: Did participants gain more contacts, including from other affiliates or geographies, and did those relationships endure over time? Did participants deepen their collaborative relationships with others, and to what degree did these relationships endure over time?

Why might I want to evaluate my event?

[edit]

Evaluation can have many benefits, the most immediate being information that you can use to improve the design or content of your event in the future. Having data on the impact of events can also be helpful to demonstrate past success when seeking funding for future events, whether from Wikimedia or non-Wikimedia entities.

Registration Questions

[edit]

Movement and event participation

[edit]
In what ways do you participate in the Wikimedia movement? Please select all that apply.
I create, contribute, and/or edit content on at least one Wikimedia project (e.g., Wikipedia, Wikimedia Commons, Wikidata, etc.) [ ]
I represent an affiliate [ ]
I organize Wikimedia projects, events, campaigns, or groups [ ]
I work for pay for the Wikimedia Foundation [ ]
I work for pay for another Wikimedia entity [ ]
I participate in another way [ ]
I prefer not to say [ ]
In what year did you begin contributing to Wikimedia projects? ___________
Display if the event occurs regularly:
Have you attended [event] before?
Yes (how many times?) [ ]

___________

No [ ]
Display if the question above is answered with "no" or is not displayed:
How did you learn about this event? Please select all that apply.
Another Wikimedia contributor told me about it [ ]
I saw an announcement on a wiki [ ]
I saw an announcement on a Wikimedia mailing list [ ]
I saw an announcement on social media (e.g., Facebook, Twitter, etc.) [ ]
I heard about it on a messaging app (e.g., Telegram, WhatsApp, etc.) [ ]
I heard about it from an affiliate or usergroup [ ]
I learned about it in a different way [ ]

___________

I prefer not to say [ ]
Exclude the present event type:
Which of the following types of in-person events have you attended? Select all that apply.
Wikimania [ ]
Wikimedia Summit (formerly known as Wikimedia Conference), the annual conference for Wikimedia affiliate organizations (chapters, thematic organizations, user groups) [ ]
International hackathons (Wikimania Hackathon or Wikimedia Hackathon) [ ]
Regional events, such as CEE Meeting, IberoConf, Wiki Indaba, WikiCon North America [ ]
National or local Wikimedia conferences such as Wikicamps, local hackathons, and local conferences hosted by Wikimedia affiliates [ ]
Local contributor meet-ups and events such as edit-a-thons or editing parties, photo tours and other volunteer-led initiatives in your area [ ]
Thematic events, such as GLAM Wiki Conference and Diversity Conference [ ]
None of the above [ ]
Auto-populate with events selected in previous question:
How many times have you attended this type of event?
Wikimania _______
Wikimedia Summit (formerly known as Wikimedia Conference), the annual conference for Wikimedia affiliate organizations (chapters, thematic organizations, user groups) _______
International hackathons (Wikimania Hackathon or Wikimedia Hackathon) _______
Regional events, such as CEE Meeting, IberoConf, Wiki Indaba, WikiCon North America _______
National or local Wikimedia conferences such as Wikicamps, local hackathons, and local conferences hosted by Wikimedia affiliates _______
Local contributor meet-ups and events such as edit-a-thons or editing parties, photo tours and other volunteer-led initiatives in your area _______
Thematic events, such as GLAM Wiki Conference and Diversity Conference _______

Demographic Questions

[edit]
Where do you currently live? ___________
In what language or languages are you a fluent speaker? ___________

Baseline feelings of community

[edit]
How much do you agree or disagree with the following statements? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
I take pride in contributing to the Wikimedia projects. [ ] [ ] [ ] [ ] [ ] [ ]
I feel like I belong in the Wikimedia movement. [ ] [ ] [ ] [ ] [ ] [ ]
Most Wikimedians are sincerely interested in supporting each other. [ ] [ ] [ ] [ ] [ ] [ ]
Most Wikimedians are committed to building long-term cooperative relationships. [ ] [ ] [ ] [ ] [ ] [ ]
I feel like my work is recognized by other Wikimedians. [ ] [ ] [ ] [ ] [ ] [ ]

Post-event questions for participants

[edit]
Did you attend any portion of [event]?
Yes [ ]
No (skip to end) [ ]
Did you apply for a scholarship to attend [event]?
Yes, and I received one [ ]
Yes, but I did not receive one [ ]
No, I did not apply [ ]
Skip if the previous question is answered "no":
How much do you agree or disagree with the following statements? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
It was easy to understand the scholarship application process. [ ] [ ] [ ] [ ] [ ] [ ]
The scholarship application was easy to complete. [ ] [ ] [ ] [ ] [ ] [ ]
I was informed of my application status clearly and in a timely manner. [ ] [ ] [ ] [ ] [ ] [ ]
Skip if the question above is not displayed:
What else would you like us to know about your experience with the scholarship process? ___________
Skip this question if no part of the event was held in-person.
How much do you agree or disagree with the following statements about the event venue and environment? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
The venue was accessible to my physical needs. [ ] [ ] [ ] [ ] [ ] [ ]
The venue was accessible to my language needs. [ ] [ ] [ ] [ ] [ ] [ ]
It was easy to get to where I needed to go at the venue. [ ] [ ] [ ] [ ] [ ] [ ]
The physical environment of the event allowed me to fully see or hear, concentrate, and participate. [ ] [ ] [ ] [ ] [ ] [ ]
I felt safe, respected, and comfortable participating at this conference. [ ] [ ] [ ] [ ] [ ] [ ]
This conference allowed me to participate in a way that is comfortable to me. [ ] [ ] [ ] [ ] [ ] [ ]
I was able to secure lodging that was convenient to the event location. [ ] [ ] [ ] [ ] [ ] [ ]
I was, overall, satisfied with the event venue and environment. [ ] [ ] [ ] [ ] [ ] [ ]
Skip if the question above is not displayed:
What else would you like us to know about the event venue and environment? ___________
Skip this question if no part of the event was held virtually.
How much do you agree or disagree with the following statements about the event platform and environment? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
The platform and event content was technically accessible to me. [ ] [ ] [ ] [ ] [ ] [ ]
The event was accessible to my time zone. [ ] [ ] [ ] [ ] [ ] [ ]
The event content was accessible to my language needs. [ ] [ ] [ ] [ ] [ ] [ ]
It was easy to navigate and use the event platform. [ ] [ ] [ ] [ ] [ ] [ ]
The event platform and format allowed me to fully see or hear, concentrate, and participate. [ ] [ ] [ ] [ ] [ ] [ ]
I felt safe, respected, and comfortable participating at this conference. [ ] [ ] [ ] [ ] [ ] [ ]
This conference allowed me to participate in a way that is comfortable to me. [ ] [ ] [ ] [ ] [ ] [ ]
I was, overall, satisfied with the event platform and environment. [ ] [ ] [ ] [ ] [ ] [ ]
Skip if the question above is not displayed:
What else would you like us to know about the event platform and environment? ___________
How much do you agree or disagree with the following statements about the information or skills you gained at [event]? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
The content was appropriate for my skill and experience level. [ ] [ ] [ ] [ ] [ ] [ ]
The content was interesting to me. [ ] [ ] [ ] [ ] [ ] [ ]
I gained knowledge relevant to my activities in the Wikimedia Movement. [ ] [ ] [ ] [ ] [ ] [ ]
I gained practical skills relevant to my activities in the Wikimedia Movement. [ ] [ ] [ ] [ ] [ ] [ ]
I left with new or different ideas about how I may do my Wikimedia work in the future. [ ] [ ] [ ] [ ] [ ] [ ]
I intend to apply what I learned to my Wikimedia work. [ ] [ ] [ ] [ ] [ ] [ ]
I intend to share what I learned with my home Wikimedia community or communities. [ ] [ ] [ ] [ ] [ ] [ ]
Based on my experience at [event], I intend to begin contributing to a Wikimedia project that I haven't contributed to before. [ ] [ ] [ ] [ ] [ ] [ ]
What was the most useful or exciting knowledge or skill you gained at [event]? ___________
How do you plan to apply or share what you learned at [event]? ___________
How much do you agree or disagree with the following statements about the format of [event]? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
I find lecture or panel presentation-style sessions effective. [ ] [ ] [ ] [ ] [ ] [ ]
I find working groups, hands-on training, or other active learning-style sessions effective. [ ] [ ] [ ] [ ] [ ] [ ]
Most of the sessions I attended had a format (eg. lecture presentations, panels, or active learning sessions) that I found effective for my own learning style. [ ] [ ] [ ] [ ] [ ] [ ]
How much do you agree or disagree with the following statements about your experience with other attendees of [event]? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
I talked to others about a topic or issue that we intend to begin working on. [ ] [ ] [ ] [ ] [ ] [ ]
I collaborated with others about new potential solutions to an existing problem or issue. [ ] [ ] [ ] [ ] [ ] [ ]
I collaborated with others to create a new community or coalition. [ ] [ ] [ ] [ ] [ ] [ ]
I talked to others about new ways of convening and/or collaborating. [ ] [ ] [ ] [ ] [ ] [ ]
What was the most exciting new idea that you discussed or advanced at [event]? ___________
How much do you agree or disagree with the following statements about the projects you work on relevant to [event]? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
We completed one or more projects at [event]. [ ] [ ] [ ] [ ] [ ] [ ]
[If not strongly disagree/disagree] Can you tell us more about the project or projects you completed? _________________________
We reached an important milestone for one or more projects at [event]. [ ] [ ] [ ] [ ] [ ] [ ]
[If not strongly disagree/disagree] Can you tell us more about the milestone(s) you reached? _________________________
We agreed to temporarily or permanently stop work on an existing project. [ ] [ ] [ ] [ ] [ ] [ ]
[If not strongly disagree/disagree] Can you tell us more about the project or projects you stopped and why? _________________________
How much do you agree or disagree with the following statements? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
I take pride in contributing to the Wikimedia projects. [ ] [ ] [ ] [ ] [ ] [ ]
I feel like I belong in the Wikimedia movement. [ ] [ ] [ ] [ ] [ ] [ ]
Most Wikimedians are sincerely interested in supporting each other. [ ] [ ] [ ] [ ] [ ] [ ]
Most Wikimedians are committed to building long-term cooperative relationships. [ ] [ ] [ ] [ ] [ ] [ ]
I feel like my work is recognized by other Wikimedians. [ ] [ ] [ ] [ ] [ ] [ ]
I made new connections with other Wikimedians at [event]. [ ] [ ] [ ] [ ] [ ] [ ]
I connected with friends and colleagues I don't usually see at [event]. [ ] [ ] [ ] [ ] [ ] [ ]
Here is a quick quiz to see how aware Wikimedia communities are of the Wikimedia Foundation’s goals! Which of the following is the central goal of the 2030 strategic direction?
Wikimedia will clarify and strengthen our global perception and our free knowledge mission, growing awareness and affinity with people in places we don’t yet serve.

[ ]

We will champion and defend the integrity and independence of free knowledge, the Wikimedia community, projects, and values with governments, businesses, civil society, and other stakeholders. [ ]
Wikimedia will become the essential infrastructure of the ecosystem of free knowledge, and anyone who shares our vision will be able to join us. [ ]
We will co-create, grow, and cultivate a safe and welcoming, diverse, sustainable, and thriving movement of leaders, contributors, advocates, and partners for free knowledge. [ ]
I am not sure. [ ]
This is the central long-term goal of the Wikimedia movement:

By 2030, Wikimedia will become the essential infrastructure of the ecosystem of free knowledge, and anyone who shares our vision will be able to join us.

Not at all Somewhat Mostly Completely I am not sure
How relevant is this goal to you personally? [ ] [ ] [ ] [ ] [ ]
How relevant is this goal to the Wikimedia projects, campaigns, or communities in which you participate? [ ] [ ] [ ] [ ] [ ]
How much do you agree or disagree with the following statement? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
The Wikimedia movement's long-term goal inspires me to contribute more to Wikimedia projects than I would somewhere else. [ ] [ ] [ ] [ ] [ ] [ ]
What did you most hope to learn, gain, or achieve by attending [event]? ___________
What were your favorite moments of [event]? ___________
What suggestions do you have for improving [event] in the future? ___________
Sometimes, Wikimedians contact one another to share or seek information, best practices, or strategies helpful for the work that they do. In your estimation, how many Wikimedians have you been in contact with in this capacity in the last three months? ___________
Of these, how many are outside of your home affiliate or language project? ___________
Sometimes, Wikimedians collaborate with one another to develop projects, activities, groups, or events. In your estimation, how many Wikimedians have you been in contact with in this capacity in the last three months? ___________
Of these, how many are outside of your home affiliate or language project? ___________
Could you tell us more about the projects, activities, or groups that you have collaborated with other Wikimedians in organizing in the last three months? ___________
Do you intend to attend [event] in the future?
Yes [ ]
No (why not?) [ ]

___________

Would you recommend attending [event] to others?
Yes [ ]
No [ ]
Are you interested in organizing a community event in the future? If you select "yes," you will not be committing to organizing an event, but a Wikimedia Foundation staff member from the Events team may contact you via email to follow up.
Yes [ ]
No [ ]

Follow-up questions for participants

[edit]
How much do you agree or disagree with the following statements about what you learned at [event]? Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree I am not sure
What I learned at [event] has changed how I think about my work in the Wikimedia movement. [ ] [ ] [ ] [ ] [ ] [ ]
I have applied what I learned at [event] to my work in the Wikimedia movement in the last three months. [ ] [ ] [ ] [ ] [ ] [ ]
[If not strongly disagree/disagree] How have you applied what you learned at Wikimania to your Wikimedia-related work in the last three months? ___________________
[If not strongly agree/agree] What, if anything, has kept you from applying what you learned at Wikimania to your Wikimedia-related work as much as you might have liked? ___________________
I have shared what I learned at [event] with others in my home community in the last three months. [ ] [ ] [ ] [ ] [ ] [ ]
[If not strongly disagree/disagree] How have you shared what you learned at [event] with others in the last three months? ___________________
[If not strongly agree/agree] What, if anything, has kept you from sharing what you learned at [event] with others  as much as you might have liked? ___________________
Because of my experience at [event], I started contributing to a new Wikimedia project in the last 3 months. [ ] [ ] [ ] [ ] [ ] [ ]
[If not strongly disagree/disagree] What new project have you started contributing to? ___________________
Sometimes, Wikimedians contact one another to share or seek information, best practices, or strategies helpful for the work that they do. In your estimation, how many Wikimedians have you been in contact with in this capacity in the last three months? ___________
Of these, how many are outside of your home affiliate or language project? ___________
Sometimes, Wikimedians collaborate with one another to develop projects, activities, groups, or events. In your estimation, how many Wikimedians have you been in contact with in this capacity in the last three months? ___________
Of these, how many are outside of your home affiliate or language project? ___________
Could you tell us more about the projects, activities, or groups that you have collaborated with other Wikimedians in organizing in the last three months? ___________
Do you intend to attend [event] in the future?
Yes [ ]
No (why not?) [ ]

___________

Would you recommend attending [event] to others?
Yes [ ]
No [ ]
Are you interested in organizing a community event in the future? If you select "yes," you will not be committing to organizing an event, but a Wikimedia Foundation staff member from the Events team may contact you via email to follow up.
Yes [ ]
No [ ]