Wikimedia Conference 2016/Program/17
17: Listening to community voices using surveys
[edit]How to move forward
[edit]- What was this session about?
How survey can be used to understand better the communities you serve.
- What are the next steps to be taken?
- Form a list of people who have survey experience in the movement to increase knowledge exchange around surveys
- Build up a list of surveys that can be borrowed and re-used by affiliates, program and other community leaders (see also Surveys/Learning) .
- Who is the person to reach out to?
- Edward Galvez, Survey Specialist at the WMF
- Slides
This session documentation was approved by the facilitator.
- Original Description
- Listening to community voices in a balanced way is key to making decisions that are collaborative and legitimate, important qualities for our movement. Surveys are one way to bring in the voices that may otherwise not be heard. They are not just about getting input of data, but are a way to gather insights about needs, challenges, or opportunities of communities. In this session, participants will learn about how organizations are applying new approaches to making decisions through surveys, using case studies to highlight the potential of surveys. More importantly, participants would be able to learn from each other about when surveys are used, when other approaches to community listening are appropriate, and what should be avoided.
- Desired Outcome
- Participants will have a better understanding for when to use surveys and how to use surveys
- Create a detailed best practice list about what participants would like to see about surveys (design elements & SWOT), which can be used by participants.
- Participants will get to know survey resources and people with survey experience
- Session Format
- Conversing & Working (Panel & workshop. We will have 4 presenters (5 minutes each) with discussion groups following)
- Speaker(s)
- Edward Galvez, with the panelists Felix Nartey (Wikimedia Ghana User Group), Sandra Rientjes (Wikimedia Nederland) and Anna Torres (Wikimedia Argentina)
- Summary of the session
Edward Galvez opened the session on how to understand communities using surveys. He gave a short introduction explaining that surveys mainly can be used as “flashlight in the dark” to understand certain aspects of communities. You see some aspects highlighted, but you don’t necessarily see the context. Edward pointed out that trying to learn from a large group of people was definitely a challenge.
After his short introduction, Edward first handed over to Felix Nartey (Wikimedia Ghana User Group), Anna Torres (Wikimedia Argentina) and then Sandra Rientjes (Wikimedia Nederland). Felix told the audience the experience of his user group doing a professional survey (with the support of WMF) with the goal to sum up activities of a pilot project for scaling up offline Kiwix in Ghanaian schools. The main goal (reasons) of the project was to find alternative/auxiliary means to increase the reach and (or) use of Wikipedia and its sister projects. The group learned a lot doing this survey, e.g. that the sample size didn't necessarily reflect the entire population that used the resource (see the slides).
After Felix’ presentation, Anna took over and presented Wikimedia Argentina’s experience with surveys. They conducted a survey to meet their community of editors to expand their activities outside of Buenos Aires. The main goal of the survey, which was created together with its board and WMF staff, was to “localize” the editors around Argentina and to be aware of their knowledge and activities. Two findings of the survey were important: 41% didn't want to participate in Wikimedia Argentina’s activities and a great number of editors believed that the main support from Wikimedia Argentina should be providing information and resources to editors.
Sandra from Wikimedia Nederland explained her chapter’s experience with a big survey they had conducted last year. There were signals that many Wikipedians had seen problems in the working atmosphere; dealing with newbies, gender gap, etc. Wikimedia Nederland felt, on the other hand, that Wikipedians were difficult to reach, as the chapter is only in contact with around 200 Wikipedians. This was a reason to do a proper survey and focus on community health issues. As the chapter did not want to create and conduct the survey completely by itself, an expert was hired to create the survey and to process the results. The questions were created together with the staff and community members. The survey was announced (and conducted) using a SiteNotice for about ten days. Finally, 452 of around 1200 completed the survey. The idea, Sandra explained, was to do such a survey in ~ 2 years with a different focus.
After the lightning talks, the audience was split up into four groups. After the group discussions, they summarized the following important points for creating, conducting and processing surveys:
- Make clear whom you are targeting with your survey
- Clear and exact questions → no language problems, everyone understands the same
- Problem: sometimes you set an agenda with the questions with ask (bias in the survey itself)
- You can also bias the results with the method (e. g. Google survey: some people don’t want to use Google)
- It is helpful to have an expert who knows how to design a survey to get meaningful results
- Make sure you have not too many questions and not the only one type of questions → balance
- Central point: make clear why you are doing the survey → also makes it easier to explain and communicate the survey
- Pretest the survey: show it to the board, community members or other stakeholders; possibly refine it
- Check the questions in detail: is each single question really useful?
- Don’t do it too often
At the end, Edward mentioned further resources around surveys
- Learning Patterns
- Survey Support Desk
- [[Category:Survey skills]] (people who have experience with surveys)
- Qualtrics, the online survey platform provided by the Wikimedia Foundation
- surveys
wikimedia.org for help
The groups wrote down the following aspects:
| 1st Group |
|
| 2nd Group |
|
| 3rd Group |
|
| 4th Group | Community & Process (How)
|