User:VHargyono (WMF)/Test
This page is under construction. Please help review and edit this page. |
Commonly Used Data Collection Methods[edit]
This section will present eight of the commonly used data collection methods that people usually use in evaluation. Each section contains basic information of each method; its strengths and limitations; overview on how to do it; and further resources.
- What is it? And how do you do it?
Desk based literature review. The documents can include internal documents within a program or organization, interview notes/summary, academic literature, journal articles, transcripts, meeting notes, policy papers, grey literature, newsletters, personal documents, etc.
- How?
- Create a scope, timescope, geographic, languages etc.
- Organized around central themes/questions
- Methodologies, scoping, systematic, exhaustive
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
Review/analysis of statistical data is the process of examining quantitative data using statistical methods for us to describe, assess, compile, and compare the data that you have collected. Quantitative data are information in numeric form. There are two commonly used methods on doing this:
- Quantitative (secondary data), statistical records (for example: number of students, schools records)
- Secondary analysis based on official records/databases
- How?
- Assess baseline data
- Depends on the context/organization culture whether we can use the existing data
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
It originally aims to include rural people's knowledge and opinions into the project's planning and management. This method is using a qualitative approach which i used to get a more thorough understanding of situation in the community or about the community itself. It is usually used in collaborative project that involves community. The main goal of this method is to help assist the community to analyze their own situation in a project, create learnings, and to make sure we can transform the learning into action in the future.
- How?
- Participative and community-led. Project implementers are facilitators
- Project' goals and objectives are formulated based on community's needs and agreed by the community
- Tools used, approaches, and data analysis are participatory
- Common tools and techniques used include: observation, semi-structured interviews, focus group discussions, photography, videos, case studies, charts and diagrams, maps
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
It's an activity of filming or photographing certain activities or conditions. It can also include other visual or arts methods, such as painting or sculpture. The purpose is to present information in a more engaging way and increase understanding of a project or program implementation' results. This method can also help in visualizing different kinds of information, in many cases it helps to reveal hidden meaning that may not be possible through a survey approach.
- How?
- Storyboard
- Community mapping
- Photography which represents their experience on the issue (for example: gender inequality)
- Photo Voice (also a participatory method)
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
A focus group is one of the qualitative approaches that can help us obtain a more in-depth understanding of problems that our program is trying to solve. Since it is a qualitative approach, the data collected is descriptive. It is done by conducting a group interview with people who are impacted, have shared experiences, interests, and/or characteristics, and the discussion will focus on a particular topic. Usually it is conducted in a small discussion group from 4 up to 10 people participating. This group is purposefully selected based on certain criterias. The discussion is led by a facilitator, whose role is very important. The facilitator must be able to build a safe environment that will encourage participants to express their responses, perceptions, and point of view genuinely. The facilitator uses a set of questions or topics to guide the conversation. The questions and topics are predetermined based on the key evaluation questions.
- How?
- Develop an interview guide with 7-10 questions based on your key evaluation questions
Prepare a consent form to be signed by each participant
- When introducing the interview to participants, make sure to provide clear information about the objectives of the Focus Group Discussion, reason why the participant was selected, and don’t forget to introduce the interviewer and organization as well
- Can be conducted face-to-face with participants or via online video conferencing. Be mindful when using teleconferencing, facilitator must make sure that everyone participates in the discussion
- The discussion is usually conducted for 60 to 90 minutes using a semi-structured method. Facilitator can help stimulate the discussion by writing down participants’ answers on a whiteboard or virtual board, or on sticky notes
- Start comparing results with other interviews conducted to other group of participants
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
- Systematic data collection approach to examine the natural behaviour of individuals
- How?
- Can be direct observation: Observing project activities in action without interaction, or Participant observation: the observer also joins in activities and interacts with others
- Qualitative and quantitative
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
- How?
- Based on an ‘interview guide’ but able to follow topical trajectories in the discussion that may stray from the guide, where appropriate
- Includes open-ended questions: best to tape record and transcript tapes for analysis (as discussions may stray from the guide and go deeper)
- Interviewer and respondents engage in a formal interview
- Able to bring up new ideas from the conversations
Strengths | Limitations |
---|---|
|
|
- What is it? And how do you do it?
- How?
- Structured interviews
- Quantitative
- Measurable
- Capacity building
Strengths | Limitations |
---|---|
|
|
Data Collection Method Recommendation for Evaluating Wikimedia Programs[edit]
Many Wikimedia project activities, particularly related to education, involve young students. However, not only students can participate in the project itself, we can also engage them into evaluating the program. Eventually, the students are the ones who are supposed to benefit from the program, it will be interesting to see more of their perspectives in all aspects of the project.
First, let’s understand what youth engagement means. According to ACT For Youth, youth engagement is the outcome of when we involve young people in creating positive social change. Youth can be involved from the planning stage up, decision making, and until evaluation stage. So not only having youth as participants, but also having them involved in the beginning/planning stage and the implementation phases, including evaluation, as well.
- What is it?
A photograph can be a powerful tool to tell a story and to advocate on the issues or represent changes that were made. Just like text document or video footage, a photograph can hold a large amount of information but quicker. This way, a photograph does not depend on the audience to spend their time trying to digest the information by reading and watching the document and videos.
Photovoice can be a powerful tool to enable participants, particularly students, to express their perspectives and points of view as well as representing their communities. It is done by youth taking photographs of scenes that are related to topics or themes that you are trying to evaluate. For example: students can take photos of their activities, record interviews among their peers about how they think the activities, or record themselves with their opinion about the activities conducted in the Wikicamp. After capturing the photographs, groups of youth can discuss and/or interpret what the photos highlight a particular theme.
Tips: use digital photos for more instant results. Participants and project staff can discuss about the results right as soon as the photos are taken.
- Who is involved
Individual student or divide individuals into groups, program staffs
- What do we need
- Digital cameras and/or cell phones with cameras
- Voice recorders
- Whiteboard/papers and markers
- Application example
One of your evaluation objectives is to understand what the activities and experience means to students as participants. To measure this, you can ask students to capture photos of the activities within the program, for example, during 2 days of camping. When you wrap up the camping activities, you can divide students into small groups or a large group (depending on the number of students participating). Ask students to discuss the best photos they captured. You can prepare some topics or question prompts to encourage students to voice their stories/opinions. The prompts can cover some basic questions, such as why they took a particular photo, what does the photo mean to them, what aspects of the photo that they like/dislike, etc. Or you can also frame the question prompts based on your predetermined key evaluation questions. The discussions can be audio recorded. Then you can use the photos and recording as data for your program evaluation. Then later, you can use the photos to highlight the program’s experiences, which can be displayed online by uploading and creating albums of the photographs in Wikicommons. You can also ask students to write down their perception of the photos in the albums and include them in Wikicommons.
- What is it?
Storyboard activity is a very participative evaluation method/tool so people involved in the project can be engaged in telling their stories. This tool helps people understand that evaluation is something that everyone can do. Making the evaluation process fun and engaging to stakeholders, will make it easier for us to capture rich and meaningful data. It also helps co-creation of evaluation frameworks that can represent our organization/project’s work and impacts.
The Storyboard activities particularly will be most enjoyable to engage between youth and adults, such as in a Wikicamp project. Project participants will be asked to tell their experiences in the form of stories.
- How do you do it
- Tools Needed
Participants can use any tools, materials, and images they like to create their own storyboards. You can provide, crayons, papers, printed, cardboards, stickers, glitters, pictures, photos, anything that will help participants to tell their stories!
Indicators Recommendation[edit]
Outcomes | Indicators |
---|---|
Participant's Increased self-esteem / Increased Participation |
|
Participants's Sense of Belonging in the Community |
|
Participants Retention |
|
Increased Inclusion and Accessibility |
|