Jump to content

Learning and Evaluation/Archive/Learning modules/3Rely on second-hand data sparsely

From Meta, a Wikimedia project coordination wiki
Jump to page

Part 1: Introduction

Welcome!
Why Survey?
Why Surveys Are Useful
Constructs
Operationalize
Survey instruments
Types of information
Attributes - a special case
Survey Objective and Planning

Part 2: Reliability & Validity

Reliability & Validity
Reliability
Validity
Face Validity
Content Validity
Criterion Validity
Construct Validity

Part 3: Question Construction

Writing Good Questions
Questions from Existing Surveys
Constructing your own Questions
Be Specific
Be Concise
Avoid Double Negatives
Minimize Social Desirability Bias
Avoid Double-barreled questions
Avoid abbreviations, jargon, technical terms, or slang
Avoid leading questions
Avoid loaded questions
Use appropriate wording
Ask useful questions
Rely on second-hand data sparsely
Use caution when asking personal questions

Part 4: Response Options

Question types
Fill-in-the-blank
Dichotomous pairs
Multiple choice
Check all that apply
Ranking
Scales
Choosing response options

Part 5: Questionnaire structure

Important considerations
Questions order
Additional Resources
Feedback

  Wikimedia Training Designing Effective Questions Menu

Rely on second-hand opinion only when first-hand information is not available


Objective
After an international conference, the coordinator wants to know what participants learned from the conference workshops.


Very Poor
The conference coordinator decides to not evaluate participant experiences because the workshop has ended and cannot survey participants directly.
Although not ideal, if an event is planned where surveys are not administered, it is possible to survey participants or workshop leaders after the event.
Poor
The conference coordinator decides to survey workshop presenters to ask questions around their learning objectives and participant comprehension of the material.
This approach continues to be poor because leaders may not have an accurate understanding of the extent to which participants actually understood and are able to integrate and apply the information presented.
Better
The conference coordinator surveys conference participants, asking questions around learning objectives and participant comprehension of the material, to get an idea of how the conference went.
Assessing participants' experiences from the perspective provides useful information because you collect their self-report of learning outcomes as well as test for comprehension.


Note: In some cases, second-hand opinion may be part of your evaluation design; for example to measure whether program leaders are accurately assessing satisfaction of participants. It is a method to confirm first-hand data.