This page documents the semi-structured interview questions for the formative study of the research on community-centered evaluation of AI models on Wikipedia.

Interview questions

Please describe your experience with AI tools, such as ORES, for counter-vandalism, new page review, or other tasks on Wikipedia.
Please share your experience as a [participant’s role] within your community on Wikipedia.
How did your community make decisions about the design and use of AI tools?
How were you involved in this decision-making process?
How did your community evaluate whether these AI tools fit the community’s needs and values before or after the deployment?
How were you involved in the evaluation process?
How did community members collaborate and resolve disagreements during the evaluation process?
Was the evaluation process effective or not? Why?
Are there forms of support that would be particularly helpful to have from your perspective as a [participant’s role]?
Are there forms of support that would be particularly helpful to have for the entire community?
Did the evaluation change the community’s perception and acceptance of AI tools? How?
Have you ever participated in data labeling campaigns on Wikipedia? If so, would you please describe your experience?
Can you envision ways to better support communities in evaluating AI tools before they are deployed to make more informed decisions about whether or not the community should adopt them?