Community health initiative/Administrator Confidence Survey

The Community Health initiative Administrator Confidence Survey is a survey designed to help the Foundation better understand the unique challenges faced by our volunteer administrators in conflict resolution situations. We aim to better understand what types of training, tools, and resources currently exist, and would be most beneficial to implement.

There have been two iterations of the Administrator Confidence Survey. The first, conducted in 2017, was a short, focused survey sent out to administrators of English Wikipedia. The second, currently under way, will be sent out to administrators on multiple projects in multiple languages.

2019

The Foundation is running an updated version of the Admin Confidence Survey, to be sent out to administrators on English, German, Arabic and Italian Wikipedias. Our aim is to have a more up-to-date measure of some of the results from the 2017 survey, as well as to expand the range of questions we ask to get a better understanding of issues facing administrators, and critical areas that require support. We plan to find participants by posting announcements on the appropriate local noticeboards, and by inviting administrators directly through user talk messages.

2017

The Wikimedia Foundation Anti-Harassment Tools Team conducted a survey to create measurements on how to better support admins who are addressing disputes on English Wikipedia. This includes measuring how well current tools, training, and information exists for admins in recognizing and mitigating things like sockpuppetry, vandalism, and harassment. This is specifically designed to learn more about and improve the admin experience. This survey was only given to English Wikipedia administrators.

Methodology

We recruited participants through a public announcement on the English Wikipedia Administrators Noticeboard and other announcement style pages that are read by administrators, as well as through invitations on User talk pages sent to 400 randomly selected active administrators. Lastly, we sent out a recruitment email on the English Wikipedia Functionaries mailing list. We aimed for a completion rate of around 100 responses.

Summary

The Wikimedia Foundation Anti-Harassment Tools team ran a survey to measure administrator confidence on English Wikipedia during September 2017. We had 117 participants fill out the survey. The survey is designed to not be a widely comprehensive survey but a specific and narrow one. The Anti-Harassment Tools Team is interested in measuring how admins feel about different kinds of conflict specific activities (wikihounding, vandalism, harassment, sockpuppetry), how confident they feel spotting, mitigating, and intervening these case types, and if they feel supported with tools and other resources from the Wikimedia Foundation.

Results summary

Survey information
- The survey was open for any active English Wikipedia administrator from September 12 to 24, 2017.
- This survey is a semi-annual survey for the Wikimedia Foundation's Anti-Harassment Tools Team to gauge how well materials, tools, training, and information exists for admins in recognizing and mitigating things like sockpuppetting, vandalism, and harassment. This will be integral in our team in helping determine what can be updated, what works, what doesn't, and how to better support the Wikipedia community.
- Our privacy statement is listed here: https://wikimediafoundation.org/wiki/Semi-Annual_Admin_Survey_Privacy_Statement
- These results were collated on September 25, 2017. Questions are presented here in the order they were asked on the survey.
- The survey received 117 responses.
Demographics
- How old are you?
  - under 18 – 0.9%
  - 18–24 – 10.3%
  - 25–35 – 21.4%
  - 36–45 – 25.6%
  - 46–60 – 22.2%
  - above 60 – 14.5%
  - Prefer not to say – 5.1%
- Where are you from?
  - North America – 56.4%
  - Europe – 27.4%
  - Asia – 1.7%
  - Middle East – 0.9%
  - Australia – 11.1%
  - Africa – 0.9%
  - South America – 0%
  - Prefer not to say – 1.7%
- Sex and gender?
  - Female – 9.4%
  - Male – 86.3%
  - Non binary/Third Gender – 0%
  - Prefer to self describe – 1.7%
  - Prefer not to say – 2.6%
- How long have you been editing Wikipedia?
  - under a year – 0%
  - 1–5 years – 8.5%
  - 6–10 years – 15.4%
  - 10+ years – 76.1%
- Are you an administrator on Wikipedia?
  - Yes – 100%
  - No – 0%
Recognizing misconduct
- How confident are you in recognizing sockpuppetry?
  - 1 (Not confident) – 1.7%
  - 2 – 18.8%
  - 3 – 25.6%
  - 4 – 37.6%
  - 5 (Very confident) – 16.2%
- How confident are you in recognizing vandalism?
  - 1 (Not confident) – 0%
  - 2 – 0%
  - 3 – 0.9%
  - 4 – 21.4%
  - 5 (Very confident) – 77.8%
- How confident are you in recognizing wikihounding?
  - 1 (Not confident) – 3.4%
  - 2 – 15.4%
  - 3 – 29.1%
  - 4 – 39.3%
  - 5 (Very confident) – 12.8%
- How confident are you in recognizing harassment?
  - 1 (Not confident) – 0.9%
  - 2 – 8.5%
  - 3 – 21.4%
  - 4 – 48.7%
  - 5 (Very confident) – 20.5%
Skills and tools
- Do you feel like you have the skills or tools to intervene or stop cases of sockpuppeting?
  - Rarely – 13.7%
  - Sometimes, though it depends – 32.5%
  - Usually – 35%
  - Almost always – 18.8%
- Do you feel like you have the skills or tools to intervene or stop cases of vandalism?
  - Rarely – 0%
  - Sometimes, though it depends – 2.6%
  - Usually – 17.9%
  - Almost always – 79.5%
- Do you feel like you have the skills or tools to intervene or stop cases of wikihounding?
  - Rarely – 14.8%
  - Sometimes, though it depends – 47.8%
  - Usually – 28.7%
  - Almost always – 8.7%
- Do you feel like you have the skills or tools to intervene or stop cases of harassment?
  - Rarely – 28.2%
  - Sometimes, though it depends – 44.4%
  - Usually – 28.2%
  - Almost always – 11.1%
Preparedness
- Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of sockpuppetry.
  - 1 (Strongly disagree) – 6%
  - 2 – 17.2%
  - 3 – 30.2%
  - 4 – 31%
  - 5 (Strongly agree) – 15.5%
- Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of vandalism.
  - 1 (Strongly disagree) – 0.9%
  - 2 – 0.9%
  - 3 – 5.2%
  - 4 – 24.1%
  - 5 (Strongly agree) – 69%
- Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of wikihounding.
  - 1 (Strongly disagree) – 6%
  - 2 – 23.3%
  - 3 – 33.6%
  - 4 – 28.4%
  - 5 (Strongly agree) – 8.6%
- Wikipedia has provided me enough resources to solve, mitigate, or intervene in cases of harassment.
  - 1 (Strongly disagree) – 10.3%
  - 2 – 23.1%
  - 3 – 30.8%
  - 4 – 26.5%
  - 5 (Strongly agree) – 9.4%

Comments about tech tools

Numbers:

Feedback comments by admins-
- 45 by survey
- 4 by email

Most common themes by admins about tech tools

“Better tools to technically prevent sockpuppetry”
“Checkuser tool is extremely outdated and kludgy with regard to the technical data and output it provides.”
“Stronger bot-detection and reporting of these activities, for administrator confirmation, would be helpful.”
Requests for improvement to blocking tools was mentioned, eg. rangeblocks, user page/topic blocks, smartblocks

Different types of tools discussed in admin confid survey

Checkuser
- Reporting improvements
- Type of data pulled
- Better interface with blocking and reporting
Vandalism patrol tools
- Twinkle
- Rollback
- Recent changes
Functional CAPTCHA
Blacklists
Suppression/deletion user history
Notification of future edits of warned vandals
Detection tools
- ClueBot NG
- ORES
improved reporting systems of bots used for abuse
AbuseFilter
Blocking tool
- Rangeblocks
- Smart blocking- the system would block accounts who used an IP address with the same OS and browser as the person who made the contentious edit instead of just same IP address
- User page or topic blocks (instead of full wiki site blocks)
Feed to flag formerly open proxies to be unblocked
RangeContribs tool
User Interaction History tool
Improve Autoconfirmed criteria, aims to prevent abusive bot accounts
Mute
- Third