Talk:Survey Support Desk

Want your survey reviewed? Need help with strategy and your survey? Post your questions here!

Title

The title seems too generic... Nemo 09:11, 28 October 2015 (UTC)

Hey Nemo - do you mean "Survey Support Desk" or "Surveys" for the link? Survey Support Desk is too long, especially if there will be child pages. I figure Surveys would be a good option since its for all surveys, whether WMF or not WMF. Ideas? --EGalvez (WMF) (talk) 17:53, 30 October 2015 (UTC)--

Hi Nemo - just pinging you about my question. Thanks.--EGalvez (WMF) (talk) 00:57, 4 February 2016 (UTC)

I think "Survey help" or similar would be a better title. This should be a disambiguation page for surveys. I keep getting caught by this page and I'm forced to click the category, go to the upper category and then down again to what I'm really interested in. --Nemo 09:22, 24 June 2017 (UTC)

Posted a move proposal at Meta:Proposed page moves. --Yair rand (talk) 01:08, 14 September 2018 (UTC)

Survey stats

WMF Metrics and activities meetings/Quarterly reviews/Community Engagement, October 2015:

• Lila: how many surveys did we run over the year?
• Rose: over 40

I have a few questions about that:

• 1) What were these over 40 surveys? I'd like to see a list with subject, links, date, number of participants and returns, responsible owner of the respective survey.
• 2) What results and what benefits did these surveys have? Anything tangible for Wikipedia?
• 3) How much volunteer time was spent on these surveys? (A 20 minute surveys on 150 volunteers would be 50 hours of volunteer time cost.)
• 4) How much staff time was spent on these surveys?
• 5) How do you prevent survey fatigue? How do you notice survey fatigue setting in and biasing results?

Thank you! --Atlasowa (talk) 14:31, 8 December 2015 (UTC)

Hi Atlasowa -- Thanks for your questions. About number 1, this was a misspoken. I did a short project to map out all the surveys and consultations that the WMF has done in the past and there were over 40 (of course!), so this comment in the minutes was confusion. About survey fatigue, the WMF is actively thinking about this. I'm testing out a calendar on meta at the moment for staff to use at the Community Engagement/Calendar, to try to make sure our work doesn't overlap. Since its very new, as you can see, december has quite a few projects happening. But thankfully, WMF teams are starting to use it more. This is an experiment, so we'll see how it goes. If its successful, I will see how we can share it more broadly and even translate in the next couple of months. Also - out of curiosity - why are you interested in surveys? What are your thoughts about surveys that the foundation does? Any thoughts comments on what the WMF does well or what it doesn't do well with surveys? Ideas for what we can do? Thanks! --EGalvez (WMF) (talk) 07:24, 10 December 2015 (UTC)

HI Atlasowa - was I able to answer you question from december? Thanks --EGalvez (WMF) (talk) 00:56, 4 February 2016 (UTC)

Recommendations for future surveys

In light of numerous survey results posted by the Wikimedia Foundation, where the survey data seem to suggest out-of-whack "findings" that appear to be projectable to a larger population, but are not thus projectable, because the sample was not set up as a random probability selection, there are numerous observers who would like to request that the Wikimedia Foundation make more effort to design surveys that are based on random probability samples of readers and editors, rather than an "open call" for anyone interested in responding. - Thekohser (talk) 00:03, 4 February 2016 (UTC)

Thanks Thekohser for the post. This is something we are working to learn and figure out how to address. For example, some teams have started to use talk page messages, rather than banners to create a more representative sample, and with a high response rate. See the Research:Tech_support_satisfaction_poll as an example. Do you conduct surveys? I'm wondering also if you are a member of an affiliate or a functionary on a project? Thanks! --EGalvez (WMF) (talk) 17:26, 2 March 2016 (UTC)

Yes, EGalvez (WMF), I conduct surveys. That is my profession, and over the past 25 years I have authored and fielded at least 1,500 different survey instruments against both probability and non-probability samples. I apologize, but I am unclear on what you mean by asking if I am "a member of an affiliate or a functionary on a project". - Thekohser (talk) 13:37, 22 March 2016 (UTC)

Thanks Thekohser (talk) - Wow - that is a lot of surveys. Are you a researcher? I think I was more broadly wondering what you do on the wikis - are you an admin? another role? Or if you are part of a (chapter, user group, etc.)? Thanks! --EGalvez (WMF) (talk)

I am a market research practitioner for a Fortune 50 company. I am an occasional volunteer editor on Meta, Commons, Outreach, Wikibooks, Wikiversity, Wikivoyage, Wikiquote, and other Wikimedia wikis. I own and maintain a publicly-edited wiki with over 70,000 pages. I am indefinitely blocked on English Wikipedia because of a policy dispute that I had with a Wikimedia Foundation board member and a few of his most avid supporters, even though he eventually apologized for how he initially engaged with me. However, this block has not prevented me from conducting periodic experimental content analyses on English Wikipedia, such as my exercise that found that with simple techniques, in 63% of cases of phony information being inserted into Wikipedia, it will persist for at least a week (link). I sense that you are potentially asking me to volunteer my professional acumen to aid the effort here at WMF to create more credible market research surveys. Is that a correct assumption, User:EGalvez (WMF)? - Thekohser (talk) 20:56, 22 March 2016 (UTC)

Please see how carefully Pew tries to obtain a representative sample, and how many adjustments they make. There are all sorts of separate issues involved with the Wikipedia userbase, and nobody is expecting the WMF to show the same level of care as Pew, but there is no reason to not at least be aware of the many issues involved and make an effort to address them. In particular, if indeed the problem is as bad, there is no reason to not use a reputable and/or experienced survey designer to get a good handle on the problem. Kingsindian (talk) 05:43, 4 February 2016 (UTC)

This is very useful. Thanks Kingsindian - I've added this to a learning pattern. I think it is hard to adjust or weight the results of surveys when we have gaps in knowledge of demographic proportions of users. The best data we have about demographics are surveys that use poor techniques, so its a major challenge for us. Do you use surveys? Wondering if you are involved in any affiliate or are a functionary on a project? Thanks! --EGalvez (WMF) (talk) 17:29, 2 March 2016 (UTC)

Is this page only about "Qualtrics"?

Hello everybody :)

In order to avoid to rely only on proprietary solutions, I'm contributing to this initiative: Wikimedia Italia/LimeSurvey.

My questions:

• is the Survey Support Desk only about Qualtrics?
• how do you suggest to promote that initiative and similar ones? maybe converting this page as a kind of index of tools

I would be glad to invest my spare time to free as much people as possible, trying to be pro-active and give credible alternatives.

Thank you so much for your help :) --Valerio Bozzolan (talk) 18:17, 23 February 2021 (UTC)

I've edited the page linking LimeSurvey initiatives :) --Valerio Bozzolan (talk) 07:37, 9 March 2021 (UTC)

Please publish survey questions from previous 2023 Community Insights survey

Hi User:TAndic (WMF) and anyone else that may be able to answer this. I am a researcher and community member requesting that the WMF publish previous survey questions for the Community Insights surveys. In particular, it would be most useful to see the questions you asked for the Community Insights/Community Insights 2023 Report. My main question “for the first time, we asked respondents whether they belong to an ethnic minority or discriminated group in their country of residence.” But without knowing what question you asked to respondents, its not that helpful. Did you ask people "are you part of a discriminated group in your country of residence"? Did you give people enough information to know how to respond if they have a disability, which is a category of discrimination for instance? The Community Insights/2024 Survey Questions were published, so it makes sense to publish questions from earlier surveys so we can compare. Thanks for responding in advance. Hexatekin (talk) 18:50, 28 January 2025 (UTC)

Hi @Hexatekin, the 2022 survey schedule (the survey data used for the 2023 report) is coincidentally linked in the main text section of the information you quoted, but my apologies for the oversight in not linking it more clearly in the report (usually it gets updated on the main page with each survey wave and somehow until now nobody has asked for the older questionnaires!); I’ll add it to the Methodology section for both the 2023 and 2024 reports. -TAndic (WMF) (talk) 19:37, 31 January 2025 (UTC)

Yes please publish the survey questions in the 2022-2023 methodology. Also, I do feel the naming could be improved in this documentation. It seems like the same survey/report is being referred to as a 2022 and 2023 report, but it's the same survey. Hexatekin (talk) 19:55, 31 January 2025 (UTC)