Research talk:Investigating editing anxiety in new users

Please write any comments about the research here for the researchers

Measuring alters reality

One big issue for newbies is the lack of feedback and communication. This is a big site and we concentrate much of our guidance and feedback on problematic editors, so if someone is doing good work they can find the place isolating and cliquey. I think there is a real risk that this sort of survey could dramatically alter editors perception of feedback and thereby give skewed results. Depending on the wording and the psychology this could be a positive skew - "someone is paying attention" or a negative one with prompted questions starting chains of thought and an awareness that "their main interaction was with a computerised survey. WereSpielChequers 10:38, 5 September 2011 (UTC)

Questionnaires

Please may we see the intended questionnaires. WereSpielChequers 10:52, 5 September 2011 (UTC)

Selection criteria

Most new accounts never complete an edit. What will be the selection criteria for these accounts, 0 edits 1 edit, 1 edit and email enabled? That decision will make a big difference to the research. If the decision is to survey editors who have actually edited then you have the opportunity to limit this to "editors whose first edit was neither vandalism nor spam". I suggest you consider doing that as with a sample size of only 300 there will be little opportunity to drill down many levels, and I believe you will find vandals, spammers and goodfaith editors to be very disparate groups. WereSpielChequers 10:52, 5 September 2011 (UTC)

WMF feedback

I reviewed this proposal with my colleagues at WMF. This research is totally within the scope of the new editor engagement program, but we think it would be hard to integrate the questionnaire into Moodbar (I hope you don't mind me posting on this page, so others RCom members can chime in):

• I don't know whether you would be using only a subset of the questions listed in the Wiki Anxiety/Usability Inventory (a document that was shared privately with WMF), but the questionnaire in its current state (46+22 questions) is way to long to be integrated into Moodbar as a post-feedback call to action (see this page for examples of CTAs used in the context of the Article Feedback feature).
• A much shorter list of questions could potentially be presented via a post-feedback CTA, but the questionnaire would need to be built directly within the interface and this requires allocating more developer resources than we can afford at present. Asking new users for further feedback right after they submitted their mood also sounds potentially distracting, but there might be ways to design this process in a non-intrusive way.
• Recruiting newly registered users for an off-site questionnaire after their very first edit is definitely not an option we can consider.

An infrastructure like Wikimedia Labs would definitely help make this kind of experimentation easier to run, but unfortunately the Labs project is still in its planning phase and it won't see the light before Q1 2012 at the earliest. I hope we can have another conversation soon to discuss possible alternatives to run this study, which is both timely and relevant to our current agenda. --DarTar 00:00, 8 September 2011 (UTC)