IRC office hours/Office hours 2012-07-31
| Day | Date | Time | Location | With | Topic(s) |
|---|---|---|---|---|---|
| Tuesday | 2012-07-31 | 17:00 UTC until 18:00 UTC | #wikimedia-officeconnect | User:Tbayer (WMF) and User:Christine (WMF) | The next Wikipedia Editor Survey |
(Timestamps are in UTC-7.)
Jul 31 09:59:24 * Philippe has changed the topic to: IRC office hours | Community Survey
Jul 31 09:59:31 * Philippe has quit
Jul 31 09:59:47 * preilly has joined #wikimedia-office
Jul 31 09:59:47 * preilly has quit
Jul 31 09:59:47 * preilly has joined #wikimedia-office
Jul 31 10:02:26 * IWorld has joined #wikimedia-office
Jul 31 10:02:44 <ChristineM> Okay folks i think we're about to get started
Jul 31 10:03:01 <IWorld> :)
Jul 31 10:03:10 * mysterytrey has joined #wikimedia-office
Jul 31 10:03:11 <ChristineM> I'm Christine, I worked with the Foundation last year in the Community department, and I've come back to work on this new Editor Survey
Jul 31 10:03:59 <Fluffernutter> aha, so you ARE an evil minion again!
Jul 31 10:04:04 <ChristineM> only kind of :)
Jul 31 10:04:05 * rneumann has joined #wikimedia-office
Jul 31 10:04:10 * jvandavier has joined #wikimedia-office
Jul 31 10:04:12 <ChristineM> i'm a contractor, here for a limited time only!
Jul 31 10:04:43 <ChristineM> act now!
Jul 31 10:04:43 <IWorld> ah
Jul 31 10:04:46 <ChristineM> anyway :)
Jul 31 10:05:12 <ChristineM> and I'm working with HaeB on getting the survey up and running
Jul 31 10:05:14 <IWorld> So, what are our points?
Jul 31 10:05:25 <IWorld> of this office hour
Jul 31 10:05:33 * dungodung|away has quit
Jul 31 10:05:46 * jorm has quit
Jul 31 10:05:58 <ChristineM> IWorld: we're here to talk about the new Editor Survey... answer questions, talk about the goals of the survey, etc.
Jul 31 10:06:08 <IWorld> I see
Jul 31 10:06:15 * thekaryn has quit
Jul 31 10:06:17 <ChristineM> https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012
Jul 31 10:06:24 <ChristineM> is the link for the info on the survey
Jul 31 10:06:31 * erry has left #wikimedia-office
Jul 31 10:06:31 * dungodung|away has joined #wikimedia-office
Jul 31 10:06:32 * lyzzy has joined #wikimedia-office
Jul 31 10:06:34 * jorm has joined #wikimedia-office
Jul 31 10:07:16 * jorm has quit
Jul 31 10:07:18 * DarTar has quit
Jul 31 10:07:31 <HaeB> Hi, I'm Tilman Bayer from the WMF's GLobal Development department (https://wikimediafoundation.org/wiki/User:Tbayer )
Jul 31 10:07:58 * tfinc has joined #wikimedia-office
Jul 31 10:08:09 <HaeB> So to recap, this survey is the third in a series that the foundation began last year, as part of the strategic plan
Jul 31 10:08:18 <HaeB> to take the pulse of the community, identify problems, inform decisions, etc.
Jul 31 10:08:25 <rneumann> Hi, I'm Beckie Neumann. I'm a research assistant helping Tilman with the survey
Jul 31 10:08:27 <HaeB> here are our blog postings about the results of the last survey: https://blog.wikimedia.org/tag/december-2011-editor-survey/
Jul 31 10:08:47 <Nemo_bis> hi
Jul 31 10:08:52 <HaeB> Beckie is our qualtrics wrangler ;) (the software used for the survey)
Jul 31 10:08:54 <ElizabethMH> To introduce myself, I'm Elizabeth. I helped design some of the new questions.
Jul 31 10:09:15 <HaeB> The last surveys were conducted by Mani Pande and Ayush Khanna from the GLobal Dev department, i'm basically building on their work
Jul 31 10:09:34 <HaeB> and a lot of the question fro the upcoming survey are taken from the last one
Jul 31 10:09:52 <HaeB> so that we can conduct longitudinal analysis, identify trends etc
Jul 31 10:10:06 <HaeB> and, to finish the introduction..
Jul 31 10:10:14 <Fluffernutter> (I'm multitasking outside this window, checking back in regularly...please feel free to ping me if there's something i definitely want to look at or respond to)
Jul 31 10:10:26 <HaeB> ..it is intended to run the surveys more frequently from now on, probably quarterly
Jul 31 10:10:34 <HaeB> with different topics
Jul 31 10:10:47 <HaeB> (we're also happy to take suggestion for the upcmong one in the fall)
Jul 31 10:10:54 * gwicke has joined #wikimedia-office
Jul 31 10:11:00 * xlorm has quit
Jul 31 10:11:03 <Nemo_bis> HaeB: also for sister projects then, I hope
Jul 31 10:11:05 <HaeB> thjs time, emphasis is on community's satisfaction with the work wmf is doing
Jul 31 10:11:19 <HaeB> ok, nemo is starting the questions phase, good..
Jul 31 10:11:29 <HaeB> well, so far it has focused on wikipeida
Jul 31 10:11:48 * Mh7kJ has joined #wikimedia-office
Jul 31 10:11:49 <HaeB> but for this one, we want to include at least commons, too
Jul 31 10:11:55 <HaeB> for the satisfaction part
Jul 31 10:12:06 * thekaryn has joined #wikimedia-office
Jul 31 10:12:09 * awjr has joined #wikimedia-office
Jul 31 10:12:21 <HaeB> the other questions are too much wp-specific to change them without losing consistency with the previosu survyes
Jul 31 10:12:21 <Nemo_bis> and then? no survey ever tried to understand the sisterprojects' communities
Jul 31 10:12:24 <Nemo_bis> and editors
Jul 31 10:12:38 <Nemo_bis> sure, should be an entirely new survey probably
Jul 31 10:12:39 * LLogan has joined #wikimedia-office
Jul 31 10:12:47 <Nemo_bis> I'd like to know who's in charge of deciding this
Jul 31 10:13:37 <HaeB> so the draft for the present survey is a colaborative work of wmf staff from various departments
Jul 31 10:13:41 <Nemo_bis> (I know it's not your "fault")
Jul 31 10:13:44 * Beria_ has joined #wikimedia-office
Jul 31 10:13:59 <Nemo_bis> Yes but who decides what kind of srvey to run?
Jul 31 10:14:19 <Nemo_bis> Just so that I can bother the right people rather than you :)
Jul 31 10:14:37 <HaeB> various c-level staff, this time ;)
Jul 31 10:14:43 * Fluffernutter is now known as Fluff|brb
Jul 31 10:15:04 <HaeB> yeah i'm focusing on runnning the surveys, together with beckie
Jul 31 10:15:21 <Nemo_bis> what c-levels? :)
Jul 31 10:16:00 <HaeB> and personaly i'm pretty open on what kind of questions to taking onboard (we also thought about invitng researchers to suggest questons next time)
Jul 31 10:16:06 <HaeB> mostly all of them
Jul 31 10:16:11 <Nemo_bis> ok
Jul 31 10:16:20 <Nemo_bis> Anyway, I don't want to take too much time
Jul 31 10:16:22 <HaeB> so any feedback specifically about the current question set?
Jul 31 10:16:38 <Nemo_bis> I didn't ask any question or have more comments after reading them :)
Jul 31 10:16:48 <Nemo_bis> I've seen several improvements though
Jul 31 10:17:02 * mysterytrey is now known as iDangerMouse
Jul 31 10:17:10 * Beria has quit
Jul 31 10:17:12 * iDangerMouse is now known as mysterytrey
Jul 31 10:17:14 <Nemo_bis> 1) I'd like to point out that this is still relevant for me https://strategy.wikimedia.org/wiki/Editor_survey_feedback#Performance
Jul 31 10:17:14 <HaeB> yeah we know that some questions were criticized last time
Jul 31 10:17:16 <ChristineM> you can view the questions here if you haven't seen them yet: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012/Questions
Jul 31 10:17:37 * dungodung|away has quit
Jul 31 10:17:38 <Nemo_bis> 2) Maybe you want to comment why chapters are no longer mentioned (I know, I suppose, but it's worth noting)
Jul 31 10:18:04 <HaeB> some of those actually arent' in this edition any more (mostly because it was questions where a one-time answer was enough, and folowing trends wasn't too important)
Jul 31 10:19:00 <Nemo_bis> I appreciate the "99 I don’t know" option btw
Jul 31 10:20:03 <HaeB> so that question actually still retains the part about chapters
Jul 31 10:20:37 * Fluff|brb is now known as Fluffernutter
Jul 31 10:21:04 <Nemo_bis> oh, right, there's still that one
Jul 31 10:21:06 <rneumann> question Q10 asks about participation in chapter work
Jul 31 10:21:17 <HaeB> but yes, the emphasis this time is on satisfaction/dissatisfaction with wmf ;)
Jul 31 10:21:23 <rneumann> it asks how often people have done this activity in the past 30 days
Jul 31 10:21:50 <HaeB> again, we are already thinking forward to the next survey where we will have room for other things
Jul 31 10:22:51 <HaeB> one person from an european chapter suggested to ask "Please tell us what you think were the top three/five/few achievements/contributions of [your country's] chapter in the past year. "
Jul 31 10:23:27 <HaeB> i think that generates technical problmes (it's hard to evaluate free-form answers if you have >6000 respondents)
Jul 31 10:23:44 <HaeB> but we may look to include something similar
Jul 31 10:23:50 <Nemo_bis> it would be interesting to include the achievements of everyone
Jul 31 10:23:56 <Nemo_bis> and then see what are they
Jul 31 10:24:22 <Nemo_bis> most people will see something they like or dislike but don't know nor care who's the responsible
Jul 31 10:27:09 * Jan-Bart has joined #wikimedia-office
Jul 31 10:27:30 <ChristineM> well, like Tilman said, a big focus of the survey is on Satisfaction/Dissatisfaction with the Foundation
Jul 31 10:27:38 <ChristineM> that's the part Elizabeth and I worked on mostly
Jul 31 10:28:21 <ChristineM> (those are the questions in Sectin V)
Jul 31 10:28:24 <ChristineM> Section too.
Jul 31 10:28:26 <ChristineM> i can spell. really.
Jul 31 10:28:53 * dungodung|away has joined #wikimedia-office
Jul 31 10:29:56 * Amgine has joined #wikimedia-office
Jul 31 10:30:28 <ChristineM> we're hoping to find out how the community feels about the work the Foundation is doing, and how supported the community feels by the Foundation
Jul 31 10:31:19 * Ironholds ducks
Jul 31 10:31:41 <lyzzy> when will the translation process start?
Jul 31 10:31:55 * ChristineM eyeballs Ironholds
Jul 31 10:32:03 <HaeB> this week, hopefully
Jul 31 10:32:29 <HaeB> meaning, if we don't identify any major concerns or roadblocks now
Jul 31 10:32:35 <Amgine> Is the survey instrument one which has been used by others? vetted and normalized?
Jul 31 10:32:36 <HaeB> with the questionaire
Jul 31 10:32:59 <rneumann> we used qualtrics in the november survey and were pretty happy with it
Jul 31 10:33:13 <rneumann> we determined that it was better than lime survey which was used for the april one
Jul 31 10:33:14 <lyzzy> ok
Jul 31 10:33:16 <rneumann> but had security concerns
Jul 31 10:33:24 * xlorm has joined #wikimedia-office
Jul 31 10:33:39 * DarTar has joined #wikimedia-office
Jul 31 10:33:47 <HaeB> https://en.wikipedia.org/wiki/Qualtrics
Jul 31 10:33:48 <Amgine> That's a survey serving service. It is not the survey *instrument*, which is the actual questionaire.
Jul 31 10:34:54 <HaeB> Amgine, so you are asking whether the questionnaire has been adapted from other surveys?
Jul 31 10:35:14 * StevenW has joined #wikimedia-office
Jul 31 10:35:25 <Amgine> Yes, preferably ones which have been normalized and which have statistical validity.
Jul 31 10:36:41 * kkay has quit
Jul 31 10:37:08 <ElizabethMH> Hi Amgine, I'm a methodologist.
Jul 31 10:37:16 * kkay has joined #wikimedia-office
Jul 31 10:37:28 * tewwy has joined #wikimedia-office
Jul 31 10:37:28 * tewwy has quit
Jul 31 10:37:28 * tewwy has joined #wikimedia-office
Jul 31 10:37:29 <HaeB> obviously there are some standard techniques (e.g. we used the net promoter tool for one question)
Jul 31 10:37:31 <ElizabethMH> We carefully adjusted the likert scales to make sure they would give us useful data
Jul 31 10:37:37 <HaeB> (https://en.wikipedia.org/wiki/Net_Promoter )
Jul 31 10:37:38 <ElizabethMH> if that is what you mean
Jul 31 10:38:34 <Amgine> Well, yes, but iir the last qualtrics survey the vast majority of the questions were scalar and were (to me) almost certain to get 1 or 5s.
Jul 31 10:39:11 * PPena has joined #wikimedia-office
Jul 31 10:39:21 * Beria_ has quit
Jul 31 10:39:54 <ElizabethMH> We are hoping to have more varaition, as that is always ideal
Jul 31 10:40:39 <Amgine> Will you, however, be collapsing into 1, 0, -1?
Jul 31 10:40:44 * Beria has joined #wikimedia-office
Jul 31 10:40:44 <HaeB> Amgine, i don't think you're talking about the last editor survey? https://meta.wikimedia.org/wiki/Research:Wikipedia_Editors_Survey_November_2011/en
Jul 31 10:40:44 <ElizabethMH> Limited variation can still be statistically valid, as long as we don't treat the data as though it is continuous
Jul 31 10:41:22 <ChristineM> Amgine, If you're concerned about questions that ask for a rating on a 1-5 scale, there really aren't any in this particular survey. None that I can think of at any rate.
Jul 31 10:41:44 <Amgine> So, in short, you'll be norming this on-the-fly.
Jul 31 10:41:49 * Tanvir has quit
Jul 31 10:41:52 <ChristineM> (you can view the questions here: https://meta.wikimedia.org/wiki/Research:Wikipedia_Editor_Survey_August_2012/Questions)
Jul 31 10:43:26 * xcombelle has quit
Jul 31 10:43:27 <Amgine> ChristineM: Most of those drafts are 1-5 Likert scales.
Jul 31 10:44:57 <Amgine> Thanks for the link, ChristineM; I withdraw my question.
Jul 31 10:45:53 <HaeB> Amgine, perhaps you were remembering some other survey - wmf has been using qualitrics for quite a few others of them recentlyh
Jul 31 10:46:12 <Amgine> Probably, HaeB.
Jul 31 10:46:37 <HaeB> (e.g. one about funds this spring)
Jul 31 10:47:24 <HaeB> other questions about the survey before we wrap up?
Jul 31 10:49:24 <ElizabethMH> amgine-- most survey data utilizes likert scales of some sort, and there are a lot of options for analysis that do not require normalization a priori. and most data in the world isn't normally distributed anyway (e.g., age, income)
Jul 31 10:50:20 <Amgine> Yes, but as an example, all of your demographic questions have extensive research, and yours are... somewhat unique.
Jul 31 10:51:24 <HaeB> btw, we are going to release the raw data (anonymized where needed for privacy), like here for the first one http://dumps.wikimedia.org/other/surveys/editorsurvey2011/
Jul 31 10:51:47 <ElizabethMH> can you give me an example? i'm trying to make sure i am totally following your question.
Jul 31 10:52:37 <HaeB> what does "having research" mean in that context?
Jul 31 10:52:43 * Beria_ has joined #wikimedia-office
Jul 31 10:53:03 <rneumann> i just looked up the data from the last survey's scalar questions to double check and the results are very skewed towards the extremes
Jul 31 10:53:07 <rneumann> arent*
Jul 31 10:53:13 <rneumann> sorry
Jul 31 10:53:26 <rneumann> responses in the middle range were far more numberous
Jul 31 10:53:32 <rneumann> numerous
Jul 31 10:53:35 <Amgine> Well, for gender you have male, female, transsexual, transgender. My guess is your results will have upwards of a 2% skip rate if that is an option, and I've never seen a sample survey with solely those 4 questions. (usually also do not know and choose not to answer).
Jul 31 10:53:47 <Amgine> But most of my research is in health sciences not social.
Jul 31 10:54:23 <HaeB> hm yes i think that question achieved its current form after some criticism that transsexual/transgender was not included
Jul 31 10:54:46 <HaeB> so it may have departed from its original, more standard form
Jul 31 10:54:56 <Amgine> Right, but you won't be able to compare the results with other normed populations, HaeB, because they way you ask the question is unique.
Jul 31 10:55:29 <ElizabethMH> we won't be able to make a *direct* comparison to the census, for example
Jul 31 10:55:35 <HaeB> yes good point, but that hasnt really been a focus so far
Jul 31 10:55:53 <Amgine> Simple observations are cool, but having *similar* observations allows you to know more about the population because you can leverage what other research has been down.
Jul 31 10:56:34 * Tanvir has joined #wikimedia-office
Jul 31 10:56:50 * Beria has quit
Jul 31 10:57:08 * mpeel has joined #wikimedia-office
Jul 31 10:58:55 <HaeB> in general, as i said, we have to balance between optmizing the questions now (so we may get more significant results thsi time, or ones that are more comparable to external data), and keeping them consistent with the previous surveys, so that we can track trends
Jul 31 10:59:10 <HaeB> one last point, we would like to provide more language-specific reports in the future
Jul 31 10:59:16 <HaeB> i.e. focusing on one project
Jul 31 10:59:51 <HaeB> (which will be a smaller sample of course, but should still yield interesting insights fo rthat particular community)
Jul 31 11:00:14 * LLogan has quit
Jul 31 11:01:09 <ChristineM> so it looks like we're at the end of our time
Jul 31 11:01:17 <ChristineM> unless there are any last minute pressuring questions
Jul 31 11:01:25 * Beria has joined #wikimedia-office
Jul 31 11:01:29 <ChristineM> we'll post the log of this office hour on Meta, as they always are
Jul 31 11:01:37 <HaeB> thanks everyone!
Jul 31 11:01:42 <ChristineM> and I want to thank everyone for coming and participating!
Jul 31 11:01:46 <HaeB> and the talk page is still open for feedback
Jul 31 11:02:20 <ElizabethMH> Bye everyone! and thank you for your thoughtful comments.
Jul 31 11:02:22 <rneumann> ok thanks for the input, bye everybody
Jul 31 11:02:42 * lyzzy has left #wikimedia-office
Jul 31 11:02:45 * ElizabethMH has quit
Jul 31 11:02:50 * rneumann has quit
Jul 31 11:03:06 * ChristineM has left #wikimedia-office