Research talk:Online knowledge sharing

Undergrad project?

I went through the badly-designed survey. Here are some problems:

• Misrepresentation: The description on the Research page does not fairly represent the survey questions. Using Wikipedia to attract sexual partners? Don't be ridiculous.
• Unnecessary/personal questions: No way to opt out, forcing the respondent to invent a false answer. A good survey will accept empty answers as a trade-off for accuracy in the answers actually given. Some demographic information I was asked to provide, I was forced to fabricate rather than leave blank. Repeat over hundreds of respondents, and the data gathered will be useless.
• Selection bias: By spamming established editors to answer, the results will naturally be skewed, invalidating the results (unless that's the intent).
• No disclosure of what meta-data is collected: Cookies, IP address, geolocation, etc. and how it will be used.

I can't help thinking that this is an undergraduate project that has been designed with little or no supervision. 108.65.78.29 03:58, 31 July 2013 (UTC)

This deserves a response. I can't see approving this survey until those deficiencies are addressed. Amatulic (talk) 13:53, 3 August 2013 (UTC)

Agreed. Although I would say that 108.65.78.29 brought up these issues in an overly-abrasive way, they are still important and need addressing. --EpochFail (talk) 23:27, 4 August 2013 (UTC)

Why 500 participants!?

It seems very unlikely that you'll be able to get 500 of the most active Wikipedians to take time away from their work to fill out (yet another) motivation survey. Why do you need so many? --EpochFail (talk) 14:36, 31 July 2013 (UTC)

We aimed to contact the most active contributes because we thought they are the best representatives of what we assume as the online knowledge sharing behavior among Wikipedians. But as you mentioned they happen to be the busiest of all, we omitted "the top 500" and it is now just "500". So any interested contributor can participate.

And the sample size of around 400 is needed to be the representative of Wikipedians population. Also we predict some would not provide accurate results (considered additional 100) and should be omitted.Epistemophil (talk) 21:26, 31 July 2013 (UTC)

400 seems like another magic number? Have you performed a statistical power analysis that suggested 400 would be necessary? Additionally, how will you know which 100 results are inaccurate? --EpochFail (talk) 21:45, 31 July 2013 (UTC)

I replied in an email Epistemophil (talk) 14:20, 1 August 2013 (UTC)

There's no reason that cannot be discussed here. Here's my response to the paper you linked me to:

I looked at the sample size paper that you've shared. I see that 400 observations seems to be the asymptote as the population size approaches infinity, but you aren't sampling from a pool of 1 million active editors. Arguably, if you are polling the most active editors, your sampling the entire set. I don't believe you'll be able to obtain a response from 500 highly active Wikipedians without contacting thousands. In the past, we've seen near 50% response rates depending on the survey request(e.g. Research_talk:Anonymity_and_conformity_over_the_net#Pilot_Study_Results).

This paper also does no answer my question about how you'll know which 100 survey responses are "inaccurate". --EpochFail (talk) 10:28, 4 August 2013 (UTC)

As I mentioned before, I decided to change "top 500" to just "500" participants! So its about any Wikipedian who is interested to participate. (see the Research tab). We just need people who have contributed enough; not the beginners! So roughly, how many individuals participate to the Wikipedia (considering the population would not be the top 500)?

Also we use some criteria to omit fake answers, e.g. at this moment 6 people have chosen 99 as their age! I assume their results as inaccurate. Also Kwiksurvey decodes just the country of participants (but do not provide IP; any pessimist Wikipedian who is concerned about data engineering or so can create a free account and test it himself/herself) and the participant who has provided Afghanistan as his residency country and Kwiksurvey has provided US as his/her country, I assume is not providing accurate responses. Epistemophil (talk) 21:19, 4 August 2013 (UTC)

So you'll be manually removing survey results that you think have been invalidated? What criteria will you use to remove survey responses? It's extremely important that you come up with this criteria beforehand in order to not invalidate your study.

Also, for what it's worth, I imagine that some absurd answers (e.g. age = 99) are an attempt to opt-out of answering the question in a way that you'll find to be obvious so that you can discard that answer, but not the entire set of answers he/she gave. So, by removing them, you'd be biasing your results against those unwilling to share their age. --EpochFail (talk) 23:23, 4 August 2013 (UTC)

As there was a discussion going on Wikipedia:Administrators' noticeboard/Incidents page regarding our survey and everyone was concerned about someone who has contacted top editors and was concerned about the nature of the survey, I believe many tried to test the survey with fake answers. But if the survey be advertised from appropriate channel (your approval), I think people may trust and we would see lesser absurd answers. Epistemophil (talk) 06:40, 5 August 2013 (UTC)

This does not answer my questions or address the concerns I have raised. --EpochFail (talk) 06:51, 5 August 2013 (UTC)

Providing 99 as the age could be the first cue to check that given participant responses. If the provided country differs from that of reported by the survey, that's the second cue. Here I would check some other responses to see if he/she is making data by checking if for example he/she has responded all questions by the same choice. Then I could say with good confidence that this participant has provided absurd responses, and at this stage, I MIGHT omit him/her. Why MIGHT? It still depends on many thing: e.g. is it a same pattern for many participants? If so, then no. How many have participated at all and how many seem to have fabricated the data? etc. Epistemophil (talk) 13:06, 5 August 2013 (UTC)

┌─────────────────────────────────┘

This is a problem. If you're going to perform a statistical analysis, you can't just sort of feel out which surveys to remove. You need to have a methodology for doing for removing responses or you risk introducing a bias to your measurements. --EpochFail (talk) 22:22, 5 August 2013 (UTC)

"[...] the relationship between competitive altruism and online knowledge sharing"

How exactly do you plan to test this? Your methods section should be as explicit as possible. What will you measure? How will you measure it? How will you know when a measured difference is interesting? --EpochFail (talk) 14:39, 31 July 2013 (UTC)

I afraid more explanation would reveal the aim of the study to the participants (which would affect and bias participants' results). I would explain in email if someone is interested to learn more. Epistemophil (talk) 21:40, 31 July 2013 (UTC)

Yes. Please email if you feel that it would invalidate your study to post here. --EpochFail (talk) 21:47, 31 July 2013 (UTC)

I replied in the email Epistemophil (talk) 14:20, 1 August 2013 (UTC)

Why enwiki?

English Wikipedia -- especially the most active editors in Wikipedia -- is overly surveyed. Would you be able to run the same study on Simple English or Commons and get useful results? --EpochFail (talk) 18:40, 31 July 2013 (UTC)

We assume motivations behind contributors to Simple English differs from those of En Wikipedia (e.g. providing information for specific readers), so En is preferred.Epistemophil (talk) 21:31, 31 July 2013 (UTC)

Why is English Wikipedia preferred? --EpochFail (talk) 21:48, 31 July 2013 (UTC)

English is an international language and we do not want to test the behavior of for example Chinese or Italians. So we developed an English survey and we know we would have participants across the globe who contribute to English Wiki, from different countries and backgrounds. That would be something that could be called cross-cultural, but using other Wikis, e.g. Spanish, the study (participant recruitment) is biased. 14:01, 1 August 2013 (UTC)

I asked about Simple English and Commons. --EpochFail (talk) 10:25, 4 August 2013 (UTC)

Are the people in the mentioned sections representative of what one calls "free online knowledge sharing"? If you think it does, then its fine. I need more explanation here. Epistemophil (talk) 20:52, 4 August 2013 (UTC)

They are likely to be just as representative of what you refer to as "free online knowledge sharing" as English Wikipedia. Surely, I'm not the one who should be explaining your study to you. It's up to you to do the background work. --EpochFail (talk) 23:19, 4 August 2013 (UTC)

We'd better to stick with En Wiki, as I think motives behind those Wiki contributors differ. Epistemophil (talk) 06:42, 5 August 2013 (UTC)

I'm sorry, but I don't think that is a good enough reason. Why do you think that the motivations of those wikis differ? And even if that is true, why would it matter for your study? If you are trying to learn something general about "free online knowledge sharing", it ought to apply to all (at least most) spaces where people are freely sharing information online. --EpochFail (talk) 06:54, 5 August 2013 (UTC)

I think the first motivation behind contributor to Simple Wiki is to teach English language (or something language related); the knowledge sharing seems to me to be the second one. And I am not familiar with Commons (Thats why I asked for more explanation). Epistemophil (talk) 12:54, 5 August 2013 (UTC)

┌─────────────────────────────────┘

Still, I remain unconvinced. I suggest that you become familiar with Commons. To answer my question, I propose that you add your research questions to the project page and then explain here why they won't be answerable by surveying the Simple English or Commons populations. --EpochFail (talk) 22:29, 5 August 2013 (UTC)