Grants talk:IEG/WikiProject X/Literature notes

Raw notes

Burke et. al. 2010

“Online, newcomers face special difficulties as a result of the diffuse, decentralized, and anonymous text-based interactions inherent in most online groups.” 7

More replies means more participation. Replies signal value to the newcomer. 7
     A way of granting “membership” to that person. 8
     16.7% of people who got no response posted again, whereas 45.5% who received a reply came back. 20-21

Group oriented membership claims (indicating familiarity with the group and its customs)
Identity-oriented membership claims (I am one of you! I use your language!) – questionable efficacy
Information requests (Having questions answered is perceived as an invitation to participate) – very effective for response

Asking specific questions more likely to elicit a response since there is a clearer call to action. 26

Identifying as a lurker leads to positive response in groups. 29

Han and Farzan 2012

"Editors can explicitly join WikiProjects,
but only a subset of those who edit relevant articles actually
do so. To include a more complete list of editors, we defined
an editor as associated with a WikiProject as long as they
edited at least one article tagged with that WikiProject label.” 

Lack of feedback drives people away from community

Wikipedia editors engage in four primary types of activities: substantive experts, social networkers, counter vandalism, and technical editors

Types of request:
Referent
     Request/provide technical editing
     Request/provide domain information
Appraisal
     Request/provide peer review
     Provide social support
Relational
     Task coordination

JMo’s powerpoint on WikiProjects

Distinction between conventional WikiProjects (on a subject area) and alternative WikiProjects (that provide more general support for Wikipedia, or do a specific task across different subject areas)

A thought: “WikiProjects that are gathered around specific article
subjects might lose steam once coverage of that subject
becomes more complete.”

What’s next for WikiProjects?
     Connect newcomers with projects (à la Teahouse)
     Build better collaboration tools (see HotArticlesBot)

JMo’s dissertation

Organizations become more formal as they grow
Formalization is decentralization (out of individuals) through bureaucracy
Wikipedia grew rapidly, leading to emergence of formal rules, which sacrificed openness for quality

"changes to an existing
open collaboration system that disrupt established social structures can trigger a mass exodus” 22

How did Wikipedia manage to be a coherent community at the size it was?
     Watchlists and edit histories allow editors to be aware of each others’ activities
     Editors are aware they will be scrutinized
     Talk pages
     Administrators who help enforce rules
     Discussion boards
     Personal profile pages and discussion pages
     Barnstars
     WikiProjects

Editing follows the usual power rule (very few doing the most)

The policy environment helps to solidify norms but do not do much by way of social support. This is where WikiProjects get involved.

Open collaboration systems need ways to facilitate contributions despite lower commitment and higher turnover—teams help with this
     Core organizes work for a larger periphery
     Motivation; more socially engaging and pleasant

WikiProjects do not have a formal member structure; the term can refer to the workspace or those who self-identify as members

Problem: editing community is not reflective of the audience. Editors are predominantly male, white, from America or Europe – audience is more global. Substantial coverage gaps.
          Could WikiProjects help ameliorate this by recruiting people?

Despite formalization of policies, WikiProjects remain totally open. No official rules on how WikiProjects work and WikiProjects have no control over their subject areas.

People don’t formally identify with WikiProjects to the same extent but people still do the work of WikiProjects and WikiProjects attract newcomers.

"WikiProjects may become increasingly useful because they often
record extensive, actionable lists of high-priority tasks that remain to be done”

"The inclusivity and flexibility of WikiProjects suggests that they can be used to coordinate a variety of other work activities beyond
improving articles as well, a feature that may allow them to continue to play a valuable role as collaboration on Wikipedia shifts away
from direct editing work and more towards curation and community support activities.”

"Evidence suggests that the socialization mechanisms of Wikipedia have not scaled as effectively as the regulatory mechanisms”

Wikipedia is hard for newcomers. They are reverted more often, so they stick around for less time. Additionally, attrition rate higher for women.

"by creating
a local environment that affords a particular set of experiences and interactions between Wikipedia newcomers and veterans,
I hope to have an impact on the way the Wikipedia community as a whole socializes new members”
          Important: Value in trying to direct the user experience as much as possible. Does it pan out?

Morgan used WebCA (content analysis) and interviews with WikiProject members
     Only did 18 interviews; half were with MILHIST. I don’t understand this approach.

Appendix C describes interview protocol … worth adapting?
Appendix F has survey questions 

Chapter 5: WikiProjects. The good stuff here!

Membership in WikiProjects may be ambiguous; can only members edit articles? Can just anyone become a member? Obligations?
          Declaring oneself as a member of a WikiProject doesn’t really mean shit
          Bold proposition: no explicit concept of membership, “WikiProject-as-tool"? Tradeoffs. Group identification is useful but can cause hostilities

"Platform designers and community members must provide
mechanisms for helping potential contributors find productive and engaging ways to get involved, ensure productivity and project
maintenance despite lower levels of member commitment and higher levels of member turnover relative to compensated and
co-located teams."

" WikiProject talk pages have not been systematically examined in
this way, but findings from multiple studies suggest that editors use talk pages to coordinate a broader range of activities
across many different articles: Choi (Choi, Alexander, Kraut, & Levine, 2010) found evidence that project members used talk
pages to welcome new members, suggest tasks for them to perform and provide constructive criticism; and interview subjects
in Forte (Forte, Kittur, Larco, Zhu, Bruckman, & Kraut, 2012) listed project talk pages as places where members go for help
with articles."

Vast majority of talk page posts were requests of some kind (82% of studied sample)
Most talk page posts are requests for opinions or requests to participate in some other discussion
          "The frequency of messages like this one in our data suggests that many editors work autonomously and only turn to the WikiProject
when they get stuck, need advice, or just want to bounce some ideas off someone"
          Also used as a broad-scale notification system of sorts, for letting people know about something going on

Non-members posted 37% of all messages and initiated 54% of threads. That’s a lot!
          Members and non-members do not behave significantly differently
          Requests from members and non-members equally likely to result in follow-up
          Inter-member affinity is more a result of conversations than identity as a WikiProject participant, but members are more likely to have the         page on their watchlist (meaning more participation in conversations)—a thing that might change with Flow?
                    "Importantly, our findings suggest that such bonds
                    may form through the performance of coordination work within the project workspace rather than in the context of collaborative
                    content editing, even though facilitating content production is the ostensible purpose of many WikiProjects."

Loose relationship between maintaining WikiProjects and editing content; usually, WikiProject maintainers do work on behalf of others who are content producers, answering questions and the like.
          Article editors become work coordinators
          Most content producers are not involved in a collaboration per se but are working independently and using WikiProject for guidance

"Designers and community managers who wish to support WikiProjects or foster similar self-organized volunteer
projects in other open collaboration systems must provide mechanisms for coordination that not only work for large, wellorganized,
close-knit groups but also for groups that are smaller, more open, and less formal because many active and successful
work groups may reflect one or more of those tendencies."

Chapter 6 – Dynamics and Diversity of WikiProjects

Decline of subject-based WikiProjects may be a function of evolving needs: less need to create articles in subject areas, more need for maintenance

"They also found that members appreciated Military History for its social functions: participating
provided opportunities to find new collaborators, get expert help, protect and advertise their work, and network socially."

Project task lists and COTWs do not necessarily work to mobilize editors, who are independently motivated

Alternative WikiProjects—those not based on subject area—are actually doing quite well despite declines in other parts of Wikipedia

WikiProject Unreferenced BLP Rescue: "This
project’s approach—setting monthly targets and efficiently coordinating work around those specific target articles—was successful
in mobilizing participation by a large number of editors to eliminate the existing backlog over the course of 15
months.” Importance of specifically defined task and setting of targets

"One of the main reasons WikiProjects
fail is because they choose an unsuitable scope. The proposal process provides an opportunity for
someone to point that out, and ideally redirect the proposer's energy towards something more productive.” — conjecture, but is it true?

"Many members of the Wikipedia community have suspected for years that their shrinking community is partially due to new
editors having difficulty learning the ropes and having few opportunities to interact with the Wikipedia community in positive
ways, a theory supported by recent research (Halfaker, Geiger, Morgan, & Riedl, 2013; Morgan, Bouterse, Walls, &
Stierch, 2013)” — WikiProjects as vectors for positive interaction? Is that a possibility?

"Taken together with findings from previous research
on coordination in WikiProject talk pages (Morgan, Gilbert, McDonald, & Zachry, 2013), these descriptions of project life
suggest that many Wikipedians do not experience WikiProjects as groups, but rather as interest-based coordination spaces
that can support both independent and collaborative participation."

Design Recommendations
* Tools should allow WikiProjects to maintain flexibility and variety of tasks; WikiProjects are diverse
* Should minimize effort to maintain resources
* What’s core, what’s an add-on?
* Socially intelligent task routing: SuggestBot but expanded to include other tasks; revival of WikiTasks
* Social translucence; visualizing activity. Something that shows most active users. For example, Re:Flex toolbar?
* Leaderboards: CommunityCompare, progress toward group goals. (Doesn’t the article evaluation matrix already do this kind of thing?)


Chapter 7: Designing the Teahouse

Teahouse had two goals:
* Helping newbies
* Very gradually shifting culture on how we treat newbies


* Guests and hosts
* Guests can make profiles
* Designed to reduce barrier to entry
* Some scripts that expand on prior MediaWiki experience but largely is not a substantially different experience than editing Wikipedia itself.
* When started, 20 were invited to be hosts. 15 served, but more ended up becoming hosts
* Teahouse workflow categories: Requesting Technical Help; Negotiating People, Process or Policy; and Finding Collaborators and Ways to Contribute
* Mentorship programs on Wikipedia have operated at small scale, as coordination cost is high
* As part of Teahouse, Python script developed to produce a list of promising newcomers to contact them within 24 hour time window
      "Following previous research on early editing
patterns associated with an increased probability of retention, the report contained a sample of new editors who had joined
within the past 24 hours and had made at least 10 edits, as well as new editors who had joined within the past four days and
made at least 20 edits over the course of three or more edit sessions."

* "A more social approach to Q&A [as opposed to questioners getting answers from answerers] suits the needs
of new users especially well because it affords detailed and personalized explanations."

Teahouse Gadget for asking questions

Threads are shown in reverse chronological order to avoid situation of having the new content below the fold (highlight recent activity!)

"The results of research sprint 5 suggested that new editors who joined WikiProjects often
introduced themselves on the talk page, and that their first contributions to the project were often relatively minor, peripheral
contributions.” — perhaps we could encourage newcomers to leave introductory statements at WikiProjects?
               This functioned like a WikiProject member list but this encouraged people to describe themselves and to select avatars. More engaging than just putting a username on a list

Teahouse established expectations for hosts. People were expected to abide by them.

HostBot helped automate a lot of the activity
* Invitations to Teahouse


Chapter 8: Impact of Teahouse

A large share of survey respondents wanted recommendations for WikiProjects to join and other ways to get involved

What are some of the things that have been challenging for you on Wikipedia?
• Learning the Wikipedia editing interface (59%)
• Understanding Wikipedia policy (49%)
• Interacting with other Wikipedia editors (37%)
• Finding things I want to edit on Wikipedia (33%)
• Other (please specify) (40%)

So issue is not just finding people to work with but other things like learning the editing interface. Many issues converging to cause problems. Wikipedia Adventure, Co-Op can teach technical skills and build one-on-one relationships, WikiProject X can help foster networking

Generic invite template got slightly better (but statistically significant) response rate than personalized. This is surprising!
* No significant difference between manually sent messages and automatically sent messages
* Lightly personalized variant, sent automatically, is the best approach. Scale of automation with link to host who agreed to be available as a personal point of contact

Survey respondents were generally satisfied with the Teahouse, citing its friendly atmosphere and helpful participants.

More women participated than expected but their satisfaction rate was about the same as men's
* Most Teahouse hosts were male. It may be worth looking into interaction dynamics between female guests and male hosts.

Teahouse Q&A got more engagement than standard help desk
* Guests barely answered other people’s questions
* Confusion over how to answer questions to begin with, suggesting a possible interface issue

Big deal: Teahouse guests make more edits and edit for longer and participate more in discussion spaces

Opportunities to hook up the Teahouse and Co-Op with specific WikiProjects who get the redesign

Choi, Alexander, Kraut, Levine (2010)

Socialization Tactics in Wikipedia and Their Effects


Institutionalized socialization helps reduce newcomer anxiety and lessens the likelihood of departure. Institutionalized socialization is socialization that is formal, sequential, fixed, serial, and acknowledges existing skills)

Less formal orientation that is more individualized can lead to a more innovative role but this is less supported

"newcomers did not perceive a
sense of community within Wikipedia, but experienced users
did)"

"Ducheneaut [7], using ethnographic analysis, found that
well-adjusted newcomers engage in identity construction
over time and forge alliances with other group members."

Wikiprojects have pretty much no institutional socialization mechanisms. At all. No training, no curriculum, no mentors, no path for integration. Very informal process based on socialization.
          How can WikiProjects make use of the Co-Op for training?

Wikiprojects use:
* Invitations to join projects
* Welcome messages
* Requests to work on a task
* Offers of assistance
* Positive feedback on work
* Constructive criticism of work
* Personal comments.

The only institutional aspect consistently employed is standardization (e.g. through standard messages)

Invitations to WikiProjects caused people to stop editing at a faster rate than normal. Welcome messages have a better effect. Constructive criticism is very helpful.
          "One reason for the effects of invitations may be that they
     attracted people to the group who would not have otherwise
     joined because they were not sufficiently interested in the
     groups’ goals. The invitation may have introduced a
     temporary boost in motivation whose effects dissipated
     quickly."

"the number of
responses that newcomers received was associated with the
amount they subsequently edited"

"In the
present study, in contrast, standardized tactics were
negatively associated with newcomers’ contributions,
whereas personalized tactics were positively associated with
it."
          Contrast with JMo’s study above, where there is no effect between standard teahouse invitation and personalized one

Online groups work better with personal socialization; offline groups work better with institutionalized socialization

Messages should be personalized

Philippines Pilot

https://blog.wikimedia.org/2013/07/02/english-wikipedia-editor-pilot-philippines/

• Exhorting people to work on articles does not work, even if they do create user pages upon your recommendation
• WikiProjects nonetheless can be good at curating content in need of attention

" If we were to do this project in the future, we might next try setting up a clearer framework to engage new editors in discussion with existing editors from the WikiProject. This would require having enough active editors in the WikiProject interested in onboarding new editors to work on the topics, and using calls to actions to set clear expectations for engagement between new and existing editors."

Don’t recommend expanding articles. Start off with simpler tasks.

Teahouse final report

(Mostly the same as what’s in JMo’s dissertation)

• Social intro: people writing introductions for themselves
• Filling out templates and using images is complicated!

Wikipedia Adventure final report

• Good to be walked through features
• Interactivity is important

Soloman, Wash (2014)

Critical Mass of What? Exploring Community Growth in WikiProjects

When does a WikiProject achieve critical mass? Good question!

Heterogeneity in groups allows for more production. Different people with different skills leads to better, more productive outcomes.

"Velasquez, Wash,
Lampe and Bjornrud (2013) found that people leave online
communities because of policy changes to the site, personal
life changes alter priorities, or because they can no longer
fulfill their perceived role in the community."

Interesting: "High turnover on a site offers the impression that
those who are on a site are active and engaging, which increases
the perception of the site as a social entity."

“[IRC] channels survive longer when more messages are
posted to the channel in its early stages"

"the acceleration of production of
revisions is due to existing community members increasing
their individual production more than increasing production
by adding new contributors."

Critical mass of people is what matters: "Getting
smaller contributions from a larger number of people
leads to larger long term community production than getting
many early contributions from a smaller number of people."

Heterogeneity matters: "We conclude that projects whose members
participate in a greater variety of other WikiProjects
grow faster.” — perhaps we could encourage people to join other WikiProjects? (Not that that would cause heterogeneity to spring out of the ground, but it could lead to increase in WikiProject participation in general)

Having many people is important, but not all people make equal contributions. Power users!

"This model provides additional insight, showing
that revisions made by those who infrequently participate
do in fact help a project grow, but this can be offset by overparticipation
from a project’s power users."

Power users can cause projects to shrink. How is this so??
          Apparently, when people are too heavily involved, it gives the impression that there is no opportunity.
          Small amounts by many different people leads to better long term outcomes than one person doing all the work. Explains why WikiProject Pharmacology is still alive and kicking!
          It is also possible that investment of power users negatively correlates because the power users are trying to compensate for inevitable failure (basically astroturfing)

Wikipedia power users: great for articles, not so good for community. Communities thrive on many people doing a little work