This page is part of the Proceedings of Wikimania 2005, Frankfurt, Germany.

0	MISSING

1	Submitted

2	Editing

3	Author review

4	Final edit

5

DONE

Too Many Cooks Don't Spoil the Broth

Author(s):'' {{{...}}}
License: Andreas Brändle
Slides: Andreas Brändle
Video: {{{radio}}}
'Note:' ?

About the slides: {{{agenda}}}

About license

Contact the author

<include>[[Category:Wikimani templates{{#blocked:}}]]</include>

Introduction

The relevance of wiki systems is increasing. Wikipedia, other Wikimedia projects and wiki systems in general grow exponentially in popularity. In the meantime we still do not know much about the quality in wikis, the conditions that cause quality and processes that maintain quality. This research project is an approach to fill this gap from the perspective of communication and media studies. It deploys theories of quality in journalism^[1] and of the attention economy^[2]. It is the intention of this research work to identify some of the factors that cause good quality work in a wiki system.

The topic-attention-quality model

Based on attention economic considerations^[3] and inspired from ideas in the Wikipedia community^[4] a basic model called the topic-attention-quality model has been worked out. It describes quality as the dependent variable. Independent are the variables quantifying the process of writing the articles, such as topic relevance and category, number of versions, authors and page views, anonymity or vandalism. This model was the starting point for the empirical research and the hypotheses.

The model proposes three dimensions – topic, attention and quality – that are arranged along the writing process of an article. The first dimension, topic, gathers all the characteristics an article consists of, independently from what the authors have written and the resulting quality. Topics vary in relevance and category. For instance does the topic Berlin bring a much higher relevance to the wiki process than Heukewalde (a small village in Thuringia/Germany). Additionally the topic Berlin falls into the category geography that sharply differs for example from mathematics or literature.

The topic characteristics influence the second dimension, the attention. It gathers all the variables that quantify the attention the wiki contributors and users have brought in - variables such as number of versions, number of unique authors, traffic or discussion intensity. The two variables vandalism and anonymity take also an important part in the wiki process and interact with the attention dimension.

Supporters of the wiki system assume that the mentioned variables and dimensions have an effect on the third dimension, the quality. For instance, it is expected that the higher the number of authors, versions, page views, the better is the quality of the article. This research tries to verify this idea and measures the intensity of influence the topic and the attention have on the quality.

Method

The research is based on a quantitative content analysis. A random sample has been drawn. It consists of 450 articles of the German Wikipedia. There has been data gathered on the article, its version history, the discussion page and history, the authors and the traffic. Due date to draw the sample was October 19th, 2004, because there were no user stats available after this date.

The variables

Topic relevance has been measured with three variables: 1. The number of words Brockhaus writes on the topic (Brockhaus is the most important German Encyclopedia); 2. The Number of Google results; 3. The amount of news values - a method in media studies to assess the newsworthiness of a topic for journalists. After that the articles have been classified into topic categories, sub categories and special categories such as technology, nerd-stuff, sexuality, politics, sports or other.

On the attention dimension the number of versions, authors, page views, discussion words, discussion versions, discussion authors, discussion page views and backlinks have been counted. The age of the article is the difference between the due day and the first version. The variables anonymity and vandalism have been assessed through the version history

The quality was the most difficult set of variables to measure. What is quality? Wikipedians have been discussing the issue of quality since the beginning in 2001. Communication and media studies have a much longer history of discussing the quality issue without having a clear solution to it. This study concentrates the quality assessment on the content dimension. It brings together some approaches from researching the quality of journalism with content analysis and ideas from the Wikipedia community on what a good article is.

Quality variables:

W-questions: Does the article give answers to the questions who, what, when, where, why?
Lead section: Does the article have a lead section that briefly sub- and super-ordinates the topic to other concepts and does it give an impression on its relevance and its history.
Background section: Does the article have a background section that explicates the relationship with sub- and super-ordinated concepts, its relevance and its history.
Degree of cross-linking: Number of internal, external and interwiki links.
Article structure: does the article have a lead, background and commentary section?
Transparency: Number of references in the text, references to book sources and external links.
Number of media objects: pictures, tables, charts and images.
Layout/structuring: to many titles, to little titles, too much listings.
Comprehensibility: Number of non-linked foreign words and technical terms.
Size: number of words.
Objectivity: Are viewpoints marked as such? Are there also arguments that disagree with a stated viewpoint?
Diversity: Number of different viewpoints.
Balance: Balance of different viewpoints.

Data reduction with factor analysis

After the gathering of the data a factor analysis has been conducted. Factor analysis is a statistical technique to reduce the number of observed variables to a few unobserved variables, called the factors. It created five factors:

Relevance: The three variables Brockhaus, Google results and news values resulted in one single factor – the relevance
Interest: The amount of attention given to an article (Variables: Number of versions and unique authors, traffic, age, number of backlinks)
Controversy: The intensity that an article is discussed with (Number of discussion words, discussion authors, discussion edits, traffic discussion)
Richness: Main quality factor, comprehends every dimension of quality, besides the neutrality variables (W-questions, lead section, background section, degree of cross-linking, article structure, transparency, number of media objects, layout/structuring, comprehensibility, size)
Neutrality: Comprehends the neutrality factors (objectivity, diversity, balance)

These data-reduced factors have then been implemented into the basic model. Fig. 2 shows the topic-attention-quality-Model after the data reduction.

The relations between these factors/variables, indicated with arrows, have been researched with multiple regression analysis to uncover the dependencies.

First Results

The data analysis is still under its way. Final results are going to be presented at Wikimania 05. In the following you will find some first results.

Relevance

Topic relevance has a strong and highly significant impact on the factor interest (R square = 0.53, Beta = 0.73) and on the quality factor richness (R square = 0.56, Beta = 0.75), a moderately strong influence on the controversy (R square = 0.1, Beta = 0.44) and a very weak regression with the quality factor neutrality (R square = 0.03, Beta = 0.17). That means that a topic of high relevance is very likely to reach a good quality (richness). In the meantime the topic relevance isn’t a reliable predictor for neutrality. Additionally more relevant topics are also attracting more unique authors.

Attention

The attention factor interest itself has a strong influence on the quality factor richness (R square = 0.54, Beta = 0.74). In a multiple regression model the factor topic relevance and the attention factor interest already explain 64 Percent (R = 0.80, R square = 0.64) of the quality variability, which is a very strong regression. These two factors alone already predict most of the quality.

Out of the variables that constitute the factor interest (number of version, authors, page views, article age, backlinks), the number of authors is the far most important variable to predict the quality. Together with the relevance factor it also explains 64 Percent (R = 0.80, R square = 0.64) of the quality variability.

File:Authors and richness.jpg

Fig. 3: The relationship between the number of authors and the quality richness

The factor controversy that measures the discussion intensity of the articles doesn’t have a strong influence on neither the quality (richness) nor the neutrality, if the influence of interest and relevance is subtracted. The same is true for vandalism, anonymity and the topic category.

References

Quality in communication and media studies (sample)^

Beck, K., Schweiger, W. & Wirth, W. (2004). Gute Seiten – schlechte Seiten. Qualität in der Onlinekommunikation (S. 7-14). München: Verlag Reinhard Fischer.
Bucher, H. J. (2000). Publizistische Qualität im Internet. Rezeptionsforschung für die Praxis. In K. D. Altmeppen, H. J. Bucher & M. Löffelholz (Hrsg.), Online-Journalismus. Perspektiven für Wissenschaft und Praxis (S. 153-172). Wiesbaden: Westdeutscher Verlag.
Fabris, H. H. (2001). Qualität als Gewinn. Salzburger Beiträge zur Qualitätsforschung im Journalismus. Innsbruck: Studienverlag.
Hagen, L. M. (1995). Informationsqualität von Nachrichten. Messmethoden und ihre Anwendung auf die Dienste von Nachrichtenagenturen. Opladen: Westdeutscher Verlag.
McQuail, D. (1992). Media Performance. Mass Communication and the Public Interest. London: Sage.
Neuberger, C. (1997). Was das Publikum wollen könnte. Autonome und repräsentative Bewertung journalistischer Leistungen. In H. Wessler, C. Matzen, O. Jarren & U. Hasebrink (Hrsg.), Perspektiven der Medienkritik (S. 171-184). Opladen: Westdeutscher Verlag.
Neuberger, C. (2003). Onlinejournalismus: Veränderung – Glaubwürdigkeit – Technisierung. Media Perspektiven o Jg. (3), 131-138.
Pöttker, H. (2000). Kompensation von Komplexität. Journalismustheorie als Begründung journalistischer Qualitätsmaßstäbe. In M. Löffelholz (Hrsg.), Theorien des Journalismus (S. 375-390). Wiesbaden: Westdeutscher Verlag.
Rager, G. (1994). Dimensionen der Qualität. In Bentele, G. & Hesse, K. R. (Hrsg.), Publizistik in der Gesellschaft. Konstanz: UVK, S. 189-210
Ronneberger, F. (1977). Ausgewogenheit publizistischer Aussagen. Publizistik 22(2), 138-149.
Ruß-Mohl, S. (1992). Am eigenen Schopfe... Qualitätssicherung im Journalismus - Grundfragen, Ansätze, Näherungsversuche. Publizistik 37 (1), 83-96.
Ruß-Mohl, S. (1994). Der I-Faktor. Qualitätssicherung im amerikanischen Journalismus – Modell für Europa? Zürich, Osnabrück: Edition Inferion.
Schröter, D. (1995). Qualität und Journalismus: Theoretische und praktische Grundlagen journalistischen Handelns. München: Verlag Reinhard Fischer
Schulz, W. (1976). Die Konstruktion von Realität in den Nachrichtenmedien. Analyse der aktuellen Berichterstattung. Freiburg/München: Alber.
Weiß, R. (1997). Lässt sich über Qualität streiten? Versuche in der Kommunikationswissenschaft zur Verobjektivierung des Qualitätsbegriffes. In H. Wessler, C. Matzen, O. Jarren & U. Hasebrink (Hrsg.), Perspektiven der Medienkritik (S. 171-184). Opladen: Westdeutscher Verlag.
Wallisch, G. (1995). Journalistische Qualität. Defintionen-Modelle-Kritik. Konstanz: UVK
Wyss, V. (2000). Qualitätsmanagement im Journalismus: Das Konzept TQM auf Redaktionsstufe. Medienheft (1.9.2001)

References Wikipedia (sample)^

Ciffolilli, A. (2003) Phantom authority, self-selective recruitment and retention of members in virtual communities: The case of Wikipedia. First Monday: Peer-Reviewed Journal on the Internet [Online]. Erhältlich: http://www.firstmonday.org/issues/issue8_12/ciffolilli/index.html [10. Juli 2004]
Dispatches from the Frozen North (2004). How Authoritative is Wikipedia. [Online] [Weblog-Eintrag] Erhältlich: http://www.frozennorth.org/C2011481421/E652809545/index.html [2. Dezember 2004]
Fuchs, U. (2004). Erste Untersuchung der Vandalismusanfälligkeit der Wikipedia. Essen. [Online] Erhätlich: http://upload.wikimedia.org/wikipedia/de/3/37/UntersuchungVandalismus041204.pdf [13. Dezember 2004]
Leuf, B. & Cunningham W. (2001). The Wiki Way – Quick Collaboration on the Web. New York: Addison Wesley.
Lih, Andrew (2004): Wikipedia as Participatory Journalism: Reliable Sources? Metrics for evaluating collaborative media as a news resource. Paper for the 5th International Symposium on Online Journalism. Hong Kong. [Online] Erhätlich: http://journalism.utexas.edu/onlinejournalism/wikipedia.pdf [7. Juli 2004]
Möller, E. (2003a). Tanz der Gehirne. Telepolis [Online] Erhältlich: http://www.heise.de/tp/r4/artikel/14/14736/1.html
Möller, E. (2005). Die heimliche Medienrevolution. Wie Weblogs, Wikis und freie Software die Welt verändern. Hannover: Heise.
Viégas, F. B./Wattenberg, M. & Kushal, D. (2004). Studying Cooperation and Conflict between Authors with history flow Visualizations. MIT [Online] Erhältlich: http://web.media.mit.edu/~fviegas/papers/history_flow.pdf [7. Juli 2004]
Voß, J. (2005). Measuring Wikipedia. In Proceedings of the 10th International Conference of the International Society for Scientometrics and Informetrics.
Voß, J. & Danowski, P. (2004). Bibliothek, Information und Dokumentation in der Wikipedia. In Information: Wissenschaft und Praxis, 8, 457-462.

References Attention Economy (sample)^

Bade, A. (2002). Kritische Anmerkungen zur neuen Ökonomie der Aufmerksamkeit. In K. Hickethier & J. K. Bleicher (Hrsg.), Aufmerksamkeit, Medien und Ökonomie (S. 179-193), Münster: Lit.
Beck, K. & Schweiger W. (2001), Attention please! Online Kommunikation und Aufmerksamkeit. München: Verlag Reinhard Fischer.
Franck, G. (1993). Ökonomie der Aufmerksamkeit. Merkur 47 (534), 748-761.
Franck, G. (1998). Ökonomie der Aufmerksamkeit. München/Wien: Carl Hanser Verlag.
Goldhaber, M. H. (1997). The Attention Economy and the Net. First Monday [Online] Erhältlich: http://firstmonday.org/issues/goldhaber/index.html [26. Oktober 2004].
Hickethier, K. & Bleicher J.K. (2002), Aufmerksamkeit, Medien und Ökonomie. Münster: Lit.
Hummel, J. & Schmidt J. (2001). Ökonomie der Aufmerksamkeit – eine neue ökonomische Theorie? In K. Beck, K. & W. Schweiger (Hrsg.), Attention please! Online Kommunikation und Aufmerksamkeit (S. 39-55), München: Verlag Reinhard Fischer.

Citations

Sunir Shah's presentation, Transwiki:Wikimania05/Presentation-SS1, Controversy and stability: How wikis have productive conflict, will refer to this paper.

[1]

[2]

[4]