Talk:Article validation proposals/Page 1

I like it

I don't believe that we need to separate out the validated articles from the rest of the database, just to flag validated versions in the article history. That eliminates the need for "protected" or "parallel" versions, but allows a link to the "last validated version."

This would require a new field in the history table for "validated versions" of articles, and probably a separate user status of "validator access," for those approved to serve on validating committees.

One additional possibility, which might be helpful in dealing with subtle vandalism: have a button that validators can use to say "I validate this article,"; at the bottom of the page, we could print "This article has been validated by XX editors," or "This article revision has not yet been validated. See last validated version." - Seth Ilys 19:14, 8 Jun 2004 (UTC)

I accept and agree with both of Seth's proposals as a means of improving my original suggestion. Danny 02:22, 9 Jun 2004 (UTC)

An interesting proposal, which sounds partly useful when Seth presents it. I strongly agree that we should not have parallel versions.... Ward said Wikipedia will always be a wiki, and he is correct in that. (I won't imply that it's thereby useless as an encyclopedia). We can only push the quality higher; people using our content must still accept it's a wiki.

One small note: we should validate correct and well-written content, not quantity (certainly not the way FA requires thoroughness of articles). For example, en:Palantìr could be validated; it's stable, accurate and well-written... but not thorough enough to be a featured article. ✏ Sverdrup 19:36, 8 Jun 2004 (UTC)

I'm not opposed to validation per se, but I'm not so sure about the concept of committees--it seems rather insular. I like the individual voting a la "This version has been validated by X editors", though. Meelar 12:51, 9 Jun 2004 (UTC)

Arguably, each of us 'validate' the entries on our Special:Watchlist every time we sign on, at least I know I do and I suspect many others do to. We look at the history between the last edit we made and the current version and either appreciate new information, revert vandalism (all too common, sadly), correct tyops or grammar, or run a check on what has been added to see if we can confirm it; thus each time we check our watchlist we 'validate' those pages about subjects we 'know'. Sometimes we don't make any new edits to that 're-validated' page but it could be nice to be able to re-sign the thing so we (and others maybe) know we've checked it out. Basically one needs a flag that can be set but which is automatically deleted when there is a new non-flag edit to the page. --VampWillow 11:37, 5 Dec 2004 (UTC)

I don't like it

What will not work is a system where only "normal" article versions are validated and marked. Suppose one expert throws out noise from an article (say a long explanation why the earth could be flat from the Earth) and validates this version. The next experts thinks: Well, we could add still a litte here, changes the article and votes for the next version. Then the flat-earth guy comes back and puts his stuff in again. This is wiki. If a third expert wants to encorporate her little changes, she has to consolidate the last expert's changes and hers with the flat-earth-thing again. She produces a third version. Do the other two experts have to change their vote now to that third version? Its too complicated. If validation is needed (which I oppose, at least in any form too close related with wikipedia), a seperate workspace is needed where the experts can finalize an article and throw it back into Wikipedia after its ready. But that particular Wikipedia version does not need to be flagged (within WP), since it gets overridden by a new version within days, and since the expert version is available in the seperate workspace. Uli 08:09, 9 Jun 2004 (UTC)

I am rather opposed to this idea. I think that one of Wikipedia's greatest characteristics is that it is forever ongoing. Paper encyclopedias are static, only updated once a year. The Wikipedia is updated countless times a day. However, there is one possible way I could see myself supporting this "validation" idea: Continuously, committees or some other system check articles. Every article that is "validated" has a subpage, like Articlename/validated or something of the sort, where the latest validated version is posted and protected. That way, the article can always be updated, and from time to time, it will be checked, and the validated version can be updated. I am against any page in the article namespace being protected for validation. There could be a link somewhere on the main page like "View latest validated version of this article." or something of the like. We would have a live version and a validated, protected version. That's all. blankfaze | •­• 03:06, 13 Jun 2004 (UTC)

I'm also against any kind of special validation process,especially those which make changes to the database of software necessary. (see #Natural validation process, above --Ed.) Wikipedia is still a very free collaboration and we must not endanger it by dividing our contents into good/bad or dividing our volunteers into editors/validators. This is a step into a unwiki direction, and completely unnecessary if we divide the big print edition into small pieces. --TomK32 WR Internet]] 08:19, 14 Jun 2004 (UTC)

I am likewise opposed to this process in general. --Burgundavia

More on the purpose of validation

see this definition of a validated article

The idea is not to make a "perfect" article, this is more about avoiding to print just plainly poorly spelled, with facts unchecked articles. Right now, anyone downloading wikipedia is also downloading a good bunch of very poor articles. Anthere

• Well this goes a bit into the philosophical direction about facts and knowledge. I don't believe there is a real "true" or "false". And we should not rate articles for truth, which can never be really determined eitherway, neither by users nor by an explicit committee. If Users rate the usefulness of an article... everbody uses wikipedia for some aim. I.e. altough we're are pretty convinced earth is not flat, maybe a lot of users may find it "useful" if they research on the social views of flat-earth-thinkers, such article would have right to live in the sense of usefullness of wikipedia. Altough this does not really solve the complications of a possible rate-vandalism....(Thats what you meant with "sockpuppets", no?) Axel Kittenberger 15:36, 18 Jun 2004 (UTC)

The project has problems in some articles with people with strongly held views grinding other people down... This creates an unpleasant atmosphere and drives people away or into the fringes of the project. It also means articles can give undue prominance to certain points of view. At the moment article content can be determined by weight of numbers. If enough people agree with you, or you organise, then you will probably prevail. This is a serious problem that we don't address. If you never give in, but do so politely then there is nothing that can be done. If you allow yourself to get rattled then you will end up being punished.

Is validation a solution to the warriors? It may divert attention to the make up of a committee. Those who feel that the committee does not share their POV will see real or alleged biases. Those who think academia is a hotbed of people with X position will be opposed. Most people will look the other way. Some will be opposed because they see the idea of a committee of experts as very un-wiki.

However we need to remember what we are here for. Our aim is to make a really good encyclopedia. Our aim is not to set the world to rights. We need to get to the stage where we could publish (and be proud of) articles on Hugo Chavez, Pinochet, Zionism (to name a handful). How do we get there? We are nowhere close to getting agreed content between different points of view/political/national positions on these sort of topics. I think we need to accept that we will never get total consensus; some people will never be happy. Our solutions include per article bans (which seems very unwiki for content) or the approval of a stable version. With the goal of producing a publishable article in mind, article validation seems the best option so far. It may seem like the lesser of evils but I think we need to go there in some cases. Secretlondon 11:08, 18 Jun 2004 (UTC)

The whole issue of VALIDATION is NOT to VOTE for an article, it is to make it so that it is correct, no spelling mistake, no grammar flaw, no factual errors. In short, it does not require that people just lazygly push a button to say "yes" or to say "no", it is that people actually sweat a bit on it, so that to make a half good article becomes a good enough for publication article. Validators are here to give work on the article, not to merely vote on it. Anthere 17:29, 23 Jun 2004 (UTC)

Ah Okay I understand this. The idea to this proposal is to have the validator shape up an article possibly lazily written, not to encourage the original editors to make high quality articles, or the encourage any editor to improve quality? (Axel, interp by Sj, 23 Jun 2004)

/. style points systems

I know there is some kind of "points" given to contributors on planetmath ... if we want a voting procedure on wikipedia for validation or approval which takes into account problems with sock puppets, I think that might be the way to go. A suggestion:

• Anons are given 1 vote.
• Registered users are given a number of votes, which increases according to some formula (to be constructed) which depends on the number of edits and time since registration. The first day (=24 hours), they should have 1 vote as any anon. The maximal number of votes anyone could get can be .... let's say 10 for the sake of the discussion.
• If any of the developers don't have enough to do ;-) a reverted edit is not included in the article count, or even reduces the article count by 2. Properly constructed, this might actually give a few trolls negative impact on the votes, in the sense that a long time troll voting "1" in Axel's proposal above, would equal a long time 'trusted' (at least 'non-reverted') contributor voting "5" or even "6", by assigning somewhat different numbers to the options than displayed:
  • -5 instead of 1
  • -3 instead of 2
  • and so on until
  • 5 instead of 6

Of course a problem will be that the vandals quickly finds out that they should vote 'the opposite way' compared to other contributors, so maybe one should after all give the 'often-reverted' contributors 0 votes?

Multiplying the vote value by the users '# of votes' (up to +10) gives a 'point' for the article which is used to judge whether it is approved of or not.

Sorry about all work implied (for the developers) to implement this proposal, but I don't believe we can have a system of validations unless some work is done in the code. Please behead the suggestion, and not me :-) \Mikez 17:03, 18 Jun 2004 (UTC)

• I would state, just keep it simple, thats all too complicated, no? I would also actually not worry too much about vote-vandalism or missuse. As in pre-wiki times if one would postulate an idea like this website, i guess 99% of people would have turned it down, arguing editable by everyone? impossible to work out... well it has been proofed different, yes? So as long as normal users outnumber missusers there should be no problem, or the costs dealing with them or even worrying about them would eat up the benefits gained. worse not to implement any thing for fear of missuse... 81.217.12.74 10:41, 19 Jun 2004 (UTC)

It is true that the 'pedia has surpassed the expectations of a lot of people concerning the ability to handle vandalism. However, it has done so by giving, I believe, quite adequate tools to a number of users (sysops). But what kind of measures can be taken against validation vandals? For every vandal, with 10 sock puppets (which is very quickly created), who validates an awful article with all 10 puppets, how are you going to remove that "validation"? This gives, IMHO, a powerful tool to the vandals in the sense that they could with a small effort cause either "validation wars", so that one really could not trust any "Validated" sign.

we should not rush into something we regret later. And of corse my previous posting is way too complicated to be of any use :-).

Respectfully, \Mikez 12:32, 23 Jun 2004 (UTC)

Maybe a sytem inspired by slashdot? You distribute vote-points amoung the users, but not all users and more or less at random. 1 vote is 1 point. A little more than the average users (pageviews/edits/etc.), are selected... This puts away the sock-puppet and vandalism issue, since a vandal would have to use all the sock-puppets in a way a standard user uses wikipedia (meaning 10-fold expenditure for 10-fold chance to get vote-points). I browse at highlevel (show +5 comments only) and for an interesting article I read 4-5 of the most relevant scored comments, which are most also the most insightful, saving a lot of time, but getting informed well. For the regretting part, it is an addition you can easily recall, if you are cautious... just test it out on a medium (volunteer) wiki, the only thing to regret is the possibly wasted coding effot if it turns out to be a failure (which I would doubt, but I don't count :o)) .. 13:40, 23 Jun 2004 (UTC)

More on Trust metrics

Trust of users

What I haven't seen above, surprisingly, is any mention of trust metrics, as seen on en:Advogato. See: Advogato's description of their trust metric, Attack Resistant Trust Metric Metadata HOWTO.

Methods such as committee-formation and how-many-edits will manually split users into Trusted and Untrusted groups in ways that are open to attack from vandals. The Advogato system is tested and has been used for quite some time on that site.

A user's trust level could be described as a percentage (I have absolutely no idea how the math would be for this; assume a method of converting Advogato's metric into a percentage scale, perhaps by quantiles)... for instance,

• <40% trusted: Cannot validate
• 40-75% trusted: Validation votes count for half
• 75-95% trusted: Validation votes count in full
• >95% trusted: Can completely validate an article with one vote (assuming validation is a scale from $0$ to $n$)

The system used should scale along with users. By allowing users to rate each other "trustworthy" (allowing negative ratings is kind of asking for high-school cliquey flamewars), the ratings themselves can be made trustworthy. Very little extra effort is required, and the payoff is a much, much more robust system.

In the Wiki case, that trust metric could even be used to estimate the trust level of a new edit, based on the trust level of the editor. (This doesn't do anything for anonymous edits, of course.) Of all the possibilities, this seems to be the most open and community-based. Good editors will be trusted by the community, and good edits will be trusted as well. Thoughts? Grendelkhan 22:34, 3 Jul 2004 (UTC)

Trust of validator expertise, 'fiefdoms'

Validation is only possible for a person who has deeper knowledge about the content of an article. A commitee has only limited knowledge and is not able to validate all articles. If you give a validation vote to every user you have to trust the voting user that he is an expert for this article. Also automatic validation has this problem, you assume every editing user is an expert. I believe the validation comes by itself if the community behind the wiki is large enough to have several experts for every theme. In the meantime we have to live with unvalidated articles. Mijobe 13:53, 30 Jun 2004 (UTC)

I think the problem is more related to the knowledge of the validator then to the validators "trustworth". Maybe the validator is sure that an article is correct while it is incorrect. If no real expert checks the article and finds the error every other who believes this validator (especially the validator himself) no longer will think about the correctness of the article. Mijobe 21:42, 5 Jul 2004 (UTC)

Validation in the sense of correctness is a much stickier problem, as it requires a central authority to say "Username FOOFOORAW is a PhD in astropsychology". A clever distributed solution isn't worth much of anything, and (I will bet) any centralized attempts will end up in a Nupedia-style jam, where nothing gets done.

We'll end up, six months down the line, with a collection of six validated articles, all six months out of date. No thank you!

As I understand it, our central goal is to prevent someone from opening their WikiReader and seeing "GOATSE GOATSE ALL DAY LONG / GOATSE GOATSE THE GOATSE SONG". As a method of verifying spelling and structural quality---which, I think we can agree, is definitely needed---a trust metric would make the Wiki production-ready. Grendelkhan 03:44, 6 Jul 2004 (UTC)

I agree totally. Maybe we should put on the top of the page a highlighted explicit dislaimer what this going-on is about? It's IMHO about validation of SPELLING and a bit of QUALITY (e.g. having only one or two spaghetti-texts). Not about validation of correctness per se, as it would a) not really function technically b) is philosophically impossible either way. Without this disclaimer we always discuss in circles by people putting up that some proposal would not validate correctness, and people replying softening this argument, and circling in this discussion again wihtout any result. IMHO systems like trustnetworks, /.-style votes, or other if tinkered carefully, help in ensuring SPELLING and structural QUALITY, but not in absolute (god-given?) correctness (only mainstream views of things allowed?). Axel Kittenberger 10:10, 6 Jul 2004 (UTC)

Correctness in the sense of an Encyclopaedia only can mean, every known aspect is included in the article. Spelling and Structural correctness are very simple (everyone can help to extend this without having a more or less complex metric). A validated article must be validated in sense of it's content not in sense of it's layout (i am less interested in good layout, but very interested in good information). I don't want to say that there is no way to get a good validation, but we have to be carefull, that the limits of each validation are understood correct by everyone who trusts the validation. Mijobe 12:06, 6 Jul 2004 (UTC)

It's a central tenet of wiki that users are all equal to each other (as far as practicality, e.g., adminship, allows). If we're going to mess with that by allowing users' voices of validation to speak louder than other users', it's imperative that we do it in an organic, grassroots way which, while remaining secure, doesn't lead to the creation of a cabal. Having Jimbo or whoever individually bless people as queens of subfiefdoms of Wikipedia is a recipe for a mess. When I even imagine its potential magnitude, I need to check my pants. Grendelkhan 16:53, 6 Jul 2004 (UTC)

Thank you, Axel. My thoughts precisely. We can learn from the mistakes of the Slashdot moderation system (though that's not really a relevant topic, since posts can't be edited there) and come up with a validation system that works. Grendelkhan 16:53, 6 Jul 2004 (UTC)

It seems unlikely to me that 'trust of expertise' will ever be workable in a systematic way. Within a discipline, certain people will have more expertise in one aspect than in another. However, 'trust of honorable processing' should be possible. A person who is trustworthy in honorable processing should be able to to make a judgement in any part of the wiki, on the basis that the core editors of a page have arrived at a workable version that should be protected against vandalism. In the event that a page was unstable due to edit wars, the 'trust of honorable processing' means that the person is empowered to validate a version of the page, engage in a dispute-resolution process with the editors of the page, or simply wait until there is more stability before validating. NickArgall 01:38, 5 Nov 2004 (UTC)

Copyediting

As chief copyeditor for Nupedia, I am concerned with the very incomplete treatment of how we will copyedit articles. From what I have read of the English language articles in Wikipedia, I see a need for copyediting in almost every one of them. But, there are differences in validating articles for content, style, NPOV, etc., as opposed to copyediting them. In some ways, article validation poses different problems than article copyediting.

• For copyediting, there are standards of correctness, available by choosing a book/s as a reference/s.
• A copyeditor for a particular language does not have to be particularly familiar with the article content. For this reason, an English language copyeditor can work on just about every article, written in English. The same is true for every language we have content in.
• I think most will agree that an article is either sucessfully copyedited or not. A simple "Yes" or "No" seems sufficient, in judging whether an article is copyedited.
• Almost any revision of an article requires that the revised portion of the article be copyedited.

I am going to stop here for two reasons. First, I am writing very casually about this and would like to think carefully, before making any further remarks. Second, I hope there is some feedback, before I continue. --RoseParks 21:50, 18 Jul 2004 (UTC)

This is so true! Thank you for your comments, Rose; I hope you will give us the benefit of your advice. I at first imagined 'copyedit' validation being rolled into validation of style and content quality; however, you are right to point out the universal need for completeness of copyediting (and the special burden on copyeditors to be the last ones to visit an article, even after small changes). Perhaps copyediting (along with general layout/style?) should be distinguished from more subjective evaluations of content quality.

I do think copyediting is one area in which we should invest time in training interested editors -- it takes practice and certain off-wiki resources to become truly good at copyediting. We might also benefit from distinguishing structural copyediting from final-draft proofreading, as the skills and style-guidelines involved differ somewhat -- particularly if we want our final copy to be as clean and readable as that of other serious encyclopedias.

Someone recently suggested on Talk:Translators that we set up a list of copyeditors/proofreaders to help clean up translated copy; I think such a list would benefit the entire project.

If any WPans would like to hone their proofreading skills, I would be willing to parcel out and evaluate short proofreading tests (giving an article a thorough update in the process!). +sj+ 02:27, 19 Jul 2004 (UTC)

Update: see Copyediting and Proofreading. +sj+

I am pleased to see your positive response. As you say, copyediting requires training and practice. That was part of my job at Nupedia. There, we were able to require that copyeditors purchase 2 reference books. Then, they were reimbursed after they copyedited 5 articles. This contributed greatly to the uniformity of copyedit results. I have no idea if this is possible for Wikipedia. At that time, online resources were scarce, to say the least.

I would be happy to work with you on the copyediting process. One of the first problems, after planning, would seem to be recruitment, and not only for English :-).

--RoseParks 03:16, 19 Jul 2004 (UTC)

I also support the incorporation of copyediting as a distinct and necessary component of the validation process. Danny 11:48, 19 Jul 2004 (UTC)

How do you deal with the differences between American English (AE) and British English (BE)? Does each article get tagged with the standards that are to be applied? Principally this will be to do with matters of spelling, but I am sure there are plenty of stylistic differences as well. Noisy 02:15, 14 Sep 2004 (UTC)

I don't need any stinking reference books, though I may have glanced through one. It takes a genius to simply know what's right. Noisy, you mean Commonwealth English (CE). The variant spellings should be retained in articles as a sign to all of the backgrounds of each contributor; however, the punctuation and grammar should be single and objective. lysdexia 16:41, 9 Nov 2004 (UTC)

Goals

I think this whole thing would have gotten off to a better start if we'd taken the time to define some goals, a la default view of content. As it is now, this page is a mishmosh, and the 'feature' has been implemented by pretty much ignoring this whole thing. (And I still think there are really mathematically sound ways of doing this, much better than silly plain percentages of raw potentially-sockpuppet votes.) Grendelkhan 06:28, 2 Dec 2004 (UTC)

Offline publication efforts of German Wikipedia documented

For those of you interested in publishing an offline Wikipedia snapshot, validating articles or moving towards Wikipedia-1.0, I have documented the quite successful efforts of the German team at German Wikipedia. AxelBoldt 01:34, 22 December 2005 (UTC)

Wikimania session on article validation -please give comments

Paul Kobasa and I are coordinating a discussion session on fact-checking on Wikipedia, at 4pm on Friday:

• Validation on Wikipedia: How do I know this article is accurate
• ‘Just the facts, ma’am (or sir)’: Accuracy and impartiality in crafting an encyclopedia article

I thought people reading this might be interested in attending. My background is work with the English Wikipedia 1.0. Paul Kobasa is editor-in-chief at w:World Book. I am keen for us develop a complete system to deal with this old chestnut! I will suggest a system of expert review teams, based at appropriate WikiProjects, working to develop validated versions of articles that have had every individual fact & reference checked. These validated versions would be available via an extra tab, and (unlike the main article) they would not be editable other than for updating by the review team. These ideas will simply start off the discussion, and I hope that the discussion will close with a consensus and some clear action points. I'm hoping that some of the Wikimedia software people will be there, with ideas on how to solve the validation problem. I will put up an en:Wikipedia page to continue the discussion, and comments can also be made here, I hope some of you can help with that. If you have any specific proposals you'd like to see me raise, please mention them here or contact me on en. Hope to see you there, or at least helping us online! Walkerma and Walkerma 00:04, 3 August 2006 (UTC)