Article validation proposals/Page 3

The Article validation proposals have been split into multiple pages to keep them at a manageable size.

Legion of Trolls proposal

Trolls are not here to approve, and usually reject views of experts who must be certified by someone trolls grumble about. So one would expect them to be disgruntled by definition about such a mechanism. However, paradoxically, almost all trolls think they apply clear and reasonably stringent standards. The problem is that each troll has his own standards, unlike those of others!

That said, there is much to agree on: the mechanism itself must be genuinely easy to use, nothing slow and rigorous is of any value, the progress of Wikipedia and its proven process should not be impeded, and the results of the approval can be ignored. Where trolls would disagree is that verifying the expert's credentials are of any value. Any such mechanism can be exploited, as trolls know full well, often being experts at forging new identities and the deliberate disruption of any credentialing mechanism.

One might ignore this, and the trolls, but, it remains that what goes on at Wikipedia is largely a process not of approval but of impulse and then disapproval. As with morality and diplomacy, we move from systems of informal to formal disapproval. Today, even our reality game shows demonstrate the broad utility of this approach, with disapproval voting of uninteresting or unwanted or undesired candidates a well-understood paradigm.

So, imagine an entirely different way to achieve the "desirements", one that is a natural extension of Wikipedia's present process of attempt (stubs, slanted first passes, public domain documents, broad rewrites of external texts) and disapproval (reverts, neutralizing, link adds, rewrites, NPOV dispute and deletions). Rather than something new (trolls hate what is new) and unproven that will simply repeat all the mistakes of academia. Imagine a mechanism that

Begins with all approved, and makes it possible to broaden or narrow the selection of approvers (e.g., one person might only wish authors who have phd's, another would allow for anyone who has made an effort to approve any articles) for each reader, or supported class of reader, simply by disapproving editors.
Allows for extracting topic-oriented sets (e.g., in order to produce an "Encyclopedia of Music") relying on metadata that is specific to each such supported class of reader, not part of the Wikipedia as a whole
Exploits ongoing feedback ("I don't care about this." or "I don't understand this.") to adjust the list of articles of interest. Each user can begin from some class (like Simple-English-only readers), and adjust if they like.
Potentially, exploits more feedback on authors ("I can't believe this." or "I find this irrelevant.") to adjust also the list of disapproved authors/editors.
Credits each troll who has driven off a disapproved author or editor. OK, that's a joke, but what do you expect, I'm a troll neh neh neh...

By embracing and extending and formalizing the disapproval, boredom and disdain that all naturally feel as a part of misanthropy, we can arrive at a pure and effective knowledge resource. One that rarely tells us what we don't care about. And, potentially, one that can let us avoid those who we find untruthful.

Propose and veto

Include articles that have been proposed by at least one person, and vetoed by none.

Where two versions of an article are so approved, pick the later one. Where no versions of an article are so approved, have no article.

That's it.

This is too easily abused. It adds too much load to the server for the dismal results it will produce. Brian j d 10:22, 2005 Jan 29 (UTC)

Andrew A's proposal

See Referees. This proposal is consistent with much of the above.

Main features:

No impact on the existing way of doing things other than appearance of extra links on some article pages, saying that this article has been reviewed and linking to details.

All articles remain in the main namespace. The only new namespace is a special one used to keep track of who has reviewed what.

All reviews relate to a specific version. As any update to the article creates a new version, this initially has no reviewed status at all (with some possible exceptions where there is no need to maintain independence between contributor and reviewer and the software might therefore flag the approval by default). The concepts of freezing articles or promoting them in a configuration management system are not needed.

All reviews are simply check boxes or radio buttons. Discussion remains on existing talk pages.

QA on the main Wikipedia is supplied by three levels of reviewer:
- A basic level which we initially give to anyone who asks for it,
- A specialist level at which the reviewer claims expertise on the subject of the article,
- A referee level at which the reviewer claims to be a quotable authority on the subject of the article.

At the referee level only, the reviewer is expected to be independent of the article author(s). At other levels, reviewers are encouraged to edit and refactor as necessary, but not to review articles which are primarily their own work.

At all levels, collegiate responsibility is the system, identical to the existing Wikipedia. There is no attempt to formally identify areas of expertise. Other members at this level are expected to comment if a particular reviewer is performing badly, and particularly if they are venturing outside their areas of expertise at the higher levels.

At all levels, all reviewers are expected to approve only articles that they consider of high quality.

All three levels allow for dissent from the approval of others.

The same software (with extensions) can support
- Wikipedia 1.0,
- A G-rated Wikipedia,
- Specialist encyclopedias.

Readers of Wikipedia must be offered user-friendly ways of restricting what they see to their desired level of review and (hopefully) reliability. These might include
- User options,
- URL aliases (e.g. www.family.en.wikipedia.org, www.refereed.en.wikipedia.org).

David Levinson's proposal (revised Jan 2005)

Establishing Release Edition

The "Wikipedia Release Edition" would be a (hopefully large) subset of "Wikipedia Working Edition".

Users could browse the release edition or the working edition. When browsing, a header would note whether or not the latest version in the working edition was the same as the release edition.

At the bottom of every page of Wikipedia there would be two buttons: "Approve" and "Disapprove".

Any logged in user can vote once. Article versions would be scored based on number of votes for approval minus votes for disapproval. The article version with the highest score (so long as it is greater than 1) would be the released article.

A user who clicks "edit this page" would be presented with the newest version, and a note if this is not the Release Edition version. (showing differences)

Any save is automatically a vote in favor of that version and a removal of approval votes for every previous versions by that user.

Recent changes would note if a new version becomes the release version.

Advantages

Doesn't rely on "expert model",
Uses consensus approach, the article with the most favorable votes, and more favorable than unfavorable votes, is released;
If votes go awry, edit wars begin, sensible users can vote to "disapprove" bringing score negative;
Release edition can happen quickly, no approval queue waiting for votes, only one voter beside author is required (assuming the article garners no disapprovals).

Disadvantages

An older version with lots of votes in favor may be difficult to dislodge without active observation.
Editing may be more complicated with different versions for release and working edition.
Votes can be manipulated by a user with multiple log-ins.

Establishing Quality Assurance

The release edition depends on votes. It also may lead to the perception that there is a fork, or create confusion about multiple wikipedias. A more "wiki" way of doing things would be to have only one release version, but allow anyone to rate any version. This system exploits version control to give users the ability to know who has "certified" a version.

So if I know that revision 01.22.2005.10:33 of an article has been certified by User:Authority, User:Hobbyist and User:Professional, I may trust it more than an article certified by User:Vandal or User:Random.

Users who want to influence what other people read will need to establish credibility. Real world information helps here, as well as within-Wikipedia credibility.

Given the potential number of Users who might certifiy an article, and the likelihood I have never heard of most of them, users should be allowed to join certification "clubs" (or rather "clubs" should be allowed to include members, through a process like Wikipedia:Requests_for_adminship). There might be a group who certifies articles on biology, and another group that certifies articles on astronomy, and so on. There could be multiple clubs who certify articles (biology and astronomy might both certify an article on exobiology. There might be more than one astronomy club, if there are differences, and good articles would be certified by both, and controversial articles by only one. Clubs could team, so if Club:Biology and Club:Astronomy trusted each other, they would be part of a Team (say Team:Science) (and of course there might be multiple teams certifying the same article, or different versions, which would ultimately need to be hashed out in a wiki way or would stand).

If club members went off the reservation so to speak, and certified rubbish, they could be kicked out of the club, and they would no longer be able to speak for the club (there certifications would no longer hold the club imprimatur). (similarly clubs could be kicked out of teams).

The important thing is that no one is requiring that only credentialed individuals be permitted to certify an article, but that if they do, users can of their own free will give that more credance than an uncredentialed person certifying the article.

Does this make wikipedia more complicated? Yes of course it does.

Does this make wikipedia more reliable? Yes, as you would now know who thought what was accurate.

Additions to MediaWiki required by this

A way of tracking who certified each version,
A way of tracking who was in what club (and what club was in what team).
A certification history of articles.
At the top of the page it would notify me if there was a more recent uncertified version of the article, or the most recent version of the article certified by Team:X or Club:Y.
A certification or approval tab.
The ability to view only articles approved by Team:X, Club:Y, or User:Z. (This could be done like categories, but there is probably a better mechanism)
The ability to see Recent Approvals.

dml 17:09, 22 Jan 2005 (UTC)

dml

Giles Robertson's proposal

There doesn't need to be a central approval mechanism; approval is all about trust, and centrally dictating who can be trusted raises too many issues. That being said, an approval mechanism that relies on everybody voting is open to abuse, and doesn't carry the weight of approval by authority.

Instead, a decentralised proposal [which is similar to Andrew A's proposal], would be for authorities to create pages that list why they are an authority, and then list the pages and versions that they have approved. This could be done without any change to the existing codebase, but a system that marked some pages as "authority conferrers", and that marked on each page which authorities had approved it, would improve the usability.

Authority conferring pages could confer authority on to other authority-conferring pages; the aim is to build a 'web of trust' somewhat similar to that which provides the assurement in PGP keys that people are who they say they are. That said, it is important to prevent authority-conferring pages from fabrication; it may be necessary to lock these pages to certain users.

How do we establish that an authority-conferring page has authority? If we link it to another page, and that page displays text or graphic (with the Wikipedia logo) confirming that the external page trusts the authoritiy-conferring page, then we know, if the external page is free from interference, that the authority-conferrer is trusted. How strong that is as an approval, depends on what the external page(s) for the authority-conferrer are.

This also allows a selective extraction of approved pages: Pick an authority-conferrer, and extract every page that is approved by that authority, and, at your option, other pages at lower levels (e.g. page approved by authority trusted by authority you've just chosen).

The major disadvantage is that this introduces yet more metadata, in the form of the authority-conferrers. It also does not stop the creation of spurious authority-conferrers; though they won't be trusted or have much external approval, they may mislead. Any save is automatically a vote in favor of that version and a disapproval of every previous versions.

Recent changes would note if a new version becomes the release version.

Giles Robertson

m.e's proposal

I have a proposal at User talk:M.e/Production Wikipedia for creating 'production quality' pages. Highlight is the idea of collecting 'issues' and 'endorsements' against a 'frozen' version of a page. Hopefully we can then converge on a revised version that just fixes the issues. m.e. 12:16, 12 Sep 2004 (UTC)

Szopen's proposal

Pending edits. Maybe not directly about approval mechanisms, but similar. Edits would be delayed for a fixed amount of time, and if none would object to them, they would be automatically approved.

ChrisG proposal

main article: User:ChrisG/Approval_mechanism

This is a suggested approval mechanism to meet the need to make Wikipeda a more reliable source and to support Wikipedia 1.0. It is a twist on the namespace workflow mechanism suggested by Magnus Manske here. It is a process firmly based on wiki principles. It is flexible because decisions about quality are based on human judgment that each article meets the minimum standards for approval for Wikipedia 1.0. If it is agreed that standards should be raised then humans can amend their judgment appropriately during approval debates for more modern versions.

Aims

The design aims to:

Have no effect on the wiki process for creating and developing articles.
No fork within Wikipedia
Identify specific versions of articles as being stable and hence give the reader the comfort that the flagged version has been reviewed as meeting certain minimum standards.
Identify specific versions of articles as suitable for Wikipedia 1.0.
The process needs to be scaleable. It needs to be able to approve a large percentage of Wikipedia within 3-6 months.
Mark versions of each article appropriately depending on the outcome of any approval process for that version. This gives a guide to quality of particular versions in the history.
Be simple to understand.
Be easy to manage.
Be democratic.
Avoid requiring any exclusive, 'expert' clubs.
Be relatively simple to code.
Difficult to game.
Evolutionary approach rather than fundamental change in Wikipedia process.
Be acceptable to the majority of Wikipedians.

Overview of the process

The process makes use of namespaces to create a workflow process. A copy of the recommended version of an article is moved through the system, with an attached discussion page and votes. The process is best illustrated visually:

Maureen's proposal 1

The following was proposed by User:Maurreen on Wikipedia talk:Forum for Encyclopedic Standards 06:49, 19 Nov 2004 (UTC), but seems appropriate to mention here. -- Jmabel | Talk 22:45, Nov 23, 2004 (UTC)

<begin Maurreen's post>
Here's an outline of a possible plan, submitted for your suggestions. One advantage is that the computer part of it is simple. Maurreen 06:49, 19 Nov 2004 (UTC)

An article could be "approved" with at least 10 votes and no more than 10 percent of the votes objecting. Only registered users could vote. Users with multiple accounts should only vote once. Voting for each article would be open at least a week.
To allow appropriate time for review and possibly improvement, no more than three articles would be considered at a time. Others nominated would be compiled on a "pending" list.
Nominations would be accepted only from registered users who agree to maintain the article. So, in a sense, we would be voting on the nominators also. The commitment to maintain the article would allow the article to still be edited, but give us some assurance that the article wouldn't deteriorate.
"Approved" articles would be compiled on a list, along with the names of the maintainers.
Possibly the nominations or list could refer or be limited to a specific version of the article. That is, the article at such-and-so date and time.
"Approved" articles could have some indicator of that status on the article itself.
Guidelines or standards for what is worthy could be determined before voting ever starts on articles.
Nominations would be encouraged from featured articles and peer-reviewed articles. Initially, general topics (such as Electronics) would be preferred to more-specific ones (such as Ohm's Law). That could work toward the "approved" material having a broad and even general base.
Nominations of contentious articles would be discouraged, at least initally.

Maurreen's proposal 2: "Reviewed articles"

Copied from Wikipedia talk:Forum for Encyclopedic Standards. Maurreen 14:12, 27 Nov 2004 (UTC)

Advantages:

Simple and quick. This could be implemented and useful within an hour of its adoption.
Takes best-of-both-worlds approach to wiki nature and any standing of experts.

Outline:

Any registered user could review any article.
There would be a category and list of reviewed articles.
The list would indicate which version of an article was reviewed by each reviewer.
Reviews on the list would be no more than a paragraph long.

Options:

Detailed reviews could be written and linked to from the list.
Reviewers who chose to could list themselves and a paragraph about any relevant qualifications or limitations on a list of reviewers.
Reviewers should at least indicate if they have worked on the article.
We could have a list of articles for which a review is desired, or use the current peer review page.
We could choose a set of suggested levels or other indicators (such as “acceptable,” “weak,” “comprehensive,” etc.).

Maurreen 21:04, 21 Nov 2004 (UTC)

Interim measure

Maybe it would be helpful to think of my proposal for article reviews as an interim measure. It isn't intended to be perfect by any means.

It is intended to give readers some measure of the quality of any given article or article version.

It is something that could very easily be produced and used while something better is discussed, decided and developed. It does not preclude any other system. It can include, or not include, a minimum standard for Wikipedia articles, which would need to be developed.

It could be one of any number of tools that work toward an eventual paper or "release" version. Maurreen 09:34, 25 Nov 2004 (UTC)

Editorial board(s)

Copied from Wikipedia talk:Forum for Encyclopedic Standards. Maurreen 14:12, 27 Nov 2004 (UTC)

My thought on this is that there is an immediate problem with 'Any registered user could review any article'. There's no problem with having anyone review an article - however to improve the credibility and reliability of Wikipedia, I think that we (unfortunately) need a link to the 'outside world' where people have real names and qualifications, rather than 'karma' built up under a nom-de-plume. For articles' credibility to be increased, someone's, or some people's, reputation needs to be on the line. My thoughts are that a properly constituted editorial board needs to approve (and possibly modify) articles. As I've mentioned elsewhere, there could (and in my view should) be multiple competing boards aiming to set their seal upon particular article versions. For example, one such board could be a set of academics in a particular subject whose names are known, who have a publishing record in peer-reviewed journals, and have an academic reputation. This does not preclude a self selected group of people setting up their own board under noms-de-plume and producing a Wikireputation based set of approvals. Users would have the choice of using either or both or neither board's seals of approval (article tags) as a filter into Wikipedia. WLD 21:48, 22 Nov 2004 (UTC)

ChrisG's template and process

Copied from Wikipedia talk:Forum for Encyclopedic Standards. Maurreen 14:12, 27 Nov 2004 (UTC)

Thinking some more about Maurreen's proposal it occurred to me that with the combination of templates and categories we could set up a voting system to approve articles. Consider this template (I used subst to create the text, e.g. {{subst:ChrisGtest}} ):

The version of this article listed below this message is a current candidate for Wikipedia 0.1. Standards for Wikipedia 0.1 are at Wikipedia 0.1 Standard.

Voting should last for a minimum of a week, and approval requires 80% support. If opposing votes outweigh supportive votes at any stage this candidature should be removed. Full rules are listed at Wikipedia 0.1 procedure. Please vote below either for or against approval. A negative vote only counts if it provides a supporting explanation for why the article version fails to meet the standard. Article validation proposals/Page 3

If you look to the bottom of the screen this template categorises the article as a candidate for Wikipedia 0.1 and also by the current day, month and year (uses variables so need to update). This means anybody wishing to vote on articles need only check the appropriate category for articles. They could click the links to the talk page of the articles they are interested in. The talk page of the article would give the link to the specific version and the votes so far; after checking the article the person could vote as they see fit.

As time passes, the articles listed as candidates will dwindle as they are approved or rejected. The fact the candidates are categorised by date would mean we know when to close the vote of any articles that have been sitting in candidate status for too long.

Rejected articles would have the candidate category removed. Successful articles would be given a Wikipedia 0.1 category instead, again identified by the date of the version approved. In addition the specific version of the article should be listed somewhere as approved on that date, i.e Wikipedia: Approved 0.1/1 Dec 2004.

I realize we don't have a consensus on how to approve articles, largely I think because some people are talking about approving top quality articles and others are talking about minimum standards for the CD/DVD editions; but this is a method which we could apply now without changes to the software, which would scale and would thus be suitable for either purpose.

ChrisG 01:10, 25 Nov 2004 (UTC)

Hawstom's proposal

If we want a mechanism that works right now to improve our credibility a little bit without sacrificing the principles we are all comfortable with, it must be simple and open. I propose that we simply start with the existing technical hierarchy of anon/user/admin/bureaucrat/developer to create a system that can assign a level of confidence to every version of every article, then display by default the most recent version with a disclaimer and links if the confidence level is low.

Principles

An article is only as trustworthy as the last hand that touched it. Wiki works because of immediate gratification.

Viewing

We continue to show to the public the latest version. If the confidence level for that article version is lower than our established standard, we show a disclaimer along with link to any more trusted article versions available. We put in user preferences a selection for the level of credibility to show by default(Developer/Bureaucrat/Admin/User/Anonymous), and we enable anon users, via a cookie or session id, to say, "Show only article versions with credibility level 2 or higher."

Editing

For users with high permissions, we put a check box or radio button on the editing page so that they may save with artificially low trust interim work with remaining unresolved credibility problems.

How it's simple

Nobody has to take the time to review articles. The approval occurs naturally as a by-product of the way we already work.

Since an article is only as trustworthy as the last hand that touched it and wiki works because of immediate gratification, only natural and open methods such as this can work at the Wikipedia.

How it improves our credibility

Places editorial responsibility on our trusted users. While it is true we have expressly disclaimed any editorial authority for our trusted users, the implicit attitude of the community has been to give them that authority. This proposal simply recognizes the de facto arrangement. In the future, new levels of editorial authority may be created.

Endorsement idea

Copied from Wikipedia talk:Forum for Encyclopedic Standards. Maurreen 14:12, 27 Nov 2004 (UTC)

We might be able to develop a system by which anyone could endorse a particular version of an article; presumably groups could form whose endorsement would carry some weight. The mechanism would be one "reader" approval equals point one(0.1) an "editor" would rank one point (1) an "editing librarian" would rank one to ten (1 - 10) an admin would rank 10 to eleven (10 - 11), a group could be assigned a similarly weighted scoring rank, i.e. an opinion offered editorially by "The Royal College of Physicians & Surgeons" might rank one way or the other compared to a select group of its Alumni. Deriving from this an articles approval-rating would be a function of its veracity as regards the opinion of the majority of its' readers. an entry in wikipedia would have (available for review) an articles position relevant to all other articles. (Idea from anon, Maurreen moved from project page.)

mijobe's proposal

One big benefit of Wikipedia is the ease of use. This must not be effected by the approval machanism. Because aproval easily can be used to vandalism it only is to be used by known people. Here my idea to reach that goal:

only people which have contributed to eg 100 validated articles are allowed to approve new versions. This should be measured by software to ensure that all users regularly contributing to Wikipedia get the approval allowence
if an editor is allowed to approve an article she get's an additional button if she opens an unapproved version which allows to vote for approval or against approval. If she votes against approval she has to give reasons for that
the approval only can be decided unanimously
if eg 10 votes are given for approval the version will be flagged as stable
if someone is vandalizing the approvals by always voting no she can be blocked from the approval process
if a new version only differs in typos from an approved one admins are allowed to approve the new version without new approval

This way it should be impossible for vandals to misuse the approval mechanism, while it allows to increase the amount of approving people without manual intervention. Users should be able to configure if they want to see the approved version by default or the unapproved. Anonymous users should be able to set this for the current session. There should be an additional button to toggle between these two version which is greyed if they are equal. --Mijobe 14:07, 21 Mar 2005 (UTC)

Grika's proposal-Delayed update

This is really an elaboration of Szopen's proposal. This proposal is an attempt to improve article validity while maintaining the dynamic quality and community spirit of Wikipedia

First, create a new admin level called Superuser (or Reviewer, or anything else already suggested here), available to anyone with at least 3 months and 500 edits. Superusers are not sought, they are granted (denied or revoked) by any Admin upon request by a qualified user.

Articles have a lag, i.e. they don’t update until a Superuser or higher OKs the edit or the lag expires. The lag setting of an article is assigned (and changed) by an administrator. The lag amounts might be:

None
1 hour
6 hours
1 day
3 days
7 days
Admin Only

None means that the article updates immediately, used for lists and other low vandalized articles. Admin Only means that only an admin or higher can qualify the edit and the lag never expires. The edits of Superusers (and perhaps anyone qualified to be a Superuser) and higher are visible immediately. New pages would default to the longest lag, but less than Admin Only (in the example above, 7 days).

The associated Talk page would still be immediately updated for everyone. Any user can request that the lag be temporarily set to None if they have a lot of edits to do (lag should automatically reset after four days).

The following pages might be effected thus:

Watchlist - pages might now show a number set by each entry such as (2/5/4) meaning 2 edits are in lag, 5 lags expired and 4 edits were certified. It could also simply duplicate the History page change below.
Recent Changes - articles will show up one or two times; one for the original edit and again when the lag expires or the edit is certified (it will show up only once if lag was set to None).
History - each entry might show a code such as (L) meaning in lag, (E) meaning lag expired, (S) meaning Superuser certified or (A) meaning Admin certified.

In the end: popular, often vandalized pages would be protected; Admins would not be burdened any more than they are now (and maybe less so); activities by Admins and trusted non-Admins would not be effected at all; and edits and new articles would not be "lost" due to Admin or Superuser lack of interest. The result should be a better resource for casual users with almost no effect on current active editors. Grika 20:40, 20 September 2005 (UTC)[reply]

Wolfkeeper's stabilisation

Basically, only the most recent stable version of article pages are shown. A stable version is one produced by any well established editor; or versions by anonymous or new user(s) that haven't been subsequently altered after a period of time, say 24 hours. The idea is that this stabilisation period gives people a chance to check their watchlists and remove vandalism and check edits against policy.

See: Wikipedia:Wikipedia:Timed article change stabilisation mechanism

The advantage of this kind of scheme is that it is extremely lightweight to use and no voting or other big changes to the wikipedia UI are needed.

Risk's Proposal

I thought I'd add to the haystack with my own ideas. They're mostly an elaboration on things that have already been proposed. I'll start out with a simple trust metric and discuss two elabotarions.

the basics

Each article gets one button marked 'validate', which lets the user validate the current version of an article (which is the only version that users can 'vote' on). That's all. Every user can validate, even anons. There is no change to the current user hierarchy. Every user starts with a basic 'weight', how much influence the user has on the validation process, say 0.1. Anons stay at this level, registered users can increase their weight. Once enough people have voted for validation on an article to reach a certain preset validation threshold (say 100), the article becomes validated. A user's weight increases when:

An article that she has contributed to becomes validated. When a new version of an article becomes validated all the editors that have contributed between the last validated version and the new version get a small increase to their weight. In short, trusted editors get more validating power.
The user has voted to validate an article that has become validated. For every article that becomes validated, the users that voted for it get a slight increase in weight (this is very small, and based on how overwhelming the vote was). In short, trusted validators get more validating power.

Additional comments:

The first 'weight' rule poses a problem. What if a user commits vandalism, that gets reverted, and the article gets validated? The vandalism counts as an edit and contributes to the vandal's weight. The solution here is that how much weight the user gains from a successful edit is based on how much of his contribution remains in the validated article. If only one letter of the user's edit remains, the user gets pretty much nothing at all.
The validation threshold differs per article based on a number of aspects such as the popularity of the article (by number of edits).
There is no negative vote. If a user thinks an article shouldn't be validated, she can change it or mark it unsuitable for validation with a template to call attention to it. Because any article will become validated if you leave it alone long enough (because of anons that don't know what they're doing), the amount that the article has already been validated decreases by a small amount per month (or the validation threshold increases).
When an unvalidated article that already has some votes going for it is edited, the validation starts anew. A part of the votes that it already had are transfered to the new version of the article. How much this is reduced, is based on the size of the edit. For instance, if an article is at 80% of the validation threshold and gets a 1 byte edit, the new version will be 70% validated. Anything more than two sentences wil greatly reduce the percentage. If a version of an article is precisely the same as a previous unvalidated version (for instance, because an edit got reverted) it will take on the exact number of votes the previous version had, or keep it's own, whichever is highest.
How users should validate (what it means to click the button) is left up to the community. The button could be marked 'click this if this article is free of spelling errors' or 'click this if you've fact-checked the article and found it correct'.
The validation thresholds should start off very high (so that it takes a lot of users to validate an article). A page could be created in the community portal for articles that require validation, to speed things along.

Advantages:

No change to the user structure. Wikipedia remains as free and as open as it has always been.
Simplicity. Just one button to click. Wikipedia retains as much user-friendliness as possible. All the complexity is in the technology and not on the user side.

Expert Knowledge

The above system would be good enough for fighting vandalism and checking grammar. For fact checking purposes the system needs to be more complex. The following addition is a way to take expert knowledge into account.

The first thing needed is a way to measure distance between articles, based on topic. For instance, the distance between 'black hole' and 'supernova' is very small and the distance between 'black hole and 'donkey' is very large. Ideally this would be measured by the shortest chain of links between the articles, but that would be to expensive computationally. There are possible shortcuts. See the additional comments for more.

The system takes previous edits and previous validations into account as described, but it weighs them based on article distance. For instance, if I vote to validate the 'black hole' article the algorithm calculates my weight for this vote dynamically by checking my edits and validations and adding them to my weight (by the rules already described), but weighted by distance. A successfull edit in the supernova article will increase my weight a lot, whereas a successfull edit in the donkey article will only slightly increase my voting power in the black hole article. This way, users who are successfull in a certain topic, get more validating power in that topic.

additional comments:

The ideal measure of distance would be the shortest chain of links between the two articles. This is far to expensive to calculate for every article dynamically. A good approximation would be based on categories (provided that the wiki is well categorized, like wikipedia is). That way the distance could be based on the highest level category the two articles have in common. 'Black hole' and 'donkey' only have the bottom-level category 'thing' in common (great distance), whereas 'black hole' and 'supernova' have the category 'Stellar Phenomena' in common. ('Black hole' isn't a member of 'Stellar Phenomena, but rather of the category 'Black Holes', which is a subcategory of 'Stellar Phenomena'). While it may still be a little expensive to calculate this, it's far more manageable and with a little cacheing and other optimizations I think it's very doable.
Checking exvry single one of a user's edits and validations and weighting them to distance every time a user clicks validate is too expensive as well. On any kind of decent wikipedian the system would have to check over 1000 edits (and god knows how many validations) and calculate the distance for every one of them. The solution, again, lies in the category tree. Where in the first system, the user had a single value called his weight, the user now has a small category tree, with weights attached. Let's say I have a successfull edit on the black hole article, and I earn 0.1 in weight for it. This weight is added to my personal 'weight tree' at the black hole-node, at the astrophysiscs-node, then at the physics-node, all the way down to the things-node. After many successfull edits and validations, I have a tree with a weight at the things node, that represents my total weight (like in the previous proposal), and several nodes that represent my weight in various categories, like astronomy. If I've never done anything in the category 'Politics', my weight for that category is zero (and the node doesn' have to be stored, minimizing the needed space). Now when I'm trying to validate the 'black hole' article, the system can check my weight for the 'black hole' category, the 'astrophysics' category etc. (All the categories that my tree has in common with the article I'm validating) and use them to increase my weight based on my expert knowledge.

advantages:

Expert knowledge is taken into account.
Still one button, still no change in the user hierarchy.

levels of validation

A third (and smaller) addition would be to have different levels of validation. Instead of one button there would be three (or more) buttons. A user could validate an article for spelling and grammar checking, layout and structure and finally fact checking. Each would require a different level of attention and each would require different settings for the algorithm. The community could require fact checkers to explain their findings on a the discussion page.

Risk 13:53, 27 September 2005 (UTC)[reply]

_TOC_

DiamondGeezer's Proposal: Karma Quality Editing

Each registered user has a "karma" attribute assigned to them.
Each SysOp has a karma of 500. Every designated "editor" has a karma of 400 (this would be a new attribute for each contributor)
When a new article is written, the karma of the author is increased by 5 points.
Every time an article is edited, the modifications are made to the wiki instantly, but are stored for a finite period (Bureaucrat settable parameter) and the author is informed of an edit to his/her original article. The modification is shown in a different color until the edit is accepted.
If the edit is accepted, the karma of the person who made the edit is increased by +1
If the edit is not accepted, the karma of the person who made the edit is either zero or reduced by 1.
If the edit is rejected, then the editors are informed of the rejected edit. If an editor decides that the edit is justified, then the edit is put in and the karma of the person who made the edit increased by +2. If the editor decides the edit is not justified, or if the editors do not respond to the edit at all within some (bureaucrat-set) period, then the edit is junked and the originator informed that his/her karma has been decreased.
If a contributor's karma reaches the level of the editors, then the contributor becomes an editor (if he/she desires).
If a contributor's karma reaches some low point (like -25) then the contributor's edits get bumped straight to the attention of the editors.
If a contributor's karma reaches some minimum (like -50) then the contributor's edits get immediately thrown in the bin.
If a contributor does not have an account, the IP address gets a karma of zero. If the edit is accepted, then the karma of the contributor is not increased, but if rejected, then the karma is decreased.
The bureaucrats can decide that if they have an authoritative expert, then that person's karma can be raised to whatever value up to an editor.
The editors have the discretion to punish vandalism with more negative karma.
Each article has a star-value (like the barnstars). The more stars the article gets, each star adds +1 to the author's karma (up to a maximum of 10).
There can be more than one author assigned to an article, up to a maximum of five.
If an article is deleted then 0 to -10 points are taken from the author's karma at the editor's discretion.

Discussion

This means that vandalism is discouraged, sensible contributions are rewarded, idiots get less and less attention until they effectively ban themselves. Bad articles or troll articles are a quicker way to get the contributor to give up their privilege to contribute to the wiki.

Editors and stylists (such as what Project Galatea are trying to do) would exercise weight to preserve the contents of the wiki from attack, increase the intellectual weight given to authors of particular articles.

Everybody can contribute to the wiki regardless of background, see instantly the results of their efforts and quality is rewarded.

--DiamondGeezer 20:19, 15 November 2005 (UTC)[reply]

The paragraph as unit of validation

Perhaps we could use paragraphs or subsections as semantic units for validation? A paragraph has sufficient internal consistency to stand alone from the rest of the article, and it is difficult to subvert one paragraph from within another, since paragraphs usually contain some context information that anchors them to the rest of the text. By doing this, small edits to one or more paragraphs would not necessarily invalidate the validation of an entire article. -- The Anome 00:25, 16 December 2005 (UTC)[reply]