Wikimedia update feed service
 | This page is kept for historical interest. Any policies mentioned may be obsolete. If you want to revive the topic, you can use the talk page or start a discussion on the community forum. |
 | For a modern alternative to this OAI-based update feed service, see wikitech:Event Platform/EventStreams. |
Wikimedia offers an update feed service for near-real-time updates of page data and image uploads from Wikipedia and related sites.
Availability and pricing
[edit]Due to the stress this can place on the databases, at present this update feed is restricted access; the service is targeted primarily at search engines and similar bulk analysis/republishing. Per Wikimedia Head of Business Development Kul Wadhwa, Wikimedia is no longer providing the feed service to new parties for the foreseeable future.
History
[edit]In May 2004, a data feed of the list of new pages was provided to Yahoo! to ensure up to date search results of Wikipedia's content. In Spring 2005, an agreement was reached with Answers.com to provide a feed. An RSS feed was also proposed. Ask Jeeves (a search engine) and Opera (a web browser company) were interested in such a feed as well, and Amazon.com had expressed some interest, but no agreements had been reached with any of them as of October 2005. Answers.com experienced serious problems with the OAI feed that they were beta testing.
Technical requirements
[edit]The update service is implemented using the OAI-PMH harvesting protocol. We provide a custom plugin for MediaWiki to update a local database from this feed; alternatively a client could provide their own OAI harvesting software and work with the page text in another manner.
The plugin requires appropriate versions of PHP and MySQL to run MediaWiki, and should work cross-platform. In addition to standard requirements, PHP's 'domxml' module must be installed.
A new release of the OAI client for MediaWiki 1.7, offering best compatibility with current Wikipedia data rendering, is scheduled for late July 2006.
Other data availability
[edit]Full text and metadata of Wikipedia and other Wikimedia wikis are made available to the internet at large for bulk download, updated roughly every couple of weeks (for most Wikimedia projects) to once a month (for the English language Wikipedia).
See data dumps for details.