ウィキメディアのサーバー

ウィキペディアや他のウィキメディアプロジェクトは、いくつかのデータセンターにあるサーバーラック上で動作しています。

システムアーキテクチャ

ウィキメディア財団によってデプロイされているMediaWikiのソフトウェアを単純化した概要です（2022年8月時点）。本質的には、複雑に構成されたLAMP "スタック"となっています。

ネットワーク・トポロジー

ネットワーク・トポロジーは、Wikitechの「ネットワークデザイン」で説明されています。

ソフトウェア

• ウィキメディアのDNSサーバーはgdnsdを管理しています。ウィキメディアはクライアントの場所に応じて、地理的DNSを使用して、7つのデータセンター（米国に3つ、ヨーロッパに2つ、アジアに1つ、南アメリカに1つ）間で要求を分散します。
• We use Linux Virtual Server (LVS) on commodity servers to load balance incoming requests. LVS is also used as an internal load balancer to distribute MediaWiki requests. For back end monitoring and failover, we have our own system called PyBal.
• For regular MediaWiki web requests (articles/API) we use Varnish and Apache Traffic Server caching proxy servers in front of Apache HTTP Server.
• すべてのサーバーはDebian GNU/Linuxで稼働しています。
• 分散オブジェクトストレージにはSwiftを使用しています。
• メインとなるWebアプリケーションはMediaWikiで、PHP（約70%）とJavaScript（約30%）で構築されています。
• Our structured data is stored in MariaDB since 2013. We group wikis into clusters, and each cluster is served by several MariaDB servers, replicated in a single-master configuration.
• We use Memcached for caching of database query and computation results.
• For full-text search we use Elasticsearch (Extension:CirrusSearch).
• https://noc.wikimedia.org/ – Wikimedia configuration files.

Wikimedia server racks at CyrusOne

ホスティング

See also: w:Wikimedia Foundation#Technology and wikitech:Data centers

As of 2024年4月, we have the following colocation facilities (each name except for Magru is derived from an acronym of the facility’s company and an acronym of a nearby airport):

eqiad

Application services (primary) at Equinix in Ashburn, Virginia (Washington, DC area).

codfw

Application services (secondary) at CyrusOne in Carrollton, Texas (Dallas-Fort Worth area).

esams

Caching at EvoSwitch in Amsterdam, the Netherlands.

ulsfo

Caching at United Layer in San Francisco.

eqsin

Caching at Equinix in Singapore.

drmrs

Caching at Digital Realty in Marseille.

magru

Caching in São Paulo, Brazil.

歴史

The backend web and database servers are in Ashburn, with Carrollton to handle emergency fallback in the future. Carrollton was chosen for this as a result of the 2013 Datacenter RfC. At EvoSwitch, we have a Varnish cache cluster and several miscellaneous servers. The Kennisnet location is now used only for network access and routing.

Ashburn (eqiad) became the primary data center in January 2013, taking over from Tampa (pmtpa and sdtpa) which had been the main data centre since 2004. Around April 2014, sdtpa (Equinix – formerly Switch and Data – in Tampa, Florida, provided networking for pmtpa) was shut down, followed by pmtpa (Hostway – formerly PowerMedium – in Tampa, Florida) in October 2014.

In the past we've had other caching locations like Seoul (yaseo, Yahoo!) and Paris (lopar, Lost Oasis); the WMF 2010–2015 strategic plan reach target states: "additional caching centers in key locations to manage increased traffic from Latin America, Asia and the Middle East, as well as to ensure reasonable and consistent load times no matter where a reader is located."

EvoSwitch and Kennisnet are recognised as benefactors for their in-kind donations. See the current list of benefactors.

A list of servers and their functions used to be available at the server roles page; no such list is currently maintained publicly (perhaps the private racktables tool has one). It used to be possible to see a compact table of all servers grouped by type on icinga, but this is no longer publicly available. However, the puppet configuration provides a pretty good reference for the software that each server runs.

B-roll of servers in Texas in 2015

ステータスとモニタリング

You can check one of the following sites if you want to know if the Wikimedia servers are overloaded, or if you just want to see how they are doing.

• https://www.wikimediastatus.net/
• Grafana
  • Data center overview with total bandwidth, non-idle CPU and load per group of servers; versions also exist with a per-subcluster dropdown and total numbers without graphs (down in July 2019)
    • https://grafana.wikimedia.org/ should get you started
  • Networking latency
• Icinga (private)

If you are seeing errors in real time, visit #wikimedia-tech^接続 on irc.libera.chat. Check the topic to see if someone is already looking into the problem you are having. If not, please report your problem to the channel. It would be helpful if you could report specific symptoms, including the exact text of any error messages, what you were doing right before the error, and what server(s) are generating the error, if you can tell.

エネルギー消費

In 2017, the WMF board of trustees adopted a resolution containing a commitment to minimize the Foundation's overall environmental impact, especially around data centres through using green energy. The community-led Sustainability Initiative, created in 2015, aims at reducing the environmental impact of the servers by calling for renewable energy to power them.

The Wikimedia Foundation's servers are spread out in five colocation data centers in Virginia, Texas and San Francisco in the United States, Amsterdam and Marseille in Europe, and Singapore in Asia.

In 2021, the servers used 358.8 kW (kilowatts), summing up to about 3.143 GW h (gigawatt hours) of electrical energy per year. The total carbon footprint of the servers was 1,073 metric tons CO2-eq in 2021.

Only the few servers in Amsterdam and in Marseille run on renewable energy, the other use different conventional energy mixes. In 2016, just 9% of Wikimedia Foundation data centers' energy came from renewable sources, with the rest split evenly between coal, gas and nuclear power (34%, 28%, and 28%, respectively). The bulk of the Wikimedia Foundation's electricity demand is in Virginia and Texas, which have both very fossil fuel heavy grids.

サーバー名	データセンターの場所	プロバイダ	運用開始日	平均エネルギー消費量（kW）	エネルギー源	カーボンフットプリント（CO2/年）	再生可能オプションとコスト
eqiad	Ashburn, VA 20146-20149 USA	Equinix (ウェブサイト)	2011年2月	2016年5月: 130 2015年5月: 152	2016年: 32% 石炭 20% 天然ガス 25% 原子力 17% 再生可能エネルギー	1,040,000 lb = 520 short tons = 470 metric tons = 0.32 * 130 kW * 8765.76 hr/yr * 2.1 lb CO2/kWh for coal + 0.20 * 130 kW * 8765.76 hr/yr * 1.22lb CO2/kWh for nat gas + 0.25 * 130 kW * 8765.76 hr/yr * 0 lb CO2/kWh for nuclear + 0.17 * 130 kW * 8765.76 hr/yr * 0 lb CO2/kWh for renewable	In 2015, Equinix made "a long-term commitment to use 100 percent clean and renewable energy". In 2017, Equinix renewed this pledge.
codfw	Carrollton, TX 75007 USA	CyrusOne (ウェブサイト)	2014年5月	2016年5月: 77 2015年5月: 70	2016年: 23% 石炭 56% 天然ガス 6% 原子力 1% 水力/バイオマス/太陽光/その他 14% 風力 (Oncor/Ercot)	790,000 lb = 400 short tons = 360 metric tons = 0.23 * 77 kW * 8765.76 hr/yr * 2.1 lb CO2/kWh for coal + 0.56 * 77 kW * 8765.76 hr/yr * 1.22lb CO2/kWh for nat gas + 0.06 * 77 kW * 8765.76 hr/yr * 0 lb CO2/kWh for nuclear + 0.15 * 77 kW * 8765.76 hr/yr * 0 lb CO2/kWh for renewables	?
esams	Haarlem 2031 BE Netherlands	EvoSwitch (ウェブサイト)	2008年12月	2016年5月: < 10 2015年5月: 10	「風力、水力、およびバイオマスの組み合わせ」	0	n.a.
ulsfo	San Francisco, CA 94124 USA	UnitedLayer (ウェブサイト)	2012年6月	2016年5月: < 5 2015年5月: < 5	2016年: 25% 天然ガス 23% 原子力 30% 再生可能エネルギー 6% 水力 17% 詳細不明 (PG&E)	13,000 lb = 6.7 short tons = 6.1 metric tons (+ unspecified) = 0.00 * 5 kW * 8765.76 hr/yr * 2.1 lb CO2/kWh for coal + 0.25 * 5 kW * 8765.76 hr/yr * 1.22lb CO2/kWh for nat gas + 0.23 * 5 kW * 8765.76 hr/yr * 0 lb CO2/kWh for nuclear + 0.36 * 5 kW * 8765.76 hr/yr * 0 lb CO2/kWh for hydro/renewable + 0.17 * 5 kW * 8765.76 hr/yr * ? lb CO2/kWh for unspecified	?
eqsin	Singapore	Equinix (ウェブサイト)	?	?	?	?	?
drmrs	Marseille	Digital Realty (ウェブサイト)	?	?	?	?	?

関連項目

さらなるハードウェア情報

• wikitech:Clusters – technical and usually more up-to-date information on the Wikimedia clusters

Admin logs

• Server admin log – Documents server changes (especially software changes)

Offsite traffic pages

• Alexa traffic rank

歴史的情報

• Cache strategy (draft from 2006)
• PHP caching and optimization (draft from 2007)
• Hardware orders up to 2007