Wikimedia servers
- Warning: Do not rely on any information on this page being up-to-date or correct.
- The information at https://wikitech.leuksman.com/ is likely to be more accurate.
Other languages: de, eo, es, fr, it, ja, ko, zh
Wikipedia and the other Wikimedia projects are run from several racks full of servers. The names of the Florida servers are primarily based on famous historical encyclopedists, while the Kennisnet and Yahoo! machines have been named after various types of plants. (See the discussion page for name suggestions.) Recently there have been so many new servers that many have just been given numerical names.
See also:
 |
Overall system architecture
Note: much of the following is out of date as configurations change quickly and frequently. Server roles may be more up-to-date.
- The master database server runs MySQL and stores all articles.
- The Apaches are running identically-configured Apache web servers. The Apache servers accept requests from users, get data from the database if necessary, and format the requests back to the users, by running the MediaWiki software implemented in PHP with Turck MMCache PHP cache (our experience). They share their work directories by NFS, so uploads etc. should remain quite in sync.
- The Squid systems maintain large caches of pages, so that common or repeated requests don't need to touch the Apache or database servers. They serve most page requests made by visitors who aren't logged in. They are currently running at a hit-rate of approximately 75%, effectively quadrupling the capacity of the Apache servers behind them. This is particularly noticeable when a large surge of traffic arrives directed to a particular page via a web link from another site, as the caching efficiency for that page will be nearly 100%. They also load balance the Apaches. Round-robin DNS is balancing the load among the Squids. See cache strategy for more details.
The system is designed to failover to backup configurations at both the squid and Apache levels, and backup database support is in place, but not in an automatic failover.
(Details on database replication in MySQL: [1])
Hosting
see Wikimedia partners and hosts for more
The cluster near Paris.
Our servers are the three in the middle.
(From top to bottom: bleuenn, chloe, ennael.)
At present all DBs, and most Apaches and Squids, are hosted at the Florida Power Medium data center. From the start of the project until September 2004 Bomis.com paid for all bandwidth.
Lost Oasis (bandwidth) and Jexiste (rack space) in France are providing hosting and bandwidth for the European Squid cache service using chloe, bleuenn and ennael. The machines are installed in the Telecity data center in Aubervilliers near Paris and serve France, Luxembourg and Switzerland.
Kennisnet in the Netherlands is providing hosting and bandwidth for eleven servers since June 2005. They are installed at SARA in Amsterdam and provide service for the Netherlands, Belgium, Germany, UK, Austria, Poland and Scandinavia. Also the Toolserver Zedler is installed here.
Yahoo! is providing servers, hosting, and bandwidth for twenty-three servers in South Korea.
Zedler was donated by Sun Microsystems to Wikimedia Deutschland to provide hosting for miscellaneous tools written by users, and is not part of the main cluster.
Cluster Nomenclature:
- PM TPA - Powermedium / Tampa, Florida, US
- LO PAR - Lost Oasis / Paris, France
- KN AMS - KennisNet / Amsterdam, Netherlands
- YA SEO - Yahoo! / Seoul, South Korea
More equipment, hosting and bandwidth offers for squid cache clusters are welcome, see this page for requirements.
Orders and detailed hardware descriptions
See also: /hardware orders
2006 - 64 new servers
- Hardware purchase Sep-06 (part II not bought mid december)
- Hardware Purchase Jul-06 (~ $61,440)
- Hardware Purchase June-06 (~ $60,000)
- Hardware ordered February 06, 2006 (~ $138,000.)
2005 - 129 new servers
- Hardware ordered November 15, 2005
- Hardware ordered October 18, 2005
- Hardware ordered October 6, 2005
- Hardware ordered September 14, 2005
- Hardware ordered August 30, 2005
- Donated: 23 multi-purpose servers in South Korea
- Hardware ordered May 2005: 2 db servers.
- Hardware deployed May 9, 2005: 20 apache servers (10 with 1GB, 10 with 3GB for memcached use).
- Hardware ordered January 2005: 10 new servers.
- Donated: 2 squid servers in Paris (florence, sophie)
- Donated: 8 squid servers and three multipurpose servers in Amsterdam
2004 - 39 new servers
- Hardware ordered December 2004: 5 new servers: all 3GB apache/memcached/squid type.
- Hardware ordered October 2004: 7 new servers: 2 database, 5 apache
- Hardware ordered August 2004: 10 new servers. Search database server (bacon), NFS storage server (albert), 8 3.0GHz P4 web servers (diderot, goeje, avicenna, dalembert, tingxi, alrazi, friedrich, harris), gigabit ethernet switch, 146GB SCSI drive for Suda.
- Hardware ordered May 2004: 5 new servers. Replacement for Geoffrin database server (ariel), four 2.8GHz P4 general purpose machines (maurus, rabanus, yongle) and a pair of 250GB ATA drives. Based on upgrade discussion April 2004.
- Hardware ordered January 2004: 9 new servers: 8 multipurpose machines (bart, bayle, browne, coronelli, isidore, moreri, vincent, zwinger) and 1 database server (suda).
- Donated: three for Paris squids, extra RAM for them purchased :) :)
Server list
The popularity of the Wikimedia projects necessitates the use of many servers, all but two run the GNU/Linux operating system. Below may be the currentconfiguration, Server roles is likely more up-to-date.
For IP addresses, see here.
| Name | Date | Tasks | OS | Hardware | Named after | |
| 12 | MySQL Database | |||||
| samuel | 05/2005 | Database master | Fedora Core | 2U, 2 x Opteron 250 2.4GHz, 16 GB (8x2G) PC3200/DDR Registered ECC - Interleaved RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Samuel Johnson | |
| ariel | 05/2004 | Database slave | Fedora Core | 2U, 2 x Opteron 248 2.2GHz, 8 GB (8x1G) PC2700 registered ECC RAM, 6x73GB 15K SCA SCSI drives (4 RAID 1+0 (146GB), 2 RAID 1 (72GB)) | Ariel Durant | |
| holbach | 10/2004 | Database slave | Fedora Core | 2U, 2xOpteron 242 1.6GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 6x74GB WD Raptor 10K SATA RAID 0 | Baron d'Holbach | |
| ixia | 09/2005 | Database slave | Fedora Core | 2U, 2 x Opteron 275 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Ixia | |
| lomaria | 09/2005 | Database slave | Fedora Core | 2U, 2 x Opteron 275 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Lomaria | |
| adler | 05/2005 | Database slave | Fedora x86_64 | 2U, 2 x Opteron 250 2.4GHz, 16 GB (8x2G) PC3200/DDR400 Registered ECC - Interleaved RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Mortimer Adler | |
| thistle | 09/2005 | Database slave | Fedora Core | 2U, 2 x Opteron 275 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | Thistle | |
| webster | 10/2004 | Database slave | Fedora Core | 2U, 2xOpteron 242 1.6GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 6x73GB Seagate Cheetah 10K.6 U320 SCSI drives RAID 0 | Noah Webster | |
| db1 .. db4 | 02/2006 | 4x Database slave | Fedora Core | 2U, 2 x Opteron 248 (dual-core) 2.2GHz, 16 GB RAM, 6x Seagate Cheetah 15K.4 73GB U320 15KRPM SCA SCSI RAID 0 | - | |
| 14 | Miscellaneous | |||||
| albert | 08/2004 | NFS storage server. | SUSE Lin. 9.1 | 2U, 2xOpteron 242 1.6GHz, 1 GB (2x512M) PC2700 registered ECC RAM, 6x250GB SATA (RAID 5) (details) | Albertus Magnus | |
| amane | 11/2005 | NFS storage server | Fedora Core | 3U, 2x Xeon 3.4GHz, 8GB RAM, 16 x 400GB SATA disk | Nishi Amane | |
| zwinger | 01/2004 | NFS, mail, DNS, coda (experimental), misc stuff | Red Hat Lin. 9 | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 250 GB SATA (details) | Theodor Zwinger | |
| larousse | Prehistoric | IRC, servmon, cricket, rcdumper, udpmcast, VPN routing, DNS | Fedora Core | 1U, 1 x Pentium III 866MHz, 2 GB RAM, 32GB+18GB SCSI disk | Pierre Larousse | |
| bacon | 08/2004 | Former database slave (being repurposed) | Fedora Core | 2U, 2xOpteron 242 1.6GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 6x200GB SATA RAID 10 | Francis Bacon | |
| suda | 01/2004 | Former database slave (being repurposed) | Fedora Core | 2U, 2 x Opteron 246 2.0GHz, 4 GB (4x1G) PC2700 registered ECC RAM, 4x146GB SCSI RAID 1+0 (292GB) and 2x36GB SCA SCSI RAID 1 (36GB) | Suda | |
| vincent | 01/2004 | Search server | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 80 GB SATA | Vincent of Beauvais | |
| coronelli | 01/2004 | Search server | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 3 GB (2x1G? +?) unbuf non-ECC DDR400 RAM, 80 GB SATA | Vincenzo Coronelli | |
| maurus | 05/2004 | Search server | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB (4x1G) unbuf DDR 400 ECC RAM, 80 GB SATA | Rabanus Maurus | |
| avicenna | 08/2004 | LVS load balancer | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Avicenna | |
| alrazi | 08/2004 | LVS load balancer | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Al-Razi | |
| isidore | 01/2004 | Database for fundraising, misc. tasks | FreeBSD5.3-REL | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 80 GB SATA | Isidore of Seville | |
| khaldun | 10/2004 | Install server | Ubuntu 6.06 LTS | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 2x250 GB SATA RAID 0 | Ibn Khaldun | |
| browne | 01/2004 | Squid (inactive) | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 4 GB (4x1G) unbuf non-ECC DDR400 RAM, 80 GB SATA | Thomas Browne | |
| 18 | Squid | |||||
| srv5 .. srv10 | 01/2005 | 6x Squid | Fedora Core | 1U, 1x Pentium 4 3.2GHz, 3GB RAM, 2x 36GB SATA | - | |
| sq1 .. sq10 | 11/2005 | 10x Upload/Image Squids | Fedora Core | 1U, 2xOpteron 248 (2.2Ghz), 4x1GB DDR ECC RAM, 4x 36GB 10k SATA | - | |
| bayle | 01/2004 | Apache, 1120MB memcached, switched between Squid and Apache as needed | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 2 GB (2x1G) unbuf DDR400 non-ECC RAM, 80 GB SATA | Pierre Bayle | |
| will | 05/2004 | Squid | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB RAM, 2 x 200GB SATA in RAID 1 (details) | Will Durant | |
| 158 | Apache | |||||
| srv0 | 10/2004 | Apache, DB backup slave | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 2x 250GB SATA RAID 0 | Averroes | |
| srv1 .. srv4 | 01/2005 | 4x Apache | Fedora Core | 1U, 1x Pentium 4 3.2GHz, 1GB RAM, 1x 200GB SATA | - | |
| srv11 .. srv20 | 05/2005 | 10x Apache | Fedora Core | 1U, 1x Pentium 4 3.4GHz, 1GB RAM, 1x 80GB SATA | - | |
| srv21 .. srv27 | 05/2005 | 7x Apache | Fedora Core | 1U, 1x Pentium 4 3.4GHz, 3GB RAM, 1x 80GB SATA | - | |
| srv28 .. srv30 | 05/2005 | 4x Apache, external storage | Fedora Core | 1U, 1x Pentium 4 3.4GHz, 3GB RAM, 1x 80GB SATA | - | |
| srv31 .. srv50 | 09/2005 | 10x Apache | Fedora Core | 1U, 2x Opteron 248 (2.2GHz), 2x1GB DDR ECC RAM, 1x 250GB 7.2k SATA | - | |
| srv51 .. srv70 | 10/2005 | 20x Apache | Fedora Core | 1U, 2xOpteron 248 (2.2Ghz), 4x1GB DDR ECC RAM, 1x 7.2k 250GB SATA | - | |
| srv71 .. srv80 | 11/2005 | 10x Apache | Fedora Core | 1U, 2xOpteron 248 (2.2Ghz), 4x1GB DDR ECC RAM, 1x 250GB 7.2k SATA | - | |
| srv81 .. srv120 | 02/2006 | 40x Apache | Fedora Core | 1U, 2xOpteron 265 (1.8Ghz), 4x1GB DDR ECC RAM, 2x 250GB 7.2k SATA | - | |
| anthony | 12/2004 | Apache/ Memcached | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 1x200 GB SATA RAID 0 | Anthony Winkler Prins | |
| bart | 01/2004 | Apache | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 4 GB (4x1G) unbuf DDR400 non-ECC RAM, 80 GB SATA | Bartholomeus de Glanvilla | |
| benet | 12/2004 | Apache/ Memcached/ Squid. Database slave, test production use. | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 2x250 GB SATA RAID 0 | William Rose Benet | |
| biruni | 12/2004 | Apache/ Memcached/ Squid. | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 1x200 GB SATA | Al-Biruni | |
| dalembert | 08/2004 | LVS | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Jean le Rond d'Alembert | |
| diderot, friedrich, goeje, harris | 08/2004 | 4x Apache. | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 ECC RAM, 200 GB SATA | Denis Diderot, Friedrich Arnold Brockhaus, Michael Jan de Goeje, John Harris | |
| hypatia | 10/2004 | Apache, IRC bots | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 1x 200GB SATA | Hypatia of Alexandria | |
| humboldt, kluge | 10/2004 | 2x Apache | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 1 GB RAM, 1x 200GB SATA | Alexander von Humboldt, Friedrich Kluge | |
| moreri | 01/2004 | Apache, 280MB memcached | Fedora Core | 1U, 1 x Pentium 4 2.6 GHz, 1 GB (cap?) unbuf DDR400 non-ECC RAM, 80 GB SATA | Louis Moréri | |
| rabanus | 05/2004 | Apache, DHCP boot server | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB (4x1G) unbuf DDR 400 ECC RAM, 80 GB SATA | Rabanus Maurus | |
| rose, smellie | 12/2004 | 2x Apache / Memcached. | Fedora Core | 1U, 1x Pentium 4 3.0 GHz, 2x1 GB + 2x512 MB DDRII-533 RAM, 1x200 GB SATA | William Rose Benet, William Smellie | |
| tingxi | 08/2004 | Apache, IRC bots. Currently malfunctioning and in need of developer attention | Fedora Core | 1U, 1 x Pentium 4 3 GHz, 0.5 GB (2x256M) unbuf DDR400 RAM, 200 GB SATA | Jiang Tingxi | |
| yongle | 05/2004 | Apache, 1960MB memcached | Fedora Core | 1U, 1 x Pentium 4 2.8 GHz, 4 GB (4x1G) unbuf DDR 400 ECC RAM, 80 GB SATA | Emperor Yongle | |
| 3 | Paris Cluster | |||||
| bleuenn | 12/2004 | Former Squid in France, not in use | Debian | 1U, 1 x Celeron 600 MHz, 640 MB RAM, 20 GB ATA. HP Hosting Server Appliance sa1100 | donor request | |
| chloe | 12/2004 | Former Squid in France, not in use | Debian | 1U, 1 x Celeron 600 MHz, 640 MB RAM, 20 GB ATA. HP Hosting Server Appliance sa1100 | donor request | |
| ennael | 12/2004 | Former Squid in France, not in use | Debian | 1U, 1 x Celeron 600 MHz, 768 MB RAM, 20 GB ATA. HP Hosting Server Appliance sa1100 | donor request | |
| 12 | Kennisnet cluster | |||||
| pascal | 5/2005 | WWW, DNS, NFS | Fedora Core | Sun V40z, 2 x 1.8GHz Opteron, 2GB RAM, 2x 70GB SCSI | Blaise Pascal | |
| vandale | 5/2005 | Squid | Fedora Core | Sun V40z, 2 x 1.8Ghz Opteron, 8GB RAM, 6x 146GB SCSI | Johan Hendrik van Dale | |
| clematis | 5/2005 | Squid | Fedora Core | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| fuchsia | 5/2005 | DNS, WWW as http://www2.knams.wikimedia.org, statistics | Fedora Core | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | ||
| hawtorn, iris, lily, mayflower, mint, sage | 5/2005 | 6x Squid | Fedora Core | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| ragweed | 5/2005 | Squid/OTRS | Fedora Core | Sun V20z, 2 x 2.2GHz Opteron, 3GB RAM, 2x 70GB SCSI | - | |
| zedler | 8/2005 | Toolserver | Solaris 10 | Sun V40z, 2 x 2.2GHz Opteron, 8GB RAM, 6x 146GB SCSI | Johann Heinrich Zedler | |
| 23 | Korea Yahoo! cluster | |||||
| amaryllis | 9/2005 | NFS | Fedora Core | HP DL385, 2x Opteron 250 (2.4GHz), 8GB RAM, 6x 146GB disk | - | |
| dryas | 9/2005 | MySQL master | Fedora Core | HP DL385, 2x Opteron 250 (2.4GHz), 8GB RAM, 6x 146GB disk | - | |
| henbane | 9/2005 | MySQL slave | Fedora Core | HP DL385, 2x Opteron 250 (2.4GHz), 8GB RAM, 6x 146GB disk | - | |
| yf1000 ... yf1004 | 9/2005 | 5x Squid | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | ||
| yf1005 ... yf1016 | 9/2005 | 12x Apache | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1017 | 9/2005 | Search server | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1018 | 9/2005 | LVS load balancer | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| yf1019 | 9/2005 | Squid | Fedora Core | HP DL140, 2x Opteron 246 (2.0GHz), 4GB RAM, 1x 80GB HDD | - | |
| 240 | Total | |||||
Old servers
The old web servers, which are currently not in service, were also named after historical encyclopedists:
- "Pliny" (Pliny the Elder)
- "Geoffrin" (Marie Thérèse Rodet Geoffrin)
Donations
While Wikipedia is free in both the "free speech" and "no charge" senses of the word, running the web site does cost money. You can help with purchasing new server hardware by donating to the non-profit Wikimedia Foundation: http://wikimediafoundation.org/wiki/Fundraising
Status and problems
You can check one of the following sites if you want to know if the Wikimedia servers are overloaded, or if you just want to see how they are doing.
If you are seeing errors in real time, visit #wikimedia-tech on irc.freenode.net. Check the topic to see if someone is already looking into the problem you are having. If not, please report your problem to the channel. It would be helpful if you could report specific symptoms, including the exact text of any error messages, what you were doing right before the error, and what server(s) are generating the error, if you can tell. The #wikipedia channel may be better populated and the topic there may contain more updated information about the status of the problem. (But this channel is for general conversation about Wikipedia.)
If you don't use IRC, or don't get a reply, you can also send e-mail to noc@wikimedia.org. Someone will create a report in Bugzilla (under the "Issues" product) so others will know the problem has been reported, and what the status is. You can check here to see if your problem has already been reported.
If you're wondering if it's only you experiencing this problem, you can check the following sites. Unfortunately, the Wikimedia administrators do not monitor these sites for problems.
If you are getting a "connection refused" error, that is a squid problem. Determine which IP address you are trying to connect to, and ask someone to look at that host.
See also
More hardware info
- Recent hardware orders, accounting summary
- Wikimedia partners and hosts
- Technical FAQ - How about the hardware?
Admin logs
- Server admin log - Documents server changes (especially software changes)
Offsite traffic pages
Long-term planning
Useful information about other sites
- Evolution of LiveJournal systems:
- 04/2004 MySQLCon 2004 PDF/SXI
- 07/2004 OSCON 2004 PDF/SXI
- 11/2004 LISA 2004 PDF/SXI
- 04/2005 MySQLCon 2005 PDF/PPT/SXI
- journals to watch for system details: Brad (Fitzpatrick) lj_backend lj_maintenance
- Google cluster architecture (PDF)
- MySQL User's Conference 2004 blog highlights