Content deleted Content added
Jerry Biggle (talk | contribs) No edit summary |
add link to beowulf clusters |
||
| (25 intermediate revisions by 16 users not shown) | |||
Line 1:
{{Short description|Set of computers configured in a distributed computing system}}
{{Distinguish|data cluster|grid computing}}
{{Redirect|Cluster computing|the journal|Cluster Computing (journal)}}
[[File:MEGWARE.CLIC.jpg|thumb|Technicians working on a large [[Linux]] cluster at the [[Chemnitz University of Technology]], Germany]]
[[File:Sun Microsystems Solaris computer cluster.jpg|thumb|Sun Microsystems [[Solaris Cluster]], with [[Close Coupled Cooling#In-Row Air Conditioners|In-Row cooling]]]]
[[File:Taiwania series.jpg|thumb|[[Taiwania_(supercomputer)|Taiwania]] series uses cluster architecture
A '''computer cluster''' is a set of [[
The components of a cluster are usually connected to each other through fast [[local area network]]s, with each [[Node (networking)|node]] (computer used as a server) running its own instance of an [[operating system]]. In most circumstances, all of the nodes use the same hardware<ref>{{cite web |url=https://stackoverflow.com/questions/9723040/what-is-the-difference-between-cloud-grid-and-cluster |title=Cluster vs grid computing |website=[[Stack Overflow]]}}</ref>{{better source needed|date=June 2017}} and the same operating system, although in some setups (e.g. using [[Open Source Cluster Application Resources]] (OSCAR)), different operating systems can be used on each computer, or different hardware.<ref name=pcauthority>{{cite web|url=http://www.pcauthority.com.au/Feature/306972,weekend-project-build-your-own-supercomputer.aspx|title=Weekend Project: Build your own supercomputer|date=29 June 2012|first=Darien|last=Graham-Smith|website=PC & Tech Authority|access-date=2 June 2017}}</ref>
Clusters are usually deployed to improve performance and availability over that of a single computer, while typically being much more cost-effective than single computers of comparable speed or availability.<ref>{{cite web|url=http://www.cc.gatech.edu/~bader/papers/ijhpca.html|title=Cluster Computing: Applications|last1=Bader|first1=David|author-link=David Bader (computer scientist)|date=May 2001|publisher=[[Georgia Institute of Technology College of Computing|Georgia Tech College of Computing]]|first2=Robert|last2=Pennington|access-date=2017-02-28|archive-url=https://web.archive.org/web/20071221011621/http://www.cc.gatech.edu/~bader/papers/ijhpca.html|archive-date=2007-12-21|url-status=dead}}</ref>
Computer clusters emerged as a result of the convergence of a number of computing trends including the availability of low-cost microprocessors, high-speed networks, and software for high-performance [[distributed computing]].{{citation needed|date=October 2014}} They have a wide range of applicability and deployment, ranging from small business clusters with a handful of nodes to some of the fastest [[supercomputer]]s in the world such as [[IBM Sequoia|IBM's Sequoia]].<ref>{{cite web|url=https://www.telegraph.co.uk/technology/9338651/Nuclear-weapons-supercomputer-reclaims-world-speed-record-for-US.html |archive-url=https://ghostarchive.org/archive/20220112/https://www.telegraph.co.uk/technology/9338651/Nuclear-weapons-supercomputer-reclaims-world-speed-record-for-US.html |archive-date=2022-01-12 |url-access=subscription |url-status=live|title=Nuclear weapons supercomputer reclaims world speed record for US|publisher=The Telegraph|date=18 Jun 2012|access-date=18 Jun 2012}}{{cbignore}}</ref> Prior to the advent of clusters, single
==Basic concepts==
[[File:Beowulf.jpg|thumb|150px|A simple, home-built [[Beowulf cluster]]
The desire to get more computing power and better reliability by orchestrating a number of low-cost [[commercial off-the-shelf]] computers has given rise to a variety of architectures and configurations.
Line 24 ⟶ 25:
A computer cluster may be a simple two-node system which just connects two personal computers, or may be a very fast [[supercomputer]]. A basic approach to building a cluster is that of a [[Beowulf (computing)|Beowulf]] cluster which may be built with a few personal computers to produce a cost-effective alternative to traditional [[high-performance computing]]. An early project that showed the viability of the concept was the 133-node [[Stone Soupercomputer]].<ref name="sciam">{{Cite news |title= The Do-It-Yourself Supercomputer |work= [[Scientific American]] |author= William W. Hargrove, Forrest M. Hoffman and [[Thomas Sterling (computing)|Thomas Sterling]] |volume= 265 |number= 2 |pages= 72–79 |date= August 16, 2001 |url= http://www.sciam.com/article.cfm?id=the-do-it-yourself-superc |access-date= October 18, 2011 }}</ref> The developers used [[Linux]], the [[Parallel Virtual Machine]] toolkit and the [[Message Passing Interface]] library to achieve high performance at a relatively low cost.<ref name="extreme">{{Cite news |title= Cluster Computing: Linux Taken to the Extreme |first1= William W. |last1= Hargrove |first2= Forrest M. |last2= Hoffman |work= Linux Magazine |year= 1999 |url= http://climate.ornl.gov/~forrest/linux-magazine-1999/ |access-date= October 18, 2011 |archive-url= https://web.archive.org/web/20111018122713/http://climate.ornl.gov/~forrest/linux-magazine-1999/ |archive-date= October 18, 2011 |url-status= dead }}</ref>
Although a cluster may consist of just a few personal computers connected by a simple network, the cluster architecture may also be used to achieve very high levels of performance. The [[TOP500]] organization's semiannual list of the 500 fastest
==History==
{{Main|History of computer clusters}}
{{See also|History of supercomputing}}
[[File:SPEC-1 VAX 05.jpg|thumb|150px|A [[VAX]] 11/780, c. 1977, as used in early [[VAXcluster]] development]]
Greg Pfister has stated that clusters were not invented by any specific vendor but by customers who could not fit all their work on one computer, or needed a backup.<ref>{{cite book | last = Pfister | first = Gregory | title = In Search of Clusters | edition = 2nd | publisher = Prentice Hall PTR | ___location = Upper Saddle River, NJ | year = 1998 | page = [https://archive.org/details/insearchofcluste00pfis/page/36 36] | isbn = 978-0-13-899709-0 | url = https://archive.org/details/insearchofcluste00pfis/page/36 }}</ref> Pfister estimates the date as some time in the 1960s. The formal engineering basis of cluster computing as a means of doing parallel work of any sort was arguably invented by [[Gene Amdahl]] of [[IBM]], who in 1967 published what has come to be regarded as the seminal paper on parallel processing: [[Amdahl's Law]].
Line 36 ⟶ 37:
The first production system designed as a cluster was the Burroughs [[B5700]] in the mid-1960s. This allowed up to four computers, each with either one or two processors, to be tightly coupled to a common disk storage subsystem in order to distribute the workload. Unlike standard multiprocessor systems, each computer could be restarted without disrupting overall operation.
[[File:TNSII.jpg|thumb|Tandem NonStop II circa 1980
The first commercial loosely coupled clustering product was [[Datapoint|Datapoint Corporation's]] "Attached Resource Computer" (ARC) system, developed in 1977, and using [[ARCnet]] as the cluster interface. Clustering per se did not really take off until [[Digital Equipment Corporation]] released their [[VAXcluster]] product in 1984 for the [[OpenVMS|VMS]] operating system. The ARC and VAXcluster products not only supported [[parallel computing]], but also shared [[file system]]s and [[peripheral]] devices. The idea was to provide the advantages of parallel processing, while maintaining data reliability and uniqueness. Two other noteworthy early commercial clusters were the [[Tandem Computers|''Tandem NonStop'']] (a 1976 high-availability commercial product)<ref>{{Cite book |last=Katzman |first=James A. |title=Computer Structure: Principles and Examples |publisher=McGraw-Hill Book Company |year=1982 |isbn= |editor-last=Siewiorek |editor-first=Donald P. |___location=U.S.A. |pages=470–485 |chapter=Chapter 29, The Tandem 16: A Fault-Tolerant Computing System}}</ref><ref>{{Cite web |title=History of TANDEM COMPUTERS, INC. – FundingUniverse |url=http://www.fundinguniverse.com/company-histories/tandem-computers-inc-history/ |access-date=2023-03-01 |website=www.fundinguniverse.com}}</ref> and the ''IBM S/390 Parallel Sysplex'' (circa 1994, primarily for business use).
Within the same time frame, while computer clusters used parallelism outside the computer on a commodity network, [[supercomputer]]s began to use them within the same computer. Following the success of the [[CDC 6600]] in 1964, the [[Cray 1]] was delivered in 1976, and introduced internal parallelism via [[vector processor|vector processing]].<ref name=Hill41 >{{cite book|title=Readings in computer architecture|first1=Mark Donald|last1=Hill|author-link2=Norman Jouppi|first2=Norman Paul|last2=Jouppi|first3=Gurindar|last3=Sohi|year=1999|isbn=978-1-55860-539-8|pages=41–48|publisher=Gulf Professional }}</ref> While early supercomputers excluded clusters and relied on [[shared memory architecture|shared memory]], in time some of the fastest supercomputers (e.g. the [[K computer]]) relied on cluster architectures.
==Attributes of clusters==
[[File:Load Balancing Cluster (NAT).svg|thumb|A load balancing cluster with two servers and N user stations
Computer clusters may be configured for different purposes ranging from general purpose business needs such as web-service support, to computation-intensive scientific calculations. In either case, the cluster may use a [[high-availability cluster|high-availability]] approach. Note that the attributes described below are not exclusive and a "computer cluster" may also use a high-availability approach, etc.
"[[Load balancing (computing)|Load-balancing]]" clusters are configurations in which cluster-nodes share computational workload to provide better overall performance. For example, a web server cluster may assign different queries to different nodes, so the overall response time will be optimized.<ref name=Sloan>{{cite book|title=High Performance Linux Clusters|url=https://archive.org/details/highperformancel0000sloa|url-access=registration|first=Joseph D.|last=Sloan|year=2004|publisher="O'Reilly Media, Inc." |isbn=978-0-596-00570-2}}</ref> However, approaches to load-balancing may significantly differ among applications, e.g. a high-performance cluster used for scientific computations would balance load with different algorithms from a web-server cluster which may just use a simple [[round-robin scheduling|round-robin method]] by assigning each new request to a different node.<ref name=Sloan />
Computer clusters are used for computation-intensive purposes, rather than handling [[Input/output|IO-oriented]] operations such as web service or databases.<ref name=VECPAR >{{cite book|title=High Performance Computing for Computational Science
"[[High-availability cluster]]s" (also known as [[failover]] clusters, or HA clusters) improve the availability of the cluster approach. They operate by having redundant [[Node (networking)|nodes]], which are then used to provide service when system components fail. HA cluster implementations attempt to use redundancy of cluster components to eliminate [[single point of failure|single points of failure]]. There are commercial implementations of High-Availability clusters for many operating systems. The [[Linux-HA]] project is one commonly used [[free software]] HA package for the [[Linux]] operating system.
Line 53 ⟶ 54:
==Benefits==
<!-- This used to be a list. Work has been done since, but it's still incomplete. -->
Clusters are primarily designed with performance in mind, but installations are based on many other factors. Fault tolerance (''the ability
In terms of scalability, clusters provide this in their ability to add nodes horizontally. This means that more computers may be added to the cluster, to improve its performance, redundancy and fault tolerance. This can be an inexpensive solution for a higher performing cluster compared to scaling up a single node in the cluster. This property of computer clusters can allow for larger computational loads to be executed by a larger number of lower performing computers.
Line 62 ⟶ 63:
==Design and configuration==
[[File:beowulf.png|thumb|240px|left|A typical Beowulf configuration
One of the issues in designing a cluster is how tightly coupled the individual nodes may be. For instance, a single computer job may require frequent communication among nodes: this implies that the cluster shares a dedicated network, is densely located, and probably has homogeneous nodes. The other extreme is where a computer job uses one or few nodes, and needs little or no inter-node communication, approaching [[grid computing]].
In a [[Beowulf cluster]], the application programs never see the computational nodes (also called slave computers) but only interact with the "Master" which is a specific computer handling the scheduling and management of the slaves.<ref name=VECPAR /> In a typical implementation the Master has two network interfaces, one that communicates with the private Beowulf network for the slaves, the other for the general purpose network of the organization.<ref name=VECPAR /> The slave computers typically have their own version of the same operating system, and local memory and disk space. However, the private slave network may also have a large and shared file server that stores global persistent data, accessed by the slaves as needed.<ref name=VECPAR />
A special purpose 144-node [[DEGIMA (computer cluster)|DEGIMA cluster]] is tuned to running astrophysical N-body simulations using the Multiple-Walk parallel tree code, rather than general purpose scientific computations.<ref name=Hamada>{{cite journal|first=Tsuyoshi|last=Hamada |display-authors=etal |year=2009|title=A novel multiple-walk parallel algorithm for the Barnes–Hut treecode on GPUs – towards cost effective, high performance N-body simulation|journal=Computer Science
Due to the increasing computing power of each generation of [[game console]]s, a novel use has emerged where they are repurposed into [[High-performance computing]] (HPC) clusters. Some examples of game console clusters are [[PlayStation 3 cluster|Sony PlayStation clusters]] and [[Microsoft]] [[Xbox (console)|Xbox]] clusters. Another example of consumer game product is the [[Nvidia Tesla Personal Supercomputer]] workstation, which uses multiple graphics accelerator processor chips. Besides game consoles, high-end graphics cards too can be used instead. The use of graphics cards (or rather their GPU's) to do calculations for grid computing is vastly more economical than using CPU's, despite being less precise. However, when using double-precision values, they become as precise to work with as CPU's and are still much less costly (purchase cost).<ref name=pcauthority />
Line 77 ⟶ 78:
===Data sharing===
[[File:Nec-cluster.jpg|thumb|A [[NEC]] [[Nehalem (microarchitecture)|Nehalem cluster]]]]
As the computer clusters were appearing during the 1980s, so were [[supercomputer]]s. One of the elements that distinguished the three classes at that time was that the early supercomputers relied on [[Shared memory architecture|shared memory]].
However, the use of a [[clustered file system]] is essential in modern computer clusters.{{Citation needed|date=August 2013}} Examples include the [[IBM General Parallel File System]], Microsoft's [[Cluster Shared Volumes]] or the [[Oracle Cluster File System]].
Line 83 ⟶ 84:
===Message passing and communication===
{{Main|Message passing in computer clusters}}
Two widely used approaches for communication between cluster nodes are MPI ([[Message Passing Interface]]) and PVM ([[Parallel Virtual Machine]]).<ref name=Gehrke>{{cite book|title=Distributed services with OpenAFS: for enterprise and education|first1=Franco|last1=Milicchio|first2=Wolfgang Alexander|last2=Gehrke|year=2007|isbn=9783540366348|pages=339–341|publisher=Springer |url=https://books.google.com/books?id=_HtHG2Ca5AEC}}</ref>
PVM was developed at the [[Oak Ridge National Laboratory]] around 1989 before MPI was available. PVM must be directly installed on every cluster node and provides a set of software libraries that paint the node as a "parallel virtual machine". PVM provides a run-time environment for message-passing, task and resource management, and fault notification. PVM can be used by user programs written in C, C++, or Fortran, etc.<ref name="Gehrke" /><ref name="Prabhu" />
MPI emerged in the early 1990s out of discussions among 40 organizations. The initial effort was supported by [[DARPA|ARPA]] and [[National Science Foundation]]. Rather than starting anew, the design of MPI drew on various features available in commercial systems of the time. The MPI specifications then gave rise to specific implementations. MPI implementations typically use [[TCP/IP]] and socket connections.<ref name=Gehrke /> MPI is now a widely available communications model that enables parallel programs to be written in languages such as [[C (programming language)|C]], [[Fortran]], [[Python (programming language)|Python]], etc.<ref name=Prabhu >{{cite book|url=https://books.google.com/books?id=evcgB7Qlix4C|title=Grid and Cluster Computing|last=Prabhu|first=C.S.R.|year=2008|isbn=978-8120334281|pages=109–112|publisher=PHI Learning Pvt. }}</ref> Thus, unlike PVM which provides a concrete implementation, MPI is a specification which has been implemented in systems such as [[MPICH]] and [[Open MPI]].<ref name="Prabhu" /><ref name=Gropp>{{Cite journal
|last1=Gropp |first1=William |last2=Lusk |first2=Ewing |last3=Skjellum |first3=Anthony
|year=1996|citeseerx = 10.1.1.102.9485
Line 95 ⟶ 96:
==Cluster management==
[[File:Cubieboard HADOOP cluster.JPG|thumb|Low-cost and low energy tiny-cluster of [[Cubieboard]]s, using [[Apache Hadoop]] on [[Lubuntu]]]]
[[File:Circumference C25 (41227579055).png|thumb|A pre-release sample of the Ground Electronics/AB Open Circumference C25 cluster [[
One of the challenges in the use of a computer cluster is the cost of administrating it which can at times be as high as the cost of administrating N independent machines, if the cluster has N nodes.<ref name=patter641 >{{cite book|title=Computer Organization and Design|first1=David A.|last1=Patterson|first2=John L.|last2=Hennessy|year=2011|isbn=978-0-12-374750-1|pages=641–642|publisher=Elsevier }}</ref> In some cases this provides an advantage to [[shared memory architecture]]s with lower administration costs.<ref name=patter641 /> This has also made [[virtual machine]]s popular, due to the ease of administration.<ref name=patter641 />
===Task scheduling===
Line 113 ⟶ 114:
Load balancing clusters such as web servers use cluster architectures to support a large number of users and typically each user request is routed to a specific node, achieving [[task parallelism]] without multi-node cooperation, given that the main goal of the system is providing rapid user access to shared data. However, "computer clusters" which perform complex computations for a small number of users need to take advantage of the parallel processing capabilities of the cluster and partition "the same computation" among several nodes.<ref name=Blum />
[[Automatic parallelization]] of programs remains a technical challenge, but [[parallel programming model]]s can be used to effectuate a higher [[degree of parallelism]] via the simultaneous execution of separate portions of a program on different processors.<ref name=Blum >{{cite book|title=Computer Science: The Hardware, Software and Heart of It|first1=Alfred V.|last1=Aho|first2=Edward K.|last2=Blum|year=2011|isbn=978-1-4614-1167-3|pages=156–166|publisher=Springer |url=https://books.google.com/books?id=S7QU9RRLYIYC}}</ref><ref>{{cite book|title=Parallel Programming: For Multicore and Cluster Systems|first1=Thomas|last1=Rauber|first2=Gudula|last2=Rünger|year=2010|isbn=978-3-642-04817-3|pages=94–95|publisher=Springer |url=https://books.google.com/books?id=wWogxOmA3wMC}}</ref>
===Debugging and monitoring===
Developing and debugging parallel programs on a cluster requires parallel language primitives and suitable tools such as those discussed by the ''High Performance Debugging Forum'' (HPDF) which resulted in the HPD specifications.<ref name="Prabhu"/><ref name=iosp>{{cite journal|last1=Francioni|first1=Joan M.|last2=Pancake|first2=Cherri M.|author-link2=Cherri M. Pancake|title=A Debugging Standard for High-performance computing|journal=Scientific Programming|volume=8|issue=2|date=April 2000
The [[University of California, Berkeley]] ''Network of Workstations'' (NOW) system gathers cluster data and stores them in a database, while a system such as PARMON, developed in India, allows visually observing and managing large clusters.<ref name="Prabhu"/>
Line 123 ⟶ 124:
==Implementations==
The Linux world supports various cluster software; for application clustering, there is [[distcc]], and [[MPICH]]. [[Linux Virtual Server]], [[Linux-HA]]
[[Microsoft Windows]] computer cluster Server 2003 based on the [[Windows Server]] platform provides pieces for high-performance computing like the job scheduler, MSMPI library and management tools.
Line 158 ⟶ 159:
:* [[Solaris Cluster]]
:* [[Veritas Cluster Server]]
:* [[Beowulf cluster]]
''Computer farms''
Line 178 ⟶ 180:
{{Commons category|Clusters (computing)}}
* [https://web.archive.org/web/20190219183441/https://www.ieeetcsc.org/ IEEE Technical Committee on Scalable Computing (TCSC)]
* [https://archive.today/20130103192843/http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=
* [https://www.ibm.com/developerworks/wikis/display/tivoli/Tivoli+System+Automation Tivoli System Automation Wiki]
* [https://static.googleusercontent.com/media/research.google.com/en/us/pubs/archive/43438.pdf Large-scale cluster management at Google with Borg], April 2015, by Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune and John Wilkes
| |||