Content deleted Content added
m punc |
Citation bot (talk | contribs) Add: publisher. | Use this bot. Report bugs. | Suggested by Abductive | #UCB_webform 2672/3849 |
||
Line 83:
===Message passing and communication===
{{Main|Message passing in computer clusters}}
Two widely used approaches for communication between cluster nodes are MPI ([[Message Passing Interface]]) and PVM ([[Parallel Virtual Machine]]).<ref name=Gehrke>{{cite book|title=Distributed services with OpenAFS: for enterprise and education|first1=Franco|last1=Milicchio|first2=Wolfgang Alexander|last2=Gehrke|year=2007|isbn=9783540366348|pages=339–341|publisher=Springer |url=https://books.google.com/books?id=_HtHG2Ca5AEC}}</ref>
PVM was developed at the [[Oak Ridge National Laboratory]] around 1989 before MPI was available. PVM must be directly installed on every cluster node and provides a set of software libraries that paint the node as a "parallel virtual machine". PVM provides a run-time environment for message-passing, task and resource management, and fault notification. PVM can be used by user programs written in C, C++, or Fortran, etc.<ref name="Gehrke" /><ref name="Prabhu" />
MPI emerged in the early 1990s out of discussions among 40 organizations. The initial effort was supported by [[DARPA|ARPA]] and [[National Science Foundation]]. Rather than starting anew, the design of MPI drew on various features available in commercial systems of the time. The MPI specifications then gave rise to specific implementations. MPI implementations typically use [[TCP/IP]] and socket connections.<ref name=Gehrke /> MPI is now a widely available communications model that enables parallel programs to be written in languages such as [[C (programming language)|C]], [[Fortran]], [[Python (programming language)|Python]], etc.<ref name=Prabhu >{{cite book|url=https://books.google.com/books?id=evcgB7Qlix4C|title=Grid and Cluster Computing|last=Prabhu|first=C.S.R.|year=2008|isbn=978-8120334281|pages=109–112|publisher=PHI Learning Pvt. }}</ref> Thus, unlike PVM which provides a concrete implementation, MPI is a specification which has been implemented in systems such as [[MPICH]] and [[Open MPI]].<ref name="Prabhu" /><ref name=Gropp>{{Cite journal
|last1=Gropp |first1=William |last2=Lusk |first2=Ewing |last3=Skjellum |first3=Anthony
|year=1996|citeseerx = 10.1.1.102.9485
Line 113:
Load balancing clusters such as web servers use cluster architectures to support a large number of users and typically each user request is routed to a specific node, achieving [[task parallelism]] without multi-node cooperation, given that the main goal of the system is providing rapid user access to shared data. However, "computer clusters" which perform complex computations for a small number of users need to take advantage of the parallel processing capabilities of the cluster and partition "the same computation" among several nodes.<ref name=Blum />
[[Automatic parallelization]] of programs remains a technical challenge, but [[parallel programming model]]s can be used to effectuate a higher [[degree of parallelism]] via the simultaneous execution of separate portions of a program on different processors.<ref name=Blum >{{cite book|title=Computer Science: The Hardware, Software and Heart of It|first1=Alfred V.|last1=Aho|first2=Edward K.|last2=Blum|year=2011|isbn=978-1-4614-1167-3|pages=156–166|publisher=Springer |url=https://books.google.com/books?id=S7QU9RRLYIYC}}</ref><ref>{{cite book|title=Parallel Programming: For Multicore and Cluster Systems|first1=Thomas|last1=Rauber|first2=Gudula|last2=Rünger|year=2010|isbn=978-3-642-04817-3|pages=94–95|publisher=Springer |url=https://books.google.com/books?id=wWogxOmA3wMC}}</ref>
===Debugging and monitoring===
| |||