Content deleted Content added
Citation bot (talk | contribs) Alter: url, template type. URLs might have been anonymized. Add: s2cid. Removed parameters. | Use this bot. Report bugs. | Suggested by Abductive | #UCB_webform 3342/3850 |
Rescuing 19 sources and tagging 0 as dead.) #IABot (v2.0.9.3 |
||
Line 3:
{{Distinguish|Cluster computing}}
'''Grid computing''' is the use of widely distributed [[computer]] [[System resource|resources]] to reach a common goal. A computing grid can be thought of as a [[distributed system]] with non-interactive workloads that involve many files. Grid computing is distinguished from conventional high-performance computing systems such as [[Cluster (computing)|cluster]] computing in that grid computers have each node set to perform a different task/application. Grid computers also tend to be more [[heterogeneous]] and geographically dispersed (thus not physically coupled) than cluster computers.<ref>[http://www.e-sciencecity.org/EN/gridcafe/what-is-the-grid.html What is grid computing? - Gridcafe] {{Webarchive|url=https://web.archive.org/web/20130210072831/http://www.e-sciencecity.org/EN/gridcafe/what-is-the-grid.html |date=2013-02-10 }}. E-sciencecity.org. Retrieved 2013-09-18.</ref> Although a single grid can be dedicated to a particular application, commonly a grid is used for a variety of purposes. Grids are often constructed with general-purpose grid [[middleware]] software libraries. Grid sizes can be quite large.<ref>{{cite web |url=http://www.networkworld.com/article/2339444/software/scale-grid-computing-down-to-size.html |title=Scale grid computing down to size |publisher=NetworkWorld.com |date=2003-01-27 |access-date=2015-04-21 |archive-date=2017-02-17 |archive-url=https://web.archive.org/web/20170217083952/http://www.networkworld.com/article/2339444/software/scale-grid-computing-down-to-size.html |url-status=live }}</ref>
Grids are a form of [[distributed computing]] composed of many networked [[Loose coupling|loosely coupled]] computers acting together to perform large tasks. For certain applications, distributed or grid computing can be seen as a special type of [[parallel computing]] that relies on complete computers (with onboard CPUs, storage, power supplies, network interfaces, etc.) connected to a [[computer network]] (private or public) by a conventional [[Network interface controller|network interface]], such as [[Ethernet]]. This is in contrast to the traditional notion of a [[supercomputer]], which has many processors connected by a local high-speed [[computer bus]]. This technology has been applied to computationally intensive scientific, mathematical, and academic problems through [[volunteer computing]], and it is used in commercial enterprises for such diverse applications as [[drug discovery]], [[economic forecasting]], [[seismic analysis]], and [[back office]] data processing in support for [[e-commerce]] and [[Web service]]s.
Grid computing combines computers from multiple administrative domains to reach a common goal,<ref name="autogenerated1">{{cite web|url=http://dlib.cs.odu.edu/WhatIsTheGrid.pdf|title=What is the Grid? A Three Point Checklist|access-date=2010-10-21|archive-date=2014-11-22|archive-url=https://web.archive.org/web/20141122035905/http://dlib.cs.odu.edu/WhatIsTheGrid.pdf|url-status=dead}}</ref> to solve a single task, and may then disappear just as quickly. The size of a grid may vary from small—confined to a network of computer workstations within a corporation, for example—to large, public collaborations across many companies and networks. "The notion of a confined grid may also be known as an intra-nodes cooperation whereas the notion of a larger, wider grid may thus refer to an inter-nodes cooperation".<ref>{{cite web |url=http://diuf.unifr.ch/pai/wiki/doku.php?id=Publications&page=publication&kind=single&ID=276 |title=Pervasive and Artificial Intelligence Group :: publications [Pervasive and Artificial Intelligence Research Group] |publisher=Diuf.unifr.ch |date=May 18, 2009 |access-date=July 29, 2010 |archive-url=https://web.archive.org/web/20110707004350/http://diuf.unifr.ch/pai/wiki/doku.php?id=Publications&page=publication&kind=single&ID=276 |archive-date=July 7, 2011 |url-status=dead }}</ref>
Coordinating applications on Grids can be a complex task, especially when coordinating the flow of information across distributed computing resources. [[Scientific workflow system|Grid workflow]] systems have been developed as a specialized form of a [[workflow management system]] designed specifically to compose and execute a series of computational or data manipulation steps, or a workflow, in the grid context.
==Comparison of grids and conventional supercomputers==
“Distributed” or “grid” computing in general is a special type of [[parallel computing]] that relies on complete computers (with onboard CPUs, storage, power supplies, network interfaces, etc.) connected to a [[computer network|network]] (private, public or the [[Internet]]) by a conventional [[Network interface controller|network interface]] producing commodity hardware, compared to the lower efficiency of designing and constructing a small number of custom supercomputers. The primary performance disadvantage is that the various processors and local storage areas do not have high-speed connections. This arrangement is thus well-suited to applications in which multiple parallel computations can take place independently, without the need to communicate intermediate results between processors.<ref>[http://www.e-sciencecity.org/EN/gridcafe/computational-problems.html Computational problems - Gridcafe] {{Webarchive|url=https://web.archive.org/web/20120825003633/http://www.e-sciencecity.org/EN/gridcafe/computational-problems.html |date=2012-08-25 }}. E-sciencecity.org. Retrieved 2013-09-18.</ref> The high-end [[scalability]] of geographically dispersed grids is generally favorable, due to the low need for connectivity between [[Node (computer science)|nodes]] relative to the capacity of the public Internet.<ref>{{Cite web |title=What is grid computing? |url=https://www.ionos.com/digitalguide/server/know-how/grid-computing/ |access-date=2022-03-23 |website=IONOS Digitalguide |language=en |archive-date=2022-01-28 |archive-url=https://web.archive.org/web/20220128110848/https://www.ionos.com/digitalguide/server/know-how/grid-computing/ |url-status=live }}</ref>
There are also some differences in programming and MC.{{clarify|date=July 2019}} It can be costly and difficult to write programs that can run in the environment of a supercomputer, which may have a custom operating system, or require the program to address [[Concurrency (computer science)|concurrency]] issues. If a problem can be adequately parallelized, a “thin” layer of “grid” infrastructure can allow conventional, standalone programs, given a different part of the same problem, to run on multiple machines. This makes it possible to write and debug on a single conventional machine and eliminates complications due to multiple instances of the same program running in the same shared [[computer memory|memory]] and storage space at the same time.
Line 22:
One disadvantage of this feature is that the computers which are actually performing the calculations might not be entirely trustworthy. The designers of the system must thus introduce measures to prevent malfunctions or malicious participants from producing false, misleading, or erroneous results, and from using the system as an attack vector. This often involves assigning work randomly to different nodes (presumably with different owners) and checking that at least two different nodes report the same answer for a given work unit. Discrepancies would identify malfunctioning and malicious nodes. However, due to the lack of central control over the hardware, there is no way to guarantee that [[Node (computer science)|nodes]] will not drop out of the network at random times. Some nodes (like laptops or [[dial-up]] Internet customers) may also be available for computation but not network communications for unpredictable periods. These variations can be accommodated by assigning large work units (thus reducing the need for continuous network connectivity) and reassigning work units when a given node fails to report its results in expected time.
Another set of what could be termed social compatibility issues in the early days of grid computing related to the goals of grid developers to carry their innovation beyond the original field of high-performance computing and across disciplinary boundaries into new fields, like that of high-energy physics.<ref>{{Cite journal|last1=Kertcher|first1=Zack|last2=Coslor|first2=Erica|date=2018-07-10|title=Boundary Objects and the Technical Culture Divide: Successful Practices for Voluntary Innovation Teams Crossing Scientific and Professional Fields|journal=Journal of Management Inquiry|volume=29|language=en|pages=76–91|doi=10.1177/1056492618783875|issn=1056-4926|hdl=11343/212143|s2cid=149911242|url=http://minerva-access.unimelb.edu.au/bitstream/11343/212143/5/Kertcher%20%26%20Coslor%20-%20Boundary%20Objects%20and%20the%20Technical%20Culture%20Divide%202018-02-13.pdf|doi-access=free|access-date=2019-09-18|archive-date=2022-03-28|archive-url=https://web.archive.org/web/20220328181127/https://minerva-access.unimelb.edu.au/bitstream/11343/212143/5/Kertcher%20%26%20Coslor%20-%20Boundary%20Objects%20and%20the%20Technical%20Culture%20Divide%202018-02-13.pdf|url-status=live}}</ref>
The impacts of trust and availability on performance and development difficulty can influence the choice of whether to deploy onto a dedicated cluster, to idle machines internal to the developing organization, or to an open external network of volunteers or contractors. In many cases, the participating nodes must trust the central system not to abuse the access that is being granted, by interfering with the operation of other programs, mangling stored information, transmitting private data, or creating new security holes. Other systems employ measures to reduce the amount of trust “client” nodes must place in the central system such as placing applications in virtual machines.
Line 52:
Many [[volunteer computing]] projects, such as [[BOINC]], use the CPU scavenging model. Since [[Node (computer science)|nodes]] are likely to go "offline" from time to time, as their owners use their resources for their primary purpose, this model must be designed to handle such contingencies.
Creating an '''Opportunistic Environment''' is another implementation of CPU-scavenging where special workload management system harvests the idle desktop computers for compute-intensive jobs, it also refers as Enterprise Desktop Grid (EDG). For instance, [[HTCondor]]<ref>{{cite web|url=https://research.cs.wisc.edu/htcondor/|title=HTCondor - Home|website=research.cs.wisc.edu|access-date=14 March 2018|archive-date=2 March 2018|archive-url=https://web.archive.org/web/20180302205500/http://research.cs.wisc.edu/htcondor/|url-status=live}}</ref> (the open-source high-throughput computing software framework for coarse-grained distributed rationalization of computationally intensive tasks) can be configured to only use desktop machines where the keyboard and mouse are idle to effectively harness wasted CPU power from otherwise idle desktop workstations. Like other full-featured batch systems, HTCondor provides a job queueing mechanism, scheduling policy, priority scheme, resource monitoring, and resource management. It can be used to manage workload on a dedicated cluster of computers as well or it can seamlessly integrate both dedicated resources (rack-mounted clusters) and non-dedicated desktop machines (cycle scavenging) into one computing environment.
==History==
Line 59:
CPU scavenging and [[volunteer computing]] were popularized beginning in 1997 by [[distributed.net]] and later in 1999 by [[SETI@home]] to harness the power of networked PCs worldwide, in order to solve CPU-intensive research problems.<ref name="anderson1">{{cite journal|last1=Anderson|first1=David P|last2=Cobb|display-authors=etal|first2=Jeff|title=SETI@home: an experiment in public-resource computing|journal=Communications of the ACM|date=November 2002|volume=45|issue=11|pages=56–61|doi=10.1145/581571.581573|s2cid=15439521}}</ref><ref name="durrani1">{{cite journal|last1=Nouman Durrani|first1=Muhammad|last2=Shamsi|first2=Jawwad A.|title=Volunteer computing: requirements, challenges, and solutions|journal=Journal of Network and Computer Applications|date=March 2014|volume=39|pages=369–380|doi=10.1016/j.jnca.2013.07.006}}</ref>
The ideas of the grid (including those from distributed computing, object-oriented programming, and Web services) were brought together by [[Ian Foster (computer scientist)|Ian Foster]] and [[Steve Tuecke]] of the [[University of Chicago]], and [[Carl Kesselman]] of the [[University of Southern California]]'s [[Information Sciences Institute]].<ref>{{Cite web |last=Johnson |first=Bridget |date=2019-11-06 |title=Grid Computing Pioneer Steve Tuecke Passes Away at 52 |url=https://www.hstoday.us/subject-matter-areas/cybersecurity/grid-computing-pioneer-steve-tuecke-passes-away-at-52/ |access-date=2022-11-04 |language=en-US |archive-date=2022-11-04 |archive-url=https://web.archive.org/web/20221104213215/https://www.hstoday.us/subject-matter-areas/cybersecurity/grid-computing-pioneer-steve-tuecke-passes-away-at-52/ |url-status=live }}</ref> The trio, who led the effort to create the [[Globus Toolkit]], is widely regarded as the "fathers of the grid".<ref>{{cite web|url=http://magazine.uchicago.edu/0404/features/index.shtml|title=Father of the Grid|access-date=2007-04-15|archive-date=2012-03-01|archive-url=https://web.archive.org/web/20120301194142/http://magazine.uchicago.edu/0404/features/index.shtml|url-status=live}}</ref> The toolkit incorporates not just computation management but also [[Storage Resource Management (SRM)|storage management]], security provisioning, data movement, monitoring, and a toolkit for developing additional services based on the same infrastructure, including agreement negotiation, notification mechanisms, trigger services, and information aggregation.<ref>{{Cite book |last=Salem |first=M. |url=https://www.researchgate.net/publication/258119520 |title=Grid Computing: A New Paradigm for Healthcare Technologies/Applications |year=2007 |access-date=2022-08-30}}</ref> While the Globus Toolkit remains the de facto standard for building grid solutions, a number of other tools have been built that answer some subset of services needed to create an enterprise or global grid.
In 2007 the term [[cloud computing]] came into popularity, which is conceptually similar to the canonical Foster definition of grid computing (in terms of computing resources being consumed as electricity is from the [[power grid]]) and earlier utility computing.
Line 67:
==Fastest virtual supercomputers==
* As of April 7, 2020, [[BOINC]] – 29.8 PFLOPS.<ref name="BoincStats">{{cite web|url=http://boincstats.com/en/stats/-1/project/detail|title=BOINCstats – BOINC combined credit overview|access-date=October 30, 2016|archive-date=January 22, 2013|archive-url=https://web.archive.org/web/20130122022019/http://boincstats.com/en/stats/-1/project/detail/|url-status=live}}</ref>
* As of March 2020, [[Folding@home]] – 1.1 exaFLOPS.<ref name="FAH osstats2">{{cite web|url=https://stats.foldingathome.org/os|title=Client Statistics by OS|publisher=Stanford University|author=Pande lab|work=Folding@home|access-date=March 26, 2020|archive-date=April 12, 2020|archive-url=https://archive.ph/20200412111010/https://stats.foldingathome.org/os|url-status=live}}</ref>
* As of November 2019, IceCube via OSG – 350 fp32 PFLOPS.<ref>{{cite web|url=https://www.sdsc.edu/News%20Items/PR20191119_GPU_Cloudburst.html|title=SDSC, Wisconsin U IceCube Center Conduct GPU Cloudburst Experiment|publisher=SDSC|access-date=April 22, 2022|archive-date=September 14, 2022|archive-url=https://web.archive.org/web/20220914153408/https://www.sdsc.edu/News%20Items/PR20191119_GPU_Cloudburst.html|url-status=live}}</ref>
* As of February 2018, [[Einstein@Home]] – 3.489 PFLOPS.<ref>{{cite web|url=http://boincstats.com/en/stats/5/project/detail|title=Einstein@Home Credit overview|publisher=BOINC|access-date=October 30, 2016|archive-date=August 27, 2016|archive-url=https://web.archive.org/web/20160827063611/http://boincstats.com/en/stats/5/project/detail|url-status=live}}</ref>
* As of April 7, 2020, [[SETI@Home]] – 1.11 PFLOPS.<ref>{{cite web|url=http://boincstats.com/en/stats/0/project/detail|title=SETI@Home Credit overview|publisher=BOINC|access-date=October 30, 2016|archive-date=July 3, 2013|archive-url=https://web.archive.org/web/20130703143037/http://boincstats.com/en/stats/0/project/detail|url-status=live}}</ref>
* As of April 7, 2020, [[MilkyWay@Home]] – 1.465 PFLOPS.<ref>{{cite web|url=http://boincstats.com/en/stats/61/project/detail|title=MilkyWay@Home Credit overview|publisher=BOINC|access-date=October 30, 2016|archive-date=May 20, 2012|archive-url=https://web.archive.org/web/20120520164005/http://boincstats.com/en/stats/61/project/detail|url-status=live}}</ref>
* As of March 2019, [[Great Internet Mersenne Prime Search|GIMPS]] – 0.558 PFLOPS.<ref>{{cite web|url=http://www.mersenne.org/primenet|title=Internet PrimeNet Server Distributed Computing Technology for the Great Internet Mersenne Prime Search|work=GIMPS|access-date=March 12, 2019|archive-date=May 25, 2019|archive-url=https://web.archive.org/web/20190525223313/https://www.mersenne.org/primenet/|url-status=live}}</ref>
Also, as of March 2019, the [[Bitcoin network|Bitcoin Network]] had a measured computing power equivalent to over 80,000 [[FLOPS|exaFLOPS]] (Floating-point Operations Per Second).<ref name="Bitcoin Network Statistics">{{cite web|url=http://bitcoinwatch.com/bitcoin/|title=Bitcoin Network Statistics|author=bitcoinwatch.com|work=Bitcoin|access-date=March 12, 2019|archive-date=January 20, 2023|archive-url=https://web.archive.org/web/20230120182533/https://bitcoincharts.com/|url-status=live}}</ref> This measurement reflects the number of FLOPS required to equal the hash output of the Bitcoin network rather than its capacity for general floating-point arithmetic operations, since the elements of the Bitcoin network (Bitcoin mining [[ASIC]]s) perform only the specific cryptographic hash computation required by the [[Bitcoin]] protocol.
==Projects and applications==
Line 84:
The [[European Union]] funded projects through the [[framework programme]]s of the [[European Commission]]. [[BEinGRID]] (Business Experiments in Grid) was a research project funded by the European Commission<ref>{{cite web|url=http://www.beingrid.eu/|title=beingrid.eu: Stromkosten Vergleiche -|website=beingrid.eu: Stromkosten Vergleiche|access-date=14 March 2018|archive-url=https://web.archive.org/web/20110723100417/http://www.beingrid.eu/|archive-date=23 July 2011|url-status=dead}}</ref> as an [[Integrated Project (EU)|Integrated Project]] under the [[Sixth Framework Programme]] (FP6) sponsorship program. Started on June 1, 2006, the project ran 42 months, until November 2009. The project was coordinated by [[Atos Origin]]. According to the project fact sheet, their mission is “to establish effective routes to foster the adoption of grid computing across the EU and to stimulate research into innovative business models using Grid technologies”. To extract best practice and common themes from the experimental implementations, two groups of consultants are analyzing a series of pilots, one technical, one business. The project is significant not only for its long duration but also for its budget, which at 24.8 million Euros, is the largest of any FP6 integrated project. Of this, 15.7 million is provided by the European Commission and the remainder by its 98 contributing partner companies. Since the end of the project, the results of BEinGRID have been taken up and carried forward by [[IT-Tude.com]].
The Enabling Grids for E-sciencE project, based in the [[European Union]] and included sites in Asia and the United States, was a follow-up project to the European DataGrid (EDG) and evolved into the [[European Grid Infrastructure]]. This, along with the [[LHC Computing Grid]]<ref>{{cite web|url=http://wlcg.web.cern.ch/|title=Welcome to the Worldwide LHC Computing Grid - WLCG|website=wlcg.web.cern.ch|access-date=14 March 2018|archive-date=25 July 2018|archive-url=https://web.archive.org/web/20180725112849/http://wlcg.web.cern.ch/|url-status=live}}</ref> (LCG), was developed to support experiments using the [[CERN]] [[Large Hadron Collider]]. A list of active sites participating within LCG can be found online<ref>{{cite web |url=http://goc.grid.sinica.edu.tw/gstat/ |title=GStat 2.0 – Summary View – GRID EGEE |publisher=Goc.grid.sinica.edu.tw |access-date=July 29, 2010 |archive-url=https://web.archive.org/web/20080320145926/http://goc.grid.sinica.edu.tw/gstat/ |archive-date=March 20, 2008 |url-status=dead }}</ref> as can real time monitoring of the EGEE infrastructure.<ref>{{cite web |url=http://gridportal.hep.ph.ic.ac.uk/rtm/ |title=Real Time Monitor |publisher=Gridportal.hep.ph.ic.ac.uk |access-date=July 29, 2010 |url-status=dead |archive-url=https://web.archive.org/web/20091216124323/http://gridportal.hep.ph.ic.ac.uk/rtm/ |archive-date=December 16, 2009 }}</ref> The relevant software and documentation is also publicly accessible.<ref>{{cite web |url=http://lcg.web.cern.ch/LCG/activities/deployment.html |title=LCG – Deployment |publisher=Lcg.web.cern.ch |access-date=July 29, 2010 |archive-url=https://web.archive.org/web/20101117002656/http://lcg.web.cern.ch/LCG/activities/deployment.html |archive-date=November 17, 2010 |url-status=dead }}</ref> There is speculation that dedicated fiber optic links, such as those installed by CERN to address the LCG's data-intensive needs, may one day be available to home users thereby providing internet services at speeds up to 10,000 times faster than a traditional broadband connection.<ref>{{cite web|url=https://www.thetimes.co.uk/|title=The Times & The Sunday Times|website=thetimes.co.uk|access-date=14 March 2018|archive-date=25 February 2021|archive-url=https://web.archive.org/web/20210225020225/https://www.thetimes.co.uk/|url-status=live}}</ref> The [[European Grid Infrastructure]] has been also used for other research activities and experiments such as the simulation of oncological clinical trials.<ref>{{cite journal | author=Athanaileas, Theodoros| title=Exploiting grid technologies for the simulation of clinical trials: the paradigm of in silico radiation oncology | journal=SIMULATION: Transactions of the Society for Modeling and Simulation International | volume=87 | number=10 | pages=893–910 | year=2011 | doi=10.1177/0037549710375437| s2cid=206429690 |display-authors=etal}}</ref>
The [[distributed.net]] project was started in 1997.
Line 99:
* Plaszczak/Wellner<ref>P Plaszczak, R Wellner, ''Grid computing'', 2005, Elsevier/Morgan Kaufmann, San Francisco</ref> define grid technology as "the technology that enables resource virtualization, on-demand provisioning, and service (resource) sharing between organizations."
* IBM defines grid computing as “the ability, using a set of open standards and protocols, to gain access to applications and data, processing power, storage capacity and a vast array of other computing resources over the Internet. A grid is a type of parallel and distributed system that enables the sharing, selection, and aggregation of resources distributed across ‘multiple’ administrative domains based on their (resources) availability, capacity, performance, cost and users' quality-of-service requirements”.<ref>IBM Solutions Grid for Business Partners: Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand</ref>
* An earlier example of the notion of computing as the utility was in 1965 by MIT's Fernando Corbató. Corbató and the other designers of the Multics operating system envisioned a computer facility operating “like a power company or water company”.<ref>[http://www.multicians.org/fjcc3.html Structure of the Multics Supervisor] {{Webarchive|url=https://web.archive.org/web/20140116070940/http://www.multicians.org/fjcc3.html |date=2014-01-16 }}. Multicians.org. Retrieved 2013-09-18.</ref>
* Buyya/Venugopal<ref>{{cite web|url=http://www.buyya.com/papers/GridIntro-CSI2005.pdf|title=A Gentle Introduction to Grid Computing and Technologies|access-date=May 6, 2005|archive-date=March 24, 2006|archive-url=https://web.archive.org/web/20060324161402/http://www.buyya.com/papers/GridIntro-CSI2005.pdf|url-status=live}}</ref> define grid as "a type of parallel and distributed system that enables the sharing, selection, and aggregation of geographically distributed [[Wiktionary:autonomy|autonomous]] resources dynamically at runtime depending on their availability, capability, performance, cost, and users' quality-of-service requirements".
*[[CERN]], one of the largest users of grid technology, talk of '''The Grid''': “a service for sharing computer power and data storage capacity over the [[Internet]].”<ref>{{cite web|url=http://www.gridcafe.org|title=The Grid Café – The place for everybody to learn about grid computing|publisher=[[CERN]]|access-date=December 3, 2008|archive-url=https://web.archive.org/web/20081205082353/http://www.gridcafe.org/|archive-date=December 5, 2008|url-status=dead}}</ref>
| |||