NAS Parallel Benchmarks: Difference between revisions

Revision as of 23:23, 21 March 2009

Latest revision as of 01:16, 8 July 2025

(29 intermediate revisions by 23 users not shown)

Line 1:

{{Infobox Softwaresoftware

|name=NAS Parallel Benchmarks

|author=NASA Numerical Aerodynamic Simulation Program

|developer=[[NASA Advanced Supercomputing Division]]

|releasereleased={{Start date|1991}}

|latest release version=3.34

|website=http://www.{{URL|nas.nasa.gov/Software/NPB/}}

|platform=[[Cross-platform]]

|status=Active

|website=http://www.nas.nasa.gov/Software/NPB/

}}

The '''NAS Parallel Benchmarks''' ('''NPB''') are a set of [[benchmark (computing)|benchmark]]s targeting performance evaluation of highly [[Parallel computing|parallel]] [[supercomputer]]s. They are developed and maintained by the [[NASA]] [[NASA Advanced Supercomputing Division|Advanced Supercomputing (NAS) Division]] (formerly the NASA Numerical Aerodynamic Simulation Program) based at the [[NASA Ames Research Center]]. NAS solicits performance results for NPB from all sources.<ref name="npbweb">{{cite web

|title=NAS Parallel Benchmarks Changes

|url=http://www.nas.nasa.gov/Resources/Software/npb.html

|publisher=NASA Advanced Supercomputing Division

|accessdate=2009-02-23

}}</ref>

The '''NAS Parallel Benchmarks''' ('''NPB''') are a set of [[benchmark (computing)|benchmark]]s targeting performance evaluation of highly [[Parallel computing|parallel]] [[supercomputer]]s. They are developed and maintained by the [[NASA]] [[NASA Advanced Supercomputing Division|Advanced Supercomputing (NAS) Division]] (formerly the NASA Numerical Aerodynamic Simulation Program) based at the [[NASA Ames Research Center]]. NAS solicits performance results for NPB from all sources.<ref name="npbweb">{{citeCite web |url=http://www.nas.nasa.gov/Resources/Software/npb.html |title=NAS Parallel Benchmarks Changes |publisher=NASA Advanced Supercomputing Division |access-date=2009-02-23}}</ref>

==History==

==History==

===Motivation===

Traditional benchmarks that existed before NPB, such as the [[Livermore loops]], the [[LINPACK|LINPACK Benchmark]] and the [http://www.netlib.org/benchmark/nas NAS Kernel Benchmark Program], were usually specialized for vector computers. They generally suffered from inadequacies including parallelism-impeding tuning restrictions and insufficient problem sizes, which rendered them inappropriate for highly parallel systems. Equally unsuitable were full-scale application benchmarks due to high porting cost and unavailability of automatic software parallelization tools.<ref name="rnr94007">{{Citation

|last1=BailyBailey|first1=D.

|last2=BarsczBarszcz|first2=E.

|last3=Barton|first3=J.

|last4=Browning|first4=D.

Line 28 ⟶ 22:

|last8=Fineberg|first8=S.

|last9=Frederickson|first9=P.

|last10=Lasinski|first10=T.

|last11=Schreiber|first11=R.

|last12=Simon|first12=H.

|last13=Venkatakrishnan|first13=V.

|last14=Weeratunga|first14=S.

|contribution=The NAS Parallel Benchmarks

|contribution-url=httphttps://www.nerscnas.nasa.gov/~simonassets/nas/pdf/Paperstechreports/NASA1994/RNRrnr-94-007.pdf

|title=NAS Technical Report RNR-94-007

|publisher=NASA Ames Research Center, Moffett Field, CA

|yeardate=March 1994

|last10=LasinskiWeeratunga|first10=TS.

|postscript=<!--none-->

}}</ref> As a result, NPB were developed in 1991<ref name="nas95020">{{Citation

|last1=Bailey|first1=D.

Line 46 ⟶ 37:

|last6=Yarrow|first6=M.

|contribution=The NAS Parallel Benchmarks 2.0

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/1995pdf/PDFtechreports/1995/nas-95-020.pdf

|title=NAS Technical Report NAS-95-020

|publisher=NASA Ames Research Center, Moffett Field, CA

|yeardate=December 1995

|postscript=<!--none-->

}}</ref> and released in 1992<ref name="nas03002">{{Citation

|last1=Wong|first1=P.

|last2=van der Wijngaart|first2=R.

|contribution=NAS Parallel Benchmarks I/O Version 2.4

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/2003pdf/PDFtechreports/2003/nas-03-002.pdf

|title=NAS Technical Report NAS-03-002

|publisher=NASA Ames Research Center, Moffett Field, CA

|yeardate=January 2003

|postscript=<!--none-->

}}</ref> to address the ensuing lack of benchmarks applicable to highly parallel machines.

Line 69 ⟶ 62:

In the light of these guidelines, it was deemed the only viable approach to use a collection of "paper-and-pencil" benchmarks that specified a set of problems only algorithmically and left most implementation details to the implementer's discretion under certain necessary limits.

NPB 1 defined eight benchmarks, each in two problem sizes dubbed ''Class A'' and ''Class B''. Sample codes written in [[Fortran#FORTRAN_77FORTRAN 77|Fortran 77]] were supplied. They used a small problem size ''Class S'' and were not intended for benchmarking purposes.<ref name="rnr94007"/>

===NPB 2===

Since its release, NPB 1 displayed two major weaknesses. Firstly, due to its "paper-and-pencil" specification, computer vendors usually highly tuned their implementations so that their performance became difficult for scientific programmers to attain. Secondly, many of these implementation were proprietary and not publicly available, effectively concealing their optimizing techniques. Secondly, problem sizes of NPB 1 lagged behind the development of supercomputers as the latter continued to evolve.<ref name="nas95020"/>

NPB 2, released in 1996,<ref name="npb2.2">{{Citation

|last1=Saphir|first1=W.

|last2=van der Wijngaart|first2=R.

|last3=Woo|first3=A.

|last4=Yarrow|first4=M.

|contributiontitle=New Implementations and Results for the NAS Parallel Benchmarks 2

|contribution-url=httphttps://www.nas.nasa.gov/Resourcesassets/Softwarenas/pdf/techreports/1994/npb_2.2.pdf

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref><ref name="nas02007">{{Citation

|last1=van der Wijngaart|first1=R.

|contribution=NAS Parallel Benchmarks Version 2.4

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/2002pdf/PDFtechreports/2002/nas-02-007.pdf

|title=NAS Technical Report NAS-02-007

|yeardate=October 2002

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref>, came with source code implementations for five out of eight benchmarks defined in NPB 1 to supplement but not replace NPB 1. It extended the benchmarks with an up-to-date problem size ''Class C''. It also amended the rules for submitting benchmarking results. The new rules included explicit requests for output files as well as modified source files and build scripts to ensure public availability of the modifications and reproducibility of the results.<ref name="nas95020"/>

NPB 2.2 contained implementations of two more benchmarks.<ref name="npb2.2"/> NPB 2.3 of 1997 was the first complete implemetationimplementation in [[Message Passing Interface|MPI]].<ref name="nas03002"/> It shipped with serial versions of the benchmarks consistent with the parallel versions and defined a problem size ''Class W'' for small-memory systems.<ref name="npbchanges">{{cite web

|title=NAS Parallel Benchmarks Changes

|url=http://www.nas.nasa.gov/Resources/Software/npb_changes.html

Line 99 ⟶ 94:

===NPB 3===

NPB 3 retained the MPI implementation from NPB 2 and came in more flavors, namely [[OpenMP]],<ref name="nas99011">{{Citation

|last1=Jin|first1=H.

|last2=Frumkin|first2=M.

|last3=Yan|first3=J.

|contribution=The OpenMP Implementation of NAS Parallel Benchmarks and Its Performance

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/1999pdf/PDFtechreports/1999/nas-99-011.pdf

|title=NAS Technical Report NAS-99-011

|yeardate=October 1999

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref>, [[Java (programming language)|Java]]<ref name="nas02009">{{Citation

|last1=Frumkin|first1=M.

|last2=Schultz|first2=M.

Line 114 ⟶ 110:

|last4=Yan|first4=J.

|contribution=Implementation of the NAS Parallel Benchmarks in Java

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/2002pdf/PDFtechreports/2002/nas-02-009.pdf

|title=NAS Technical Report NAS-02-009

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref> and [[High Performance Fortran]].<ref name="nas98009">{{Citation

|last1=Frumkin|first1=M.

|last2=Jin|first2=H.

|last3=Yan|first3=J.

|contribution=Implementation of NAS Parallel Benchmarks in High Performance Fortran

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/1998pdf/PDFtechreports/1998/nas-98-009.pdf

|title=NAS Technical Report NAS-98-009

|yeardate=September 1998

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref>. These new parallel implementations were derived from derived the serial codes in NPB 2.3 with additional optimizations.<ref name="npbchanges"/> NPB 3.1 and NPB 3.2 added three more benchmarks,<ref name="nas04006">{{Citation

|last1=Feng|first1=H.

|last2=van der Wijngaart|first2=F.

|last3=Biswas|first3=R.

|last4=Mavriplis|first4=C.|author4-link=Catherine Mavriplis

|contribution=Unstructured Adaptive (UA) NAS Parallel Benchmark, Version 1.0

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/2004pdf/PDFtechreports/2004/nas-04-006.pdf

|title=NAS Technical Report NAS-04-006

|yeardate=July 2004

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref><ref name="nas04013">{{Citation

|last1=Frumkin|first1=M.

|last2=Shabanov|first2=L.

|contribution=Benchmarking Memory Performance with the Data Cube Operator

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/2004pdf/PDFtechreports/2004/nas-04-013.pdf

|title=NAS Technical Report NAS-04-013

|yeardate=September 2004

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref>, which, however, were not available across all implementations; NPB 3.3 introduced a ''Class E'' problem size.<ref name="npbchanges"/> Based on the single-zone NPB 3, a set of multi-zone benchmarks taking advantage of the MPI/OpenMP hybrid programming model were released under the name '''NPB-Multi-Zone''' ('''NPB-MZ''') for "testing the effectiveness of multi-level and hybrid parallelization paradigms and tools".<ref name="npbweb"/><ref name="nas03010">{{Citation

|last1=van der Wijngaart|first1=R.

|last2=Jin|first2=H.

|contribution=NAS Parallel Benchmarks, Multi-Zone Versions

|contribution-url=httphttps://www.nas.nasa.gov/Newsassets/Techreportsnas/2003pdf/PDFtechreports/2003/nas-03-010.pdf

|title=NAS Technical Report NAS-03-010

|yeardate=July 2003

|publisher=NASA Ames Research Center, Moffett Field, CA

|postscript=<!--none-->

}}</ref>

Line 158 ⟶ 159:

{| class="wikitable"

! Benchmark !! Name derived from<ref name="rnr94007"/> !! Available since !! Description<ref name="rnr94007"/> !! Available since !! RemarkRemarks

|-

| MG || '''M'''ulti'''G'''rid

| rowspan="8" | NPB 1<ref name="rnr94007"/> ||

| Approximate the solution to a three-dimensional [[discrete Poisson equation]] using the V-cycle [[multigrid method]] ||

| rowspan="8" | NPB 1<ref name="rnr94007"/> ||

|-

| CG || '''C'''onjugate '''G'''radient

| Estimate the largestsmallest [[eigenvalue]] of a large [[Sparse matrix|sparse]] [[Symmetric matrix|symmetric]] [[positive-definite matrix]] using the [[inverse iteration]] with the [[conjugate gradient method]] as a subroutine for solving [[System of linear equations|systems of linear equations]] ||

|-

| FT || Fast '''F'''ourier '''T'''ransform

| Solve a three-dimensional [[partial differential equation]] (PDE) using the [[fast Fourier transform]] (FFT) ||

|-

| IS || [[Integer sorting|'''I'''nteger '''S'''ort]]

| Sort small integers in parallel using the [[bucket sort]]<ref name="npb2.2"/> ||

|-

| EP || [[Embarrassingly parallel|'''E'''mbarrassingly '''P'''arallel]]

Line 177 ⟶ 178:

|-

| BT || '''B'''lock '''T'''ridiagonal

| rowspan="3" | Solve a synthetic system of [[Nonlinear system#Partial_differential_equationsPartial differential equations|nonlinear PDEs]] using three different algorithms involving [[Block matrix#Block_tridiagonal_matricesBlock tridiagonal matrices|block tridiagonal]], scalar [[Pentadiagonal matrix|pentadiagonal]] and symmetric [[successive over-relaxation]] (SSOR) solver kernels, respectively

| rowspan="3" |

| Has I/O-intensive subtypes

* The BT benchmark has I/O-intensive subtypes<ref name="nas03002"/>

* All three benchmarks have multi-zone versions<ref name="nas03010"/>

|-

| SP || '''S'''calar '''P'''entadiagonal<ref name="nas02007"/> ||

|-

| LU || [[Triangular matrix#Forward_and_Back_SubstitutionForward and back substitution|'''L'''ower-'''U'''pper]] symmetric [[Gauss–Seidel method|Gauss-Seidel]]<ref name="nas02007"/> ||

|-

| UA || '''U'''nstructured '''A'''daptive<ref name="nas04006"/> ||

| rowspan="2" | NPB 3.1<ref name="npbchanges"/> || Solve [[Heat equation]] with convection and diffusion from moving ball. Mesh is adaptive and recomputed at every 5th step.||

|-

| DC || [[Data cube|'''D'''ata '''C'''ube]] operator<ref name="nas04013"/> || ||

|-

| DT || '''D'''ata '''T'''raffic<ref name="npbchanges"/> || || NPB 3.2<ref name="npbchanges"/> || ||

|}

==References==

{{Reflist}}

<references/>

== External links ==

* [http://www.nas.nasa.gov/Software/NPB/ NAS Parallel Benchmarks Changes] (official website)

[[Category:ComputerSupercomputer benchmarks]]