A parallel programming model is a set of software technologies to express parallel algorithms and match applications with the underlying parallel systems. It encloses the areas of applications, programming languages, compilers, libraries, communications systems, and parallel I/O. Due to the difficulties in automatic parallelization today, people have to choose a proper parallel programming model or a form of mixture of them to develop their parallel applications on a particular platform.
Parallel models are implemented in several ways: as libraries invoked from traditional sequential languages, as language extensions, or complete new execution models. They are also roughly categoried for two kinds of systems: shared-memory system and distributed-memory system, though the lines between them are largely blurred nowadays.
A programming model is usually judged by its expressibility and simplicity, which are by all meanings conflicting factors. The ultimate goal is to improve productivity of programming.
Example parallel programming models
As of 2006, the current main-stream parallel programming models are:
- POSIX Threads
- PVM
- MPI
- OpenMP
- Global Arrays
- Co-array_Fortran
- UPC
- HPF
- SHMEM
- Stream processing
- Pipelining
- Partitioned global address space
Other research-level models are:
- Cray's Chapel,
- Sun’s Fortress,
- IBM’s X10
References
- H. Shan and J. Pal Singh. A comparison of MPI, SHMEM, and Cache-Coherent Shared Address Space Programming Models on a Tightly-Coupled Multiprocessor. International Journal of Parallel Programming, 29(3), 2001.
- H. Shan and J. Pal Singh. Comparison of Three Programming Models for Adaptive Applications on the Origin 2000. Journal of Parallel and Distributed Computing, 62:241–266, 2002.
- About structured parallel programming: Davide Pasetto and Marco Vanneschi. Machine independent Analytical models for cost evaluation of template--based programs, University of Pisa, 1996