Programming with Big Data in R: Difference between revisions

Revision as of 21:41, 23 September 2013

Revision as of 14:55, 2 October 2013

Line 40:

|}

[[File:Pbd overview.png|thumb|The images describes how various pbdr packages are correlated.]]

Among these packages, pbdMPI provides wrapper functions to [[Message Passing Interface|MPI]] library, and it also produces a [[Library (computing)|shared library]] and a configuration file for [[MPI]] environments. All other packages rely on this configuration for installation and library loading that avoid difficulty of library linking and compiling. All other packages can directly utilize [[MPI]] functions easily.

* pbdMPI --- an efficient interface to [[MPI]] either [[Open MPI|OpenMPI]] or [[MPICH2]] with a focus on Single Program/Multiple Data ([[SPMD]]) parallel programming style

* pbdSLAP --- bundles scalable dense linear algebra libraries in double precision for R, based on [[ScaLAPACK]] version 2.0.2 which includes several scalable linear algebra packages (namely [[BLACS]], [[PBLAS]], and [[ScaLAPACK]]).

* pbdNCDF4 --- Interface to Parallel Unidata [[NetCDF|NetCDF4]]4 format data files

* pbdBASE --- low-level [[ScaLAPACK]] codes and wrappers

* pbdDMAT --- distributed matrix classes and computational methods, with a focus on linear algebra and statistics

Line 71:

mpiexec -np 2 Rscript demo.r

</source>

to execute the code where [[R|Rscript]]script is one of command line executable program.

=== Example 2 ===

The following example modified from pbdMPI illustrates the basic [[programming language syntax|syntax of the language]] of pbdR.

Since pbdR is designed in [[SPMD]], all the R scripts are stored in files and executed from the command line via [[MPI|mpiexec]], [[MPI|mpirun]], etc. Save the following code in a file called ``demo.r``

<source lang="rsplus">

### Initial MPI

Line 100:

mpiexec -np 4 Rscript demo.r

</source>

to execute the code where [[R|Rscript]]script is one of command line executable program.

=== Example 3 ===

Line 129:

mpiexec -np 2 Rscript demo.r

</source>

to execute the code where [[R|Rscript]]script is one of command line executable program.

== Further reading ==