Programming with Big Data in R: Difference between revisions

Revision as of 16:31, 3 December 2022

Latest revision as of 16:31, 28 February 2024

(3 intermediate revisions by 3 users not shown)

Line 48:

* pbdDMAT --- distributed matrix classes and computational methods, with a focus on linear algebra and statistics

* pbdDEMO --- set of package demonstrations and examples, and this unifying vignette

* pmclust --- parallel [[Mixture model|model-based clustering]] using pbdR

* pbdPROF --- profiling package for MPI codes and visualization of parsed stats

* pbdZMQ --- interface to [[ZeroMQ|ØMQ]]

Line 63:

=== Example 1 ===

Hello World! Save the following code in a file called "demo.r"

<syntaxhighlight lang="rsplusr">

### Initial MPI

library(pbdMPI, quiet = TRUE)

Line 82:

The following example modified from pbdMPI illustrates the basic [[programming language syntax|syntax of the language]] of pbdR.

Since pbdR is designed in [[SPMD]], all the R scripts are stored in files and executed from the command line via mpiexec, mpirun, etc. Save the following code in a file called "demo.r"

<syntaxhighlight lang="rsplusr">

### Initial MPI

library(pbdMPI, quiet = TRUE)

Line 111:

The following example modified from pbdDEMO illustrates the basic ddmatrix computation of pbdR which performs [[singular value decomposition]] on a given matrix.

Save the following code in a file called "demo.r"

<syntaxhighlight lang="rsplusr">

# Initialize process grid

library(pbdDMAT, quiet=T)

Line 138:

== Further reading ==

* {{cite techreporttech report|author=Raim, A.M.|year=2013|title=Introduction to distributed computing with pbdR at the UMBC High Performance Computing Facility|institution=UMBC High Performance Computing Facility, University of Maryland, Baltimore County|number=HPCF-2013-2|url=http://userpages.umbc.edu/~gobbert/papers/pbdRtara2013.pdf|accessdate=2013-06-26|archiveurl=https://web.archive.org/web/20140204051402/http://userpages.umbc.edu/~gobbert/papers/pbdRtara2013.pdf|archivedate=2014-02-04|url-status=dead}}

* {{cite techreporttech report|author=Bachmann, M.G., Dyas, A.D., Kilmer, S.C. and Sass, J.|year=2013|title=Block Cyclic Distribution of Data in pbdR and its Effects on Computational Efficiency|institution=UMBC High Performance Computing Facility, University of Maryland, Baltimore County|number=HPCF-2013-11|url=http://userpages.umbc.edu/~gobbert/papers/REU2013Team1.pdf|accessdate=2014-02-01|archiveurl=https://web.archive.org/web/20140204051351/http://userpages.umbc.edu/~gobbert/papers/REU2013Team1.pdf|archivedate=2014-02-04|url-status=dead}}

* {{cite techreporttech report|author=Bailey, W.J., Chambless, C.A., Cho, B.M. and Smith, J.D.|year=2013|title=Identifying Nonlinear Correlations in High Dimensional Data with Application to Protein Molecular Dynamics Simulations|institution=UMBC High Performance Computing Facility, University of Maryland, Baltimore County|number=HPCF-2013-12|url=http://userpages.umbc.edu/~gobbert/papers/REU2013Team2.pdf|accessdate=2014-02-01|archiveurl=https://web.archive.org/web/20140204055902/http://userpages.umbc.edu/~gobbert/papers/REU2013Team2.pdf|archivedate=2014-02-04|url-status=dead}}

* {{cite web|title=High-Performance and Parallel Computing with R|author=Dirk Eddelbuettel|date=13 November 2022 |url=https://cran.r-project.org/web/views/HighPerformanceComputing.html|author-link=Dirk Eddelbuettel}}

* {{cite news|title=R at 12,000 Cores|url=http://www.r-bloggers.com/r-at-12000-cores/}}<br />This article was read 22,584 times in 2012 since it posted on October 16, 2012, and ranked number 3<ref>{{cite news|url=http://www.r-bloggers.com/100-most-read-r-posts-for-2012-stats-from-r-bloggers-big-data-visualization-data-manipulation-and-other-languages/|title=100 most read R posts in 2012 (stats from R-bloggers) – big data, visualization, data manipulation, and other languages}}</ref>

* {{cite web|url=http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2013:mpiprofiler|archive-url=https://archive.today/20130629095333/http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2013:mpiprofiler|url-status=dead|archive-date=2013-06-29|title=Profiling Tools for Parallel Computing with R|author=Google Summer of Code - R 2013}}

* {{cite web|url=http://rpubs.com/wush978/pbdMPI-linux-pilot|title=在雲端運算環境使用R和MPI|author=Wush Wu (2014)}}