Data-intensive computing: Difference between revisions

Revision as of 14:58, 4 April 2012

Revision as of 18:24, 19 April 2012

Line 53:

===HPCC===

[[LexisNexis|LexisNexis Risk Solutions]], independently developed and implemented a solution for data-intensive computing called the [[HPCC]] (High-Performance Computing Cluster). The development of this computing platform began in 1999 and applications were in production by late 2000. The [[LexisNexis]] approach also utilizes commodity clusters of hardware running the [[Linux]] operating system as shown in Figure 1. Custom system software and middleware components were developed and layered on the base Linux operating system to provide the execution environment and distributed filesystem support required for data-intensive computing. LexisNexis also implemented a new high-level language for data-intensive computing called ECL.

The [[ECL, data-centric programming language for Big Data|ECL programming language]] is the primary distinguishing factor between HPCC and other data-intensive computing solutions. It is a high-level, declarative, data-centric, [[Implicit parallelism|implicitly parallel]] language that allows the programmer to define what the data processing result should be and the dataflows and transformations that are necessary to achieve the result. The ECL language includes extensive capabilities for data definition, filtering, data management, and data transformation, and provides an extensive set of built-in functions to operate on records in datasets which can include user-defined transformation functions. [[ECL, data-centric programming language for Big Data|ECL]] programs are compiled into optimized [[C++]] source code, which is subsequently compiled into executable code and distributed to the nodes of a processing cluster.