ECL (data-centric programming language): Difference between revisions

Revision as of 14:26, 4 December 2019

Revision as of 00:36, 29 March 2020

Line 17:

}}

'''ECL''' is a declarative, data -centric programming language designed in 2000 to allow a team of programmers to process [[big data]] across a high performance computing cluster without the programmer being involved in many of the lower level, imperative decisions.<ref>[http://www.lexisnexis.com/risk/about/guides/program-guide.html A Guide to ECL, [[Lexis-Nexis]].]</ref><ref>"Evaluating use of data flow systems for large graph analysis," by A. Yoo, and I. Kaplan. Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS, 2009</ref>

== History ==

Line 23:

== Language constructs ==

ECL, at least in its purest form, is a declarative, data -centric language. Programs, in the strictest sense, do not exist. Rather an ECL application will specify a number of core datasets (or data values) and then the operations which are to be performed on those values.

=== Hello world ===

Line 56:

=== ECL primitives ===

ECL primitives that act upon datasets include: SORT, ROLLUP, DEDUP, ITERATE, PROJECT, JOIN, NORMALIZE, DENORMALIZE, PARSE, CHOSEN, ENTH, TOPN, DISTRIBUTE

=== ECL encapsulation ===

Whilst ECL is terse and LexisNexis claims that 1 line of ECL is roughly equivalent to 120 lines of C++, it still has significant support for large scale programming including data encapsulation and code re-use. The constructs available include: MODULE, FUNCTION, FUNCTIONMACRO, INTERFACE, MACRO, EXPORT, SHARED

=== Support for Parallelism in ECL ===

Line 65:

=== Comparison to Map-Reduce ===

The Hadoop Map-Reduce paradigm actually consists of three phases which correlate to ECL primitives as follows.

{| class="wikitable"

|-

Line 78:

| SHUFFLE (Phase 1)

| DISTRIBUTE(,HASH(KeyValue))

| The records from the mapper are distributed dependentdepending upon the KEY value

|-

| SHUFFLE (Phase 2)

Line 86:

| REDUCE

| ROLLUP(,Key,LOCAL)

| The records for a particular KEY value are now combined together

|}