Lambda architecture: Difference between revisions

Revision as of 23:55, 26 August 2014

Revision as of 00:09, 27 August 2014

Line 30:

==Optimizations==

To optimize the data set and improve query efficiency, various rollup and aggregation techniques are executed on raw data,<ref name=metamarkets-lambda />{{rp|23}} while estimation techniques are employed to further reduce computation costs.<ref>Ray, Nelson. [https://metamarkets.com/2013/histograms/ "The Art of Approximating Distributions: Histograms and Quantiles at Scale"]. 12 September 2013. Metamarkets.</ref> And while expensive full recomputation is required for fault tolerance, incremental computation algorithms may be selectively added to increase efficiency, and techniques such as ''partial computation"'' and resource-usage optimizations can effectively help lower latency.<ref name=big-data />{{rp|93,287,293}}

==Lambda Architecture in Use==