Content deleted Content added
Textractor (talk | contribs) mNo edit summary |
Textractor (talk | contribs) added diagrams from wikimedia |
||
Line 3:
'''Lambda Architecture'''
[[File:Diagram of Lambda Architecture (generic).png|thumb|Diagram showing the flow of data through the processing and serving layers of lambda architecture.]]
Lambda architecture refers to a [[data processing|data-processing]] architecture designed to handle massive quantities of data by taking advantage of both batch- and stream-processing methods. This approach to architecture attempts to balance latency, throughput, and fault-tolerance by using batch processing to provide comprehensive and accurate precomputed views, while simultaneously using real-time stream processing to provide dynamic views. The two view outputs may be joined before before presentation. The rise of lambda architecture is correlated with the growth of [[big data]], real-time analytics, and the drive to mitigate the latencies of [[map-reduce]].<ref>{{cite web|last1=Schuster|first1=Werner|title=Nathan Marz on Storm, Immutability in the Lambda Architecture, Clojure|url=http://www.infoq.com/interviews/marz-lambda-architecture|website=www.infoq.com}} Interview with Nathan Marz, 6 April 2014</ref>
Line 16 ⟶ 17:
===Speed Layer===
[[File:Diagram of Lambda Architecture (named components).png|thumb|Diagram showing the flow of data through the processing and serving layers of lambda architecture. Example named components are shown.]]
The speed layer processes data streams in real time and without the requirements of fix-ups or completeness. This layer sacrifices throughput as it aims to minimize latency by providing real-time views into the most recent data. Essentially, the speed layer is responsible for filling the "gap" caused by the batch layer's lag in providing views based on the most recent data. This layer's views may not be as accurate or complete as the ones eventually produced by the batch layer, but they are available almost immediately after data is received, and can be replaced when the batch layer's views for the same data become available.<ref name=big-data>Marz, Warren, p. 203</ref>
Line 21 ⟶ 23:
===Serving Layer===
[[File:Diagram of Lambda Architecture (Druid data store).png|thumb|Diagram showing the flow of data through the processing and serving layers of lambda architecture. Example named components are shown.]]
Output from the batch and speed layers are stored in the serving layer, which responds to ad-hoc queries by returning precomputed views or building views from the processed data.
| |||