Apache Pinot: Difference between revisions

Revision as of 20:07, 4 September 2024

Latest revision as of 01:11, 15 August 2025

(One intermediate revision by one other user not shown)

Line 21:

}}

'''Apache Pinot''' is a [[Column-oriented DBMS|column-oriented]], [[open-source software|open-source]], [[Distributed database|distributed]] [[data store]] written in [[Java (programming language)|Java]]. Pinot is designed to execute [[Online analytical processing|OLAP]] queries with low latency.<ref>{{cite book |last1=Cui |first1=Tingting |last2=Peng |first2=Lijun |last3=Pardoe |first3=David |last4=Liu |first4=Kun |last5=Agarwal |first5=Deepak |last6=Kumar |first6=Deepak |title=Proceedings of the ADKDD'17 |chapter=Data-Driven Reserve Prices for Social Advertising Auctions at LinkedIn |date=14 August 2017 |pages=1–7 |doi=10.1145/3124749.3124759 |chapter-url=https://dl.acm.org/doi/abs/10.1145/3124749.3124759 |publisher=Association for Computing Machinery|isbn=9781450351942 |s2cid=12327343 }}</ref><ref>{{cite book |last1=Rosa |first1=Marcello La |title=ADVANCED INFORMATION SYSTEMS ENGINEERING: 33rd International Conference |date=2021 |publisher=Springer Nature |isbn=978-3-030-79382-1 |url=https://books.google.com/books?id=Q7k0EAAAQBAJ&dq=Apache+Pinot+-wikipedia&pg=PA384 |language=en}}</ref><ref>{{cite book |last1=Chin |first1=Francis Y. L. |last2=Chen |first2=C. L. Philip |last3=Khan |first3=Latifur |last4=Lee |first4=Kisung |last5=Zhang |first5=Liang-Jie |title=Big Data – BigData 2018: 7th International Congress, Held as Part of the Services Conference Federation, SCF 2018, Seattle, WA, USA, June 25–30, 2018, Proceedings |date=20 June 2018 |publisher=Springer |isbn=978-3-319-94301-5 |page=153 |url=https://books.google.com/books?id=eSVhDwAAQBAJ |language=en}}</ref><ref>{{cite book |last1=Im |first1=Jean-François |last2=Gopalakrishna |first2=Kishore |last3=Subramaniam |first3=Subbu |last4=Shrivastava |first4=Mayank |last5=Tumbde |first5=Adwait |last6=Jiang |first6=Xiaotian |last7=Dai |first7=Jennifer |last8=Lee |first8=Seunghyun |last9=Pawar |first9=Neha |last10=Li |first10=Jialiang |last11=Aringunram |first11=Ravi |title=Proceedings of the 2018 International Conference on Management of Data |chapter=Pinot: Realtime OLAP for 530 Million Users |series=Sigmod '18 |date=2018-05-27 |pages=583–594 |doi=10.1145/3183713.3190661 |url=https://dl.acm.org/doi/10.1145/3183713.3190661#d13801648e1 |publisher=Association for Computing Machinery|isbn=9781450347037 |s2cid=44083085 }}</ref><ref>{{cite web |title=The Apache Software Foundation Announces Apache® Pinot™ as a Top-Level Project |url=https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces76 |website=blogs.apache.org|date=2 August 2021 }}</ref> It is suited in contexts where fast analytics, such as aggregations, are needed on immutable data, possibly, with real-time data ingestion.<ref>{{cite arXiv |last1=Rogers |first1=Ryan |last2=Subramaniam |first2=Subbu |last3=Peng |first3=Sean |last4=Durfee |first4=David |last5=Lee |first5=Seunghyun |last6=Kancha |first6=Santosh Kumar |last7=Sahay |first7=Shraddha |last8=Ahammad |first8=Parvez |title=LinkedIn's Audience Engagements API: A Privacy Preserving Data Analytics System at Scale |date=16 November 2020 |class=cs.CR |eprint=2002.05839}}</ref><ref>{{cite book |last1=Javadi |first1=Seyyed Ahmad |last2=Gupta |first2=Harsh |last3=Manhas |first3=Robin |last4=Sahu |first4=Shweta |last5=Gandhi |first5=Anshul |title=2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS) |chapter=EASY: Efficient Segment Assignment Strategy for Reducing Tail Latencies in Pinot |date=July 2018 |pages=1432–1437 |doi=10.1109/ICDCS.2018.00144 |isbn=978-1-5386-6871-9 |s2cid=21659844 |chapter-url=https://ieeexplore.ieee.org/document/8416407}}</ref><ref name="pinot-joins-apache-foundation">Pawar, Neha. [https://engineering.linkedin.com/blog/2019/03/pinot-joins-apache-incubator "Pinot Joins Apache Incubator"] {{Webarchive|url=https://web.archive.org/web/20190402090136/https://engineering.linkedin.com/blog/2019/03/pinot-joins-apache-incubator |date=2019-04-02 }}, ''LinkedIn Engineering'', 01 April 2019</ref> The name Pinot comes from the [[Pinot grape]] vines that are pressed into liquid that is used to produce a variety of different wines. The founders of the database chose the name as a metaphor for analyzing vast quantities of data from a variety of different file formats or streaming data sources.<ref name="open-sourcing-pinot">{{cite web |last1=Gopalakrishna |first1=Kishore |title=Open Sourcing Pinot: Scaling the Wall of Real-Time Analytics |url=https://engineering.linkedin.com/pinot/open-sourcing-pinot-scaling-wall-real-time-analytics |website=engineering.linkedin.com |publisher=LinkedIn |accessdate=3 September 2020 |archiveurl=https://web.archive.org/web/20150910081445/http://engineering.linkedin.com/pinot/open-sourcing-pinot-scaling-wall-real-time-analytics |archivedate=10 September 2015 |language=en}}</ref>

Pinot was first created at [[LinkedIn]] after the engineering staff determined that there were no off the shelf solutions that met the social networking site's requirements like predictable low latency, data freshness in seconds, fault tolerance and scalability.<ref name="open-sourcing-pinot" /><ref>{{cite news |last1=Yegulalp |first1=Serdar |title=LinkedIn fills another SQL-on-Hadoop niche |url=https://www.infoworld.com/article/2934506/linkedins-pinot-fills-another-sql-on-hadoop-niche.html |work=InfoWorld |date=2015-06-11 |language=en}}</ref> Pinot is used in production by technology companies such as [[Uber]],<ref>{{cite book |last1=Fu |first1=Yupeng |last2=Soman |first2=Chinmay |title=Proceedings of the 2021 International Conference on Management of Data |chapter=Real-time Data Infrastructure at Uber |series=Sigmod/Pods '21 |date=9 June 2021 |pages=2503–2516 |doi=10.1145/3448016.3457552 |chapter-url=https://dl.acm.org/doi/abs/10.1145/3448016.3457552 |publisher=Association for Computing Machinery|arxiv=2104.00087 |isbn=9781450383431 |s2cid=232478317 }}</ref> [[Microsoft]],<ref name="pinot-joins-apache-foundation" /> and [[Factual]].

Line 60:

[[Category:Structured storage]]

[[Category:Free database management systems]]

[[Category:Free software programmed in Java (programming language)]]

[[Category:Database engines]]

[[Category:Big data products]]