In-memory processing: Difference between revisions

Revision as of 12:12, 9 October 2023

Revision as of 11:16, 22 November 2023

Line 19:

Memory processing can be accomplished via traditional databases such as [[Oracle Database|Oracle]], [[IBM Db2]] or [[Microsoft SQL Server]] or via [[NoSQL]] offerings such as in-memory [[data grid]] like [[Hazelcast]], [[Infinispan]], [[Oracle Coherence]] or ScaleOut Software. With both in-memory database and [[data grid]], all information is initially loaded into memory RAM or flash memory instead of [[hard disk]]s. With a [[data grid]] processing occurs at three [[orders of magnitude|order of magnitude]] faster than relational databases which have advanced functionality such as [[ACID]] which degrade performance in compensation for the additional functionality. The arrival of [[Column-oriented DBMS|column centric databases]], which store similar information together, allow data to be stored more efficiently and with greater [[Data compression|compression]] ratios. This allows huge amounts of data to be stored in the same physical space, reducing the amount of memory needed to perform a query and increasing processing speed. Many users and software vendors have integrated flash memory into their systems to allow systems to scale to larger data sets more economically. Oracle has been integrating flash memory into the [[Oracle Exadata]] products for increased performance. [[Microsoft SQL Server]] 2012 BI/Data Warehousing software has been coupled with [[Violin Memory]] flash memory arrays to enable in-memory processing of data sets greater than 20TB.<ref>{{cite web|title=SQL Server 2012 with Violin Memory|url=http://download.microsoft.com/download/6/9/C/69CFB214-0699-4448-8F32-CFE03A0706A6/SQL_Server_2012_Fast_Track_Data_Warehouse_for_Violin_Memory_Datasheet.pdf|publisher=Microsoft|access-date=2013-06-01|archive-url=https://web.archive.org/web/20130309045249/http://download.microsoft.com/download/6/9/C/69CFB214-0699-4448-8F32-CFE03A0706A6/SQL_Server_2012_Fast_Track_Data_Warehouse_for_Violin_Memory_Datasheet.pdf|archive-date=2013-03-09|url-status=dead}}</ref>

Users query the data loaded into the system’ssystem's memory, thereby avoiding slower database access and performance [[Bottleneck (software)|bottlenecks]]. This differs from [[caching (computing)|caching]], a very widely used method to speed up query performance, in that caches are subsets of very specific pre-defined organized data. With in-memory tools, data available for analysis can be as large as a [[data mart]] or small data warehouse which is entirely in memory. This can be accessed quickly by multiple concurrent users or applications at a detailed level and offers the potential for enhanced analytics and for scaling and increasing the speed of an application. Theoretically, the improvement in data access speed is 10,000 to 1,000,000 times compared to the disk.{{citation needed|date=January 2016}} It also minimizes the need for performance tuning by IT staff and provides faster service for end users.

== Advantages of in-memory processing technology ==