Distributed data store: Difference between revisions

Revision as of 15:55, 9 September 2011

Latest revision as of 12:48, 10 August 2025

(124 intermediate revisions by 96 users not shown)

Line 1:

{{Short description|Computer network with mulitple nodes to store information}}

A '''distributed data store''' is a blurred concept and means either a [[distributed database]] where users store their information on a ''number of nodes'', or a network in which a user stores their information on a ''number of peer network nodes'' .

{{Essay-like|date=May 2012}}

{{Memory types}}

A '''distributed data store''' is a [[computer network]] where information is stored on more than one [[Node (networking)|node]], often in a [[Replication (computing)|replicated]] fashion.<ref>{{Citation

|author = Yaniv Pessach

|title = Distributed Storage

|edition = Distributed Storage: Concepts, Algorithms, and Implementations

|ol = 25423189M

}}</ref> It is usually specifically used to refer to either a [[distributed database]] where users store information on a ''number of nodes'', or a [[computer network]] in which users store information on a ''number of peer network nodes''.{{fact|date=August 2025}}

==Distributed databases==

[[Distributed data storedatabase]]s are usually [[non-relational databasesdatabase]]s that makeenable a quick access to data over a large number of nodes possible. ExamplesSome fordistributed thisdatabases kindexpose rich query abilities while others are limited to a [[key-value store]] semantics. Examples of datalimited storesdistributed databases are [[Google]]'s [[BigTableBigtable]], which is much more than a [[distributed file system]] or a [[peer-to-peer network]],<ref>{{cite web

| accessdateaccess-date = 2011-04-05

| ___location = http://the-paper-trail.org/

| publisher = Paper Trail

| title = BigTableBigtable: Google's Distributed Data Store

| quote = Although GFS provides Google with reliable, scalable distributed file storage, it does not provide any facility for structuring the data contained in the files beyond a hierarchical directory structure and meaningful file names. It’s well known that more expressive solutions are required for large data sets. Google’s terabytes upon terabytes of data that they retrieve from web crawlers, amongst many other sources, need organising, so that client applications can quickly perform lookups and updates at a finer granularity than the file level. [...] The very first thing you need to know about BigTableBigtable is that it isn’t a relational database. This should come as no surprise: one persistent theme through all of these large scale distributed data store papers is that RDBMSs are hard to do with good performance. There is no hard, fixed schema in a BigTableBigtable, no referential integrity between tables (so no foreign keys) and therefore little support for optimised joins.

| url = http://the-paper-trail.org/blog/?p=86}}</ref> or [[Amazon.com|Amazon]]'s [[Dynamo (storage system)|Dynamo]].<ref>{{cite web

| archive-url = https://web.archive.org/web/20170716092550/http://the-paper-trail.org/blog/bigtable-googles-distributed-data-store

| accessdate = 2011-04-05

| archive-date = 2017-07-16

| url-status = dead

}}</ref> [[Amazon.com|Amazon]]'s [[Dynamo (storage system)|Dynamo]]<ref>{{cite web

| access-date = 2011-04-05

| author = Sarah Pidcock

| date = 2011-01-31

| ___location = http://www.cs.uwaterloo.ca/

| page = 2/22

| publisher = WATERLOO |– CHERITON SCHOOL OF COMPUTER SCIENCE

| title = Dynamo: Amazon’sAmazon's Highly Available Key-value Store

| quote = Dynamo: a highly available and scalable distributed data store

| url = http://www.cs.uwaterloo.ca/~kdaudjee/courses/cs848/slides/sarah1.pdf}}</ref>

and [[Azure Services Platform|Microsoft Azure Storage]].<ref>{{cite web|url=http://www.microsoft.com/windowsazure/features/storage/|title=Windows Azure Storage|website=[[Microsoft]] |date=2011-09-16|access-date=6 November 2011|url-status=dead|archive-url=https://web.archive.org/web/20111109002826/http://www.microsoft.com/windowsazure/features/storage/|archive-date=9 November 2011}}</ref>

or Windows Azure Storage (www.azure.com)

As the ability of arbitrary querying is not as important as the [[availability]], designers of distributed data stores have increased the latter at an expense of consistency. But the high-speed read/write access results in reduced consistency, as it is not possible to haveguarantee both [[Consistency (database systems)|consistency]], [[availability]], and partitionavailability toleranceon ofa thepartitioned network, as it has been provenstated by the [[CAP theorem]].

==Peer network node data stores==

In peer network data stores, the user can usually reciprocate and allow other users to use their computer as a storage node as well. Information may or may not be accessible to other users depending on the design of the network.

Most of the [[peer-to-peer]] networks do not have distributed data stores in that the user's data is only available when their node is on the network. However, this distinction is somewhat blurred in a system such as [[BitTorrent (protocol)|BitTorrent]], where it is possible for the originating node to go offline but the content to continue to be served. Still, this is only the case for individual files requested by the redistributors, as contrasted with a networknetworks such as [[FreenetHyphanet]], [[Winny]], [[Share (P2P)|Share]] and [[Perfect Dark (P2P)|Perfect Dark]] where allany computersnode aremay madebe availablestoring toany servepart allof the files on the network.

Distributed data stores typically use an [[error detection and correction]] technique.

Line 32 ⟶ 43:

==Examples==

===Distributed non-relational databases===

* [[Apache Cassandra]], the data store of [[Facebook]]

{| class="wikitable sortable"

* [[BigTable]], the data store of [[Google]]

|-

* [[Dynamo (storage system)|Dynamo]] of [[Amazon.com|Amazon]]

! Product

* [[Riak]]

! License

* [[Voldemort (distributed data store)|Voldemort]]

! [[High availability]]

! Notes

|-

| [[Apache Accumulo]]

| {{free|[[Apache License 2.0|AL2]]}}

|

|

|-

| [[Aerospike (database)|Aerospike]]

| {{free|[[GNU Affero General Public License|AGPL]]}}

|

|

|-

| [[Apache Cassandra]]

| {{free|[[Apache License 2.0|AL2]]}}

| {{yes}}

| formerly used by [[Facebook]]

|-

| [[Apache Ignite]]

| {{free|[[Apache License 2.0|AL2]]}}

|

|

|-

| [[Bigtable]]

| {{Proprietary}}

|

| used by [[Google]]

|-

| [[Couchbase]]

| {{free|[[Apache License 2.0|AL2]]}}

|

| used by [[LinkedIn]], [[PayPal]], and [[eBay]]

|-

| [[CrateDB]]

| {{free|[[Apache License 2.0|AL2]]}}

| {{yes}}

|

|-

| [[Apache Druid]]

| {{free|[[Apache License 2.0|AL2]]}}

|

| used by [[Netflix]], and [[Yahoo]]

|-

| [[Dynamo (storage system)|Dynamo]]

| {{Proprietary}}

|

| used by [[Amazon.com|Amazon]]

|-

| [[Container_Linux#ETCD|etcd]]

| {{free|[[Apache License 2.0|AL2]]}}

| {{yes}}

|

|-

| [[Hazelcast]]

| {{proprietary|[[Apache License 2.0|AL2]], Proprietary}}

|

|

|-

| [[HBase]]

| {{free|[[Apache License 2.0|AL2]]}}

| {{yes}}

| formerly used by Facebook

|-

| [[Hypertable]]

| {{free|[[GPL 2]]}}

|

| [[Baidu]]

|-

| [[MongoDB]]

| {{proprietary|[[Server Side Public License|SSPL]]}}

|

|

|-

| [[MySQL Cluster|MySQL NDB Cluster]]

| {{free|[[GPL 2]]}}

| {{yes}}

| SQL and NoSQL APIs

|-

| [[Riak]]

| {{free|[[Apache License 2.0|AL2]]}}

| {{yes}}

|

|-

|-

| [[Redis]]

| {{free|[[BSD License]]}}

| {{yes}}

|

|-

| [[ScyllaDB]]

| {{free|[[GNU Affero General Public License|AGPL]]}}

|

|

|-

| [[Voldemort (distributed data store)|Voldemort]]

| {{free|[[Apache License 2.0|AL2]]}}

|

| used by [[LinkedIn]]

|-

|}

===Peer network node data stores===

* [[BitTorrent (protocol)|BitTorrent]]

* [[Blockchain (database)]]

* [[Chord project]]

* [[GNUnet]]

* [[Freenet]]

* [[GNUnet]]

* [[Network News Transfer Protocol|NNTP]] (the distributed data storage protocol used for [[Usenet]] news)

* [[InterPlanetary File System|IPFS]]

* [[Mnet (Computer program)|Mnet]]

* [[Napster]]

* [[Network News Transfer Protocol|NNTP]] (the distributed data storage protocol used for [[Usenet]] news)

* Unity, of the software [[Perfect Dark (P2P)|Perfect Dark]]

* [[Share (P2P)|Share]]

* [[Siacoin]]

* DeNet

* [[Storage@home]]

* [[WualaTahoe-LAFS]]

* [[Winny]]

* [[ZeroNet]]

==References==

{{Reflist}}

==See also==

{{Portal|Computer Science}}

* [[Cooperative storage cloud]]

* [[Data store]]

* [[Distributed file system]]

* [[Keyspace (distributed data store)|Keyspace]], the DDS [[Schema (database)|schema]]

* [[Peer-to-peer]]

* [[Distributed hash table]]

* [[Distributed cache]]

* [[Cyber Resilience]]

==References==

{{Compu-network-stub}}

{{Reflist}}

<!--Categories-->

[[Category:Data management]]

[[Category:Distributed data storage| ]]

[[Category:Distributed data stores| ]]

<!--Interwikies-->

[[ja:分散ファイルシステム#分散データストア]]