Comparison of distributed file systems: Difference between revisions

Content deleted Content added
Chkno (talk | contribs)
FOSS: Fix "|section= ignored" citation template error
Chkno (talk | contribs)
FOSS: Add column: Redundancy Granularity
Line 16:
! [[Shard (database architecture)|Shards]]
! [[Erasure code|Efficient Resiliency]]
! Redundancy Granularity
! Initial release year
! Memory requirements (GB)
Line 25 ⟶ 26:
| {{no|hot standby}}
| {{no}}
| {{no|Replication}}<ref>{{cite web |url=https://docs.alluxio.io/os/user/stable/en/core-services/Caching.html#managing-data-replication-in-alluxio |title=Caching: Managing Data Replication in Alluxio}}</ref>
| {{no|Replication}}
| {{yes|File}}<ref>{{cite web |url=https://docs.alluxio.io/os/user/stable/en/core-services/Caching.html#managing-data-replication-in-alluxio |title=Caching: Managing Data Replication in Alluxio}}</ref>
| 2013
|
Line 36 ⟶ 38:
| {{yes}}
| {{yes|Pluggable erasure codes}}<ref>{{cite web |url=https://docs.ceph.com/en/latest/rados/operations/erasure-code-profile/ |title=Erasure Code Profiles }}</ref>
| {{no|Pool}}<ref>{{cite web |url=https://docs.ceph.com/en/latest/rados/operations/pools/ |title=Pools}}</ref>
| 2010
| 1 per TB of storage
Line 46 ⟶ 49:
| {{yes}}
| {{no|Replication}}
| {{no|Volume}}<ref>{{cite journal
|first1=Mahadev |last1=Satyanarayanan
|first2=James J. |last2=Kistler
|first3=Puneet |last3=Kumar
|first4=Maria E. |last4=Okasaki
|first5=Ellen H. |last5=Siegel
|first6=David C. |last6=Steere
|title=Coda: A Highly Available File System for a Distributed Workstation Environment
|url=https://www.csee.umbc.edu/courses/graduate/CMSC621/fall2006/lectures/coda.pdf
}}</ref>
| 1987
|
Line 56 ⟶ 69:
| {{yes}}
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://github.com/gluster/glusterfs/blob/master/doc/developer-guide/ec-implementation.md |title=Erasure coding implementation }}</ref>
| {{no|Volume}}<ref>{{cite web |url=https://docs.gluster.org/en/latest/Administrator%20Guide/Setting%20Up%20Volumes/ |title=Setting up GlusterFS Volumes}}</ref>
| 2005
|
Line 66 ⟶ 80:
| {{no}}
| {{no|Replication}}<ref>Only available in the proprietary version 4.x {{cite web |url=https://github.com/moosefs/moosefs/issues/8 |title=[feature] erasure-coding #8}}</ref>
| {{yes|File}}<ref>{{cite web |url=https://fossies.org/linux/moosefs/mfsmanpages/mfsgoal.1 |title=mfsgoal(1)}}</ref>
| 2008
|
Line 76 ⟶ 91:
| {{no}}
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://www.cs.utah.edu/~hari/teaching/bigdata/qfs-ovsiannikov.pdf |title=The Quantcast File System}}</ref>
| {{yes|File}}<ref>{{cite web |url=https://github.com/quantcast/qfs/blob/2.2.2/src/cc/tools/cptoqfs_main.cc#L259 |title=qfs/src/cc/tools/cptoqfs_main.cc}}</ref>
| 2012
|
Line 86 ⟶ 102:
|
| {{no|Replication}}
|
|2020
|
Line 96 ⟶ 113:
| {{no}}
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://docs.lizardfs.com/adminguide/replication.html |title=Configuring Replication Modes}}</ref>
| {{yes|File}}<ref>{{cite web |url=https://docs.lizardfs.com/adminguide/replication.html#set-and-show-the-goal-of-a-file-directory |title=Configuring Replication Modes: Set and show the goal of a file/directory}}</ref>
| 2013
|
Line 106 ⟶ 124:
| {{yes}}
| {{no|No redundancy}}<ref>{{cite web |url=https://doc.lustre.org/lustre_manual.xhtml#understandinglustre.whatislustre |title=Lustre Operations Manual: What a Lustre File System Is (and What It Isn't)}} Reed-Solomon in progress: {{cite web |url=https://jira.whamcloud.com/browse/LU-10911 |title=LU-10911 FLR2: Erasure coding}}</ref>
| {{no|No redundancy}}<ref>{{cite web |url=https://doc.lustre.org/lustre_manual.xhtml#understandinglustre.whatislustre |title=Lustre Operations Manual: What a Lustre File System Is (and What It Isn't)}} File-level redundancy plan: {{cite web |url=https://wiki.lustre.org/File_Level_Redundancy_Solution_Architecture |title=File Level Redundancy Solution Architecture}}</ref>
| 2003
|
Line 117 ⟶ 136:
| {{yes}}
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://docs.min.io/docs/minio-erasure-code-quickstart-guide.html |title=MinIO Erasure Code Quickstart Guide}}</ref>
| {{yes|Object}}<ref>{{cite web |url=https://github.com/minio/minio/tree/master/docs/erasure/storage-class |title=MinIO Storage Class Quickstart Guide}}</ref>
| 2014
|
Line 127 ⟶ 147:
|
| {{no|Replication}}
| {{no|Volume}}<ref>{{cite web |url=http://docs.openafs.org/AdminGuide/HDRWQ192.html |title=Replicating Volumes (Creating Read-only Volumes)
}}</ref>
| 2000 <ref>https://www.openafs.org/release/openafs-1.0.html</ref>
|
Line 137 ⟶ 159:
|
| {{yes|Pluggable erasure codes}}<ref>{{cite web |url=https://docs.openio.io/latest/source/admin-guide/configuration_ec.html |title=Erasure Coding}}</ref>
| {{yes|Object}}<ref>{{cite web |url=https://docs.openio.io/latest/source/admin-guide/configuration_storagepolicies.html |title=Declare Storage Policies}}</ref>
| 2015
| 0.5
Line 147 ⟶ 170:
|
| {{yes|Mojette}}<ref>{{cite web |url=http://rozofs.github.io/rozofs/develop/AboutRozoFS.html#mojette-transform |title=About RozoFS: Mojette Transform}}</ref>
| {{no|Volume}}<ref>{{cite web |url=http://rozofs.github.io/rozofs/develop/SettingUpRozoFS.html#exportd-configuration-file |title=Setting up RozoFS: Exportd Configuration File}}</ref>
| 2011<ref>{{cite web |url=https://github.com/rozofs/rozofs/commit/9818e92f73fe4432c8d29236158e271da9ee3bf2 |title=Initial commit.}}</ref>
|
Line 157 ⟶ 181:
|
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://github.com/chrislusf/seaweedfs/wiki/Erasure-coding-for-warm-storage |title=Erasure Coding for warm storage}}</ref>
| {{no|Volume}}<ref>{{cite web |url=https://github.com/chrislusf/seaweedfs/wiki/Replication |title=Replication}}</ref>
| 2015
|
Line 169 ⟶ 194:
|
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://github.com/tahoe-lafs/zfec |title=zfec -- a fast C implementation of Reed-Solomon erasure coding}}</ref>
| {{yes|File}}<ref>{{cite web |url=https://tahoe-lafs.readthedocs.io/en/latest/architecture.html#file-encoding |title=Tahoe-LAFS Architecture: File Encoding}}</ref>
| 2007
|
Line 179 ⟶ 205:
| {{no}}
| {{yes|Reed-Solomon}}<ref>{{cite web |url=https://issues.apache.org/jira/browse/HDFS-7285 |title=HDFS-7285 Erasure Coding Support inside HDFS}}</ref>
| {{yes|File}}<ref>{{cite web |url=https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setrep |title=Apache Hadoop: setrep}}</ref>
| 2005
|
Line 189 ⟶ 216:
|
| {{no|Replication}}<ref>{{cite web |url=http://www.xtreemfs.org/how_replication_works.php |title=Under the Hood: File Replication}}</ref>
| {{yes|File}}<ref>{{cite web |url=http://www.xtreemfs.org/quickstart_repl.php |title=Quickstart: Replicate A File}}</ref>
| 2009
|
Line 201 ⟶ 229:
|
| {{no|Replication}}
| {{no|Filesystem}}<ref>{{cite journal
|first1=Ali Jose |last1=Mashtizadeh
|first2=Andrea |last2=Bittau
|first3=Yifeng Frank |last3=Huang
|first4=David |last4=Mazières
|title=Replication, History, and Grafting in the Ori File System
|url=http://sigops.org/s/conferences/sosp/2013/papers/p151-mashtizadeh.pdf
}}</ref>
| 2012
|