Moose File System: Difference between revisions

Revision as of 02:31, 29 March 2023

Revision as of 14:13, 22 October 2024

Line 28:

==Design==

The MooseFS follows similar design principles as [[Fossil (file system)|Fossil]], [[Google File System]], [[Lustre (file system)|Lustre]] or [[Ceph (file system)|Ceph]]. The file system comprises three components:

* Metadata server (MDS) — manages the ___location (layout) of files, file access and namespace hierarchy. The current version of MooseFS does support multiple metadata servers and automatic [[failover]]. Clients only talk to the MDS to retrieve/update a file's layout and attributes; the data itself is transferred directly between clients and chunk servers. The Metadata server is a user-space [[daemon (computer software)|daemon]]; the metadata is kept in memory and lazily stored on local disk.

Line 38:

To achieve high reliability and performance MooseFS offers the following features:

* [[Fault-tolerance]] — MooseFS uses [[Replication (computer science)|replication]], data can be replicated across chunkservers, the replication ratio (''N'') is set per file/directory. If (''N-1''−1) replicas fail the data will still be available. At the moment MooseFS does not offer any other technique for [[fault-tolerance]]. [[Fault-tolerance]] for very big files thus requires vast amount of space - ''N*'' × filesize instead of filesize + (''N*'' × stripesize) as would be the case for [[RAID 4]], [[RAID 5]] or [[RAID 6]]. Version 4.x PRO of MooseFS implements 8+''n'' [[Erasure code|Erasure Coding]].

* [[data striping|Striping]] — Large files are divided into chunks (up to 64 [[megabyte]]s) that might be stored on different chunk servers in order to achieve higher aggregate bandwidth.

* [[Load balancing (computing)|Load balancing]] — MooseFS attempts to use storage resources equally, the current algorithm seems to take into account only the consumed space.