Moose File System: Difference between revisions

Revision as of 12:13, 15 July 2022

Latest revision as of 12:48, 15 July 2025

(7 intermediate revisions by 6 users not shown)

Line 5:

}}

{{Infobox software

| logo = MooseFS logo.png

| logo size = 100px

| name = Moose File System

| developer = Jakub Kruszona-Zawadzki<ref>[https://github.com/moosefs/moosefs/graphs/contributors Contributors to moosefs/moosefs · GitHub<!-- Bot generated title -->]</ref> / Core Technology<ref>{{cite web |url=https://moosefs.com/about/company.html |title=About us - Core Technology - MooseFS fault tolerant network distributed file system |publisher=Core Technology |access-date=2016-03-26 |archive-date=2016-04-04 |archive-url=https://web.archive.org/web/20160404172312/https://moosefs.com/about/company.html |url-status=live }}</ref>

| released = {{Start date and age|2008|05|30|df=yes}}<ref>"Date of the first public release: 2008-05-30" https://github.com/moosefs/moosefs/blob/master/README.md {{Webarchive|url=https://web.archive.org/web/20190429203006/https://github.com/moosefs/moosefs/blob/master/README.md |date=2019-04-29 }}</ref> (v. 1.5.0<ref>"MooseFS 1.5 (2008-05-30)" https://github.com/moosefs/moosefs/blob/master/NEWS</ref>)

| latest release version = 34.057.116-17

| latest release date = {{Start date and age|20212025|0807|1201|df=yes}}<ref name="MooseFS Support Status">{{cite web | url=https://moosefs.com/support/#moosefs-support-status | title=Support – documentation, status and best practices – MooseFS | access-date=2019-04-12 | archive-date=2019-04-12 | archive-url=https://web.archive.org/web/20190412014552/https://moosefs.com/support/#moosefs-support-status | url-status=live }}</ref><ref name="Latest changes in MooseFS on GitHub">{{cite web | url=https://github.com/moosefs/moosefs/blob/master/NEWS | title=moosefs/NEWS at master · moosefs/moosefs| website=[[GitHub]]| date=14 July 2022}}</ref><ref name="MooseFS releases on GitHub">{{cite web | url=https://github.com/moosefs/moosefs/releases | title=Releases · moosefs/moosefs| website=[[GitHub]]}}</ref>

| latest preview version = 34.057.116-17

| latest preview date = {{Start date and age|20212025|0807|1201|df=yes}}<ref name="MooseFS Support Status" /><ref name="Latest changes in MooseFS on GitHub" /><ref name="MooseFS releases on GitHub" />

| operating system = [[Linux]], [[FreeBSD]], [[NetBSD]], [[macOS]], [[Solaris (operating system)|Solaris]], [[OpenIndiana]],<ref>"We also successfully compiled MooseFS from sources on OpenIndiana Hipster." https://moosefs.com/download.html {{Webarchive|url=https://web.archive.org/web/20160323123210/https://moosefs.com/download.html |date=2016-03-23 }}</ref>

| genre = [[Distributed file system]]

| license = [[GPLv2]] / [[Proprietary software|proprietary]]

| website = [https://moosefs.com https://moosefs.com]

| repo = <!-- Refer to Wikidata. -->

}}

Line 28:

==Design==

The MooseFS follows similar design principles as [[Fossil (file system)|Fossil]], [[Google File System]], [[Lustre (file system)|Lustre]] or [[Ceph (file system)|Ceph]]. The file system comprises three components:

* Metadata server (MDS) — manages the ___location (layout) of files, file access and namespace hierarchy. The current version of MooseFS does support multiple metadata servers and automatic [[failover]]. Clients only talk to the MDS to retrieve/update a file's layout and attributes; the data itself is transferred directly between clients and chunk servers. The Metadata server is a user-space [[daemon (computer software)|daemon]]; the metadata is kept in memory and lazily stored on local disk.

Line 38:

To achieve high reliability and performance MooseFS offers the following features:

* [[Fault-tolerance]] — MooseFS uses [[Replication (computer science)|replication]], data can be replicated across chunkservers, the replication ratio (''N'') is set per file/directory. If (''N-1''−1) replicas fail the data will still be available. At the moment MooseFS does not offer any other technique for [[fault-tolerance]]. [[Fault-tolerance]] for very big files thus requires vast amount of space - ''N*'' × filesize instead of filesize + (''N*'' × stripesize) as would be the case for [[RAID 4]], [[RAID 5]] or [[RAID 6]]. Version 4.x PRO of MooseFS implements 8+''n'' [[Erasure code|Erasure Coding]].

* [[data striping|Striping]] — Large files are divided into chunks (up to 64 [[megabyte]]s) that might be stored on different chunk servers in order to achieve higher aggregate bandwidth.

* [[Load balancing (computing)|Load balancing]] — MooseFS attempts to use storage resources equally, the current algorithm seems to take into account only the consumed space.

Line 44:

* [[Snapshot (computer storage)|Coherent snapshots]] — Quick, low-overhead snapshots.

* Transparent "trash bin" — Deleted files are retained for a configurable period of time.

* Data tiering / storage classes — Possibility to "label" servers, create label definitions called "Storage Classes" and decide, on which types of servers the data is stored<ref>MooseFS 3.0 Storage Classes Manual https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf {{Webarchive|url=https://web.archive.org/web/20160807143350/https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf |date=2016-08-07 }}</ref>

* [[Disk quota|"Project" quotas]] support

* POSIX locks, flock locks support

Line 51:

Similarly to other cluster-based file systems MooseFS uses [[commodity hardware]] running a [[POSIX]] compliant operating system. [[TCP/IP]] is used as the interconnect.

==MooseFS in figures==

==MooseFS in figures<ref>[https://moosefs.com/factsheet MooseFS Factsheet<!-- Bot generated title -->]</ref>==

Source:<ref>{{Cite web |url=https://moosefs.com/factsheet |title=MooseFS Factsheet<!-- Bot generated title --> |access-date=2016-07-01 |archive-date=2016-09-10 |archive-url=https://web.archive.org/web/20160910023055/https://moosefs.com/factsheet/ |url-status=live }}</ref>

* Storage size is up to: 2⁶⁴ [[Byte]]s = 16 EiB = 16 384 PiB

* Single file size is up to: 2⁵⁷ Bytes = 128 PiB

* Number of files is up to: 2³¹ = 2.1 × 10⁹billion

* Number of active clients is unlimited it depends on number of file descriptors in the system