Moose File System: Difference between revisions

Revision as of 17:54, 30 August 2016

Latest revision as of 12:48, 15 July 2025

(49 intermediate revisions by 18 users not shown)

Line 1:

{{short description|Distributed file system}}

{{Multiple issues|

{{Primary sources|date=April 2015}}

{{More citations needed|date=August 2016}}

}}

{{Infobox software

| logo = [[File:MooseFS logo.png|center|100px]]

| logo size = 100px

| name = Moose File System

| developer = Jakub Kruszona-Zawadzki<ref>[https://github.com/moosefs/moosefs/graphs/contributors Contributors to moosefs/moosefs · GitHub<!-- Bot generated title -->]</ref> / Core Technology<ref>{{cite web |url=https://moosefs.com/about/company.html |title=About us - Core Technology - MooseFS fault tolerant network distributed file system |publisher=Core Technology |access-date=2016-03-26 |archive-date=2016-04-04 |archive-url=https://web.archive.org/web/20160404172312/https://moosefs.com/about/company.html |url-status=live }}</ref>

| latest release version = 3.0.81-1

| latest release date released = {{Start date and age|20162008|0705|2530|df=yes}}<ref>"Date of the first public release: name="moosefs2008-05-download30">{{cite web |url=https://moosefsgithub.com/downloadmoosefs/moosefs/blob/master/README.htmlmd | title= Download MooseFS - fault tolerant network distributed file system}}</ref><ref>{{cite web Webarchive|url=https://moosefsweb.archive.org/web/20190429203006/https://github.com/documentationmoosefs/changes-in-moosefs-2-0/blob/master/README.htmlmd | titledate=2019-04-29 Latest changes in MooseFS 2.0}}</ref> (v. 1.5.0<ref>"MooseFS 1.5 (2008-05-30)" https://github.com/moosefs/moosefs/releasesblob/master/NEWS</ref>)

| latest previewrelease version = 34.057.81-17

| latest previewrelease date = {{Start date and age|20162025|07|2501|df=yes}}<ref name="moosefs-downloadMooseFS Support Status"/><ref>{{cite web | url=https://moosefs.com/documentationsupport/changes#moosefs-insupport-moosefsstatus | title=Support – documentation, status and best practices – MooseFS | access-3date=2019-004-12 | archive-date=2019-04-12 | archive-url=https://web.htmlarchive.org/web/20190412014552/https://moosefs.com/support/#moosefs-support-status | titleurl-status=live }}</ref><ref name="Latest changes in MooseFS 3on GitHub">{{cite web | url=https://github.0com/moosefs/moosefs/blob/master/NEWS | title=moosefs/NEWS at master · moosefs/moosefs| website=[[GitHub]]| date=14 July 2022}}</ref><ref name="MooseFS releases on GitHub">{{cite web | url=https://github.com/moosefs/moosefs/releases | title=Releases · moosefs/moosefs| website=[[GitHub]]}}</ref>

| latest releasepreview version = 34.057.81-17

| operating system = [[Linux]], [[FreeBSD]], [[Solaris (operating system)|Solaris]], [[OpenIndiana]],<ref>"We also successfully compiled MooseFS from sources on OpenIndiana Hipster." https://moosefs.com/download.html</ref> [[Mac OS X]]

| latest preview date = {{Start date and age|2025|07|01|df=yes}}<ref name="MooseFS Support Status" /><ref name="Latest changes in MooseFS on GitHub" /><ref name="MooseFS releases on GitHub" />

| genre = [[Distributed file system]]

| operating system = [[Linux]], [[FreeBSD]], [[NetBSD]], [[macOS]], [[Solaris (operating system)|Solaris]], [[OpenIndiana]],<ref>"We also successfully compiled MooseFS from sources on OpenIndiana Hipster." https://moosefs.com/download.html {{Webarchive|url=https://web.archive.org/web/20160323123210/https://moosefs.com/download.html |date=2016-03-23 }}</ref> [[Mac OS X]]

| license = [[GPLv2]] / [[Proprietary software|proprietary]]

| genre = [[Distributed file system]]

| website = <!-- Refer to Wikidata. -->

| license = [[GPLv2]] / [[Proprietary software|proprietary]]

| repo = <!-- Refer to Wikidata. -->

| website = [https://moosefs.com https://moosefs.com]

| website repo = <!-- Refer to Wikidata. -->

}}

'''Moose File System''' ('''MooseFS''') is an [[Open-source software|Openopen-source]], POSIX-compliant [[distributed file system]] developed by Core Technology. The lead developer is Jakub Kruszona-Zawadzki.<ref>https://github.com/moosefs/moosefs/graphs/contributors</ref> MooseFS aims to be [[Fault-tolerant design|fault-tolerant]], highly available, highly performing, scalable general-purpose network distributed file system for [[data center]]s. Initially proprietary codesoftware, it was released to the public as [[Open-source software|open source]] on May 530, 2008.

Currently two editions of MooseFS are available:

Line 22 ⟶ 28:

==Design==

The MooseFS follows similar design principles as [[Fossil (file system)|Fossil]], [[Google File System]], [[Lustre (file system)|Lustre]] or [[Ceph (file system)|Ceph]]. The file system comprises three components:

* Metadata server (MDS) — manages the ___location (layout) of files, file access and namespace hierarchy. The current version of MooseFS does support multiple metadata servers and automatic [[failover]]. Clients only talk to the MDS to retrieve/update a file's layout and attributes; the data itself is transferred directly between clients and chunk servers. The Metadata server is a user-space [[daemon (computer software)|daemon]]; the metadata is kept in memory and lazily stored on local disk.

Line 32 ⟶ 38:

To achieve high reliability and performance MooseFS offers the following features:

* [[Fault-tolerance]] — MooseFS uses [[Replication (computer science)|replication]], data can be replicated across chunkservers, the replication ratio (''N'') is set per file/directory. If (''N-1''−1) replicas fail the data will still be available. At the moment MooseFS does not offer any other technique for [[fault-tolerance]]. [[Fault-tolerance]] for very big files thus requires vast amount of space - ''N*'' × filesize instead of filesize + (''N*'' × stripesize) as would be the case for [[RAID 4]], [[RAID 5]] or [[RAID 6]]. Version 4.x PRO of MooseFS willimplements have8+''n'' RAID6[[Erasure code|Erasure Coding]].

* [[data striping|Striping]] — Large files are divided into chunks (up to 64 [[megabyte]]s) that might be stored on different chunk servers in order to achieve higher aggregate bandwidth.

* [[Load balancing (computing)|Load balancing]] — MooseFS attempts to use storage resources equally, the current algorithm seems to take into account only the consumed space.

Line 38 ⟶ 44:

* [[Snapshot (computer storage)|Coherent snapshots]] — Quick, low-overhead snapshots.

* Transparent "trash bin" — Deleted files are retained for a configurable period of time.

* Data tiering / storage classes — Possibility to "label" servers, create label definitions called "Storage Classes" and decide, on which types of servers the data is stored<ref>MooseFS 3.0 Storage Classes Manual https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf {{Webarchive|url=https://web.archive.org/web/20160807143350/https://moosefs.com/Content/Downloads/moosefs-storage-classes-manual.pdf |date=2016-08-07 }}</ref>

* [[Disk quota|"Project" quotas]] support

* POSIX locks, flock locks support

Line 45 ⟶ 51:

Similarly to other cluster-based file systems MooseFS uses [[commodity hardware]] running a [[POSIX]] compliant operating system. [[TCP/IP]] is used as the interconnect.

==MooseFS in figures<ref>https://moosefs.com/factsheet</ref>==

Source:<ref>{{Cite web |url=https://moosefs.com/factsheet |title=MooseFS Factsheet<!-- Bot generated title --> |access-date=2016-07-01 |archive-date=2016-09-10 |archive-url=https://web.archive.org/web/20160910023055/https://moosefs.com/factsheet/ |url-status=live }}</ref>

* Storage size is up to: 2⁶⁴ [[Byte]]s = 16 EiB = 16 384 PiB

* Single file size is up to: 2⁵⁷ [[Byte]]sBytes = 128 PiB

* Number of files is up to: 2³¹ = 2.1 × 10⁹billion

* Number of active clients is unlimited it depends on number of file descriptors in the system

==See also==

{{Portal|Free and open-source software}}

* [[Distributed file systemBeeGFS]]

* [[List of file systems#Distributed fault-tolerant file systems|List of file systems, the distributed fault-tolerant file system section]]

* [[Ceph (file system)|Ceph]]

* [[LustreDistributed (file system)|Lustre]]

* [[GlusterFS]]

* [[Google File System]]

* [[List of file systems#Distributed fault-tolerant file systems|List of file systems, the§ distributedDistributed fault-tolerant file system sectionsystems]]

* [[LizardFS]]{{snd}} a fork of MooseFS v. 1.6.x

* [[Lustre (file system)|Lustre]]

==References==

Line 66 ⟶ 76:

* {{GitHub|moosefs/moosefs}}

* {{SourceForge|moosefs|MooseFS}}

* [http://lizardfs.org/ LizardFS]{{snd}} a fork of old version (1.6.27) of MooseFS licensed under the terms of the [[GPL3|GPL-3]]

* A series of articles about MooseFS installation and first usage:

** [https://sudomakemelasagne.wordpress.com/2015/03/30/moosefs-on-os-x-1-primer/ MooseFS on Ubuntu/OS X | 1 | Primer]

** [https://sudomakemelasagne.wordpress.com/2015/09/01/moosefs-on-os-x-2-laying-the-groundwork/ MooseFS on Ubuntu/OS X | 2 | Laying The Groundwork]

** [https://sudomakemelasagne.wordpress.com/2015/09/18/moosefs-on-ubuntuos-x-3-mfsmaster-the-moosefs-master-server/ MooseFS on Ubuntu/OS X | 3 | mfsmaster – the MooseFS Master Server]

** [https://sudomakemelasagne.wordpress.com/2015/09/25/moosefs-on-ubuntuos-x-4-mfschunkserver/ MooseFS on Ubuntu/OS X | 4 | mfschunkserver]

** [https://sudomakemelasagne.wordpress.com/2015/09/25/moosefs-on-ubuntuos-x-5-mfsmount/ MooseFS on Ubuntu/OS X | 5 | mfsmount]

** [https://sudomakemelasagne.wordpress.com/2015/11/30/moosefs-on-ubuntuos-x-6-goals-holy-sht/ MooseFS on Ubuntu/OS X | 6 | Goals. Holy sh*t. ]

[[Category:Distributed file systems]]