Content deleted Content added
→References: ce |
Citation bot (talk | contribs) Removed URL that duplicated identifier. Removed parameters. | Use this bot. Report bugs. | Suggested by Headbomb | Linked from Wikipedia:WikiProject_Academic_Journals/Journals_cited_by_Wikipedia/Sandbox | #UCB_webform_linked 901/1032 |
||
| (524 intermediate revisions by more than 100 users not shown) | |||
Line 1:
{{Short description|Computer filing system}}
{{About|how a computer organizes and accesses computer files|library and office filing systems|Library classification}}
{{OS}}
In [[computing]], a '''file system''' or '''filesystem''' (often abbreviated to '''FS''' or '''fs''') governs [[computer file|file]] organization and access. A ''local'' file system is a capability of an [[operating system]] that services the applications running on the same [[computer]].<ref>{{cite web | title=5.10. Filesystems
|url=https://tldp.org/LDP/sag/html/filesystems.html
|publisher= The Linux Document Project
|access-date=December 11, 2021
|quote=A ''filesystem'' is the methods and data structures that an operating system uses to keep track of files on a disk or partition; that is, the way the files are organized on the disk.}}</ref><ref>{{citation|title=File System Implementation|url=http://pages.cs.wisc.edu/~remzi/OSTEP/file-implementation.pdf|publisher= Arpaci-Dusseau Books|year = 2014|first1 = Remzi H.|last1 =Arpaci-Dusseau|first2=Andrea C.|last2 = Arpaci-Dusseau}}</ref> A [[distributed file system]] is a [[Communication protocol|protocol]] that provides file access between [[computer network|networked]] computers.
A file system provides a [[computer data storage|data storage]] [[Service (systems architecture)|service]] that allows [[application software|application]]s to share [[mass storage]]. Without a file system, applications could access the storage in [[Software incompatibility|incompatible]] ways that lead to [[resource contention]], [[data corruption]] and [[data loss]].
There are many
File systems have been developed for many types of [[Computer storage device|storage devices]], including [[hard disk drive]]s (HDDs), [[solid-state drive]]s (SSDs), [[magnetic tape]]s and [[optical disc]]s.<ref>{{cite web | title=Storage, IT Technology and Markets, Status and Evolution
|url=https://indico.cern.ch/event/713888/contributions/3122779/attachments/1719287/2774787/storage_tech_market_BPS_Sep2018_v6.pdf
|date=September 20, 2018
|quote=HDD still key storage for the foreseeable future, SSDs not cost effective for capacity}}</ref>
A portion of the computer [[random-access memory|main memory]] can be set up as a [[RAM disk]] that serves as a storage device for a file system. File systems such as [[tmpfs]] can store files in [[virtual memory]].
{{Anchor|VIRTUAL-FILE}}
A ''virtual'' file system provides access to files that are either computed on request, called ''virtual files'' (see [[procfs]] and [[sysfs]]), or are mapping into another, backing storage.
== Etymology ==
From {{circa|1900}} and before the advent of computers the terms ''file system'', ''filing system'' and ''system for filing'' were used to describe methods of organizing, storing and retrieving paper documents.<ref>{{cite book|last1=McGill|first1=Florence E.|title=Office Practice and Business Procedure|date=1922|publisher=Gregg Publishing Company|page=[https://archive.org/details/officepracticea01mcgigoog/page/n211 197]|url=https://archive.org/details/officepracticea01mcgigoog|access-date=August 1, 2016}}</ref> By 1961, the term ''file system'' was being applied to computerized filing alongside the original meaning.<ref>{{cite book|last1=Waring|first1=R.L.|title=Technical investigations of addition of a hardcopy output to the elements of a mechanized library system : final report, 20 Sept. 1961|date=1961|publisher=Svco Corporation|___location=Cincinnati, OH|oclc=310795767}}</ref> By 1964, it was in general use.<ref>{{cite book|title=Disc File Applications: Reports Presented at the Nation's First Disc File Symposium|date=1964|publisher=American Data Processing|url=https://books.google.com/books?id=hJBWAAAAMAAJ|access-date=August 1, 2016}}</ref>
== Architecture ==
A local file system's [[software architecture|architecture]] can be described as [[Abstraction layer|layers of abstraction]] even though a particular file system design may not actually separate the concepts.<ref name="JHU">{{cite web|last1=Amir|first1=Yair|title=Operating Systems 600.418 The File System|url=http://www.cs.jhu.edu/~yairamir/cs418/os7/sld001.htm|website=Department of Computer Science Johns Hopkins University|access-date=July 31, 2016}}</ref>
The ''logical file system'' layer provides relatively high-level access via an [[application programming interface]] (API) for file operations including open, close, read and write {{endash}} delegating operations to lower layers. This layer manages open file table entries and per-process file descriptors.<ref name="IBMKC">{{cite web|last1=IBM Corporation|title=Component Structure of the Logical File System|url=https://www.ibm.com/docs/en/aix/7.3?topic=overview-component-structure-logical-file-system|website=IBM Knowledge Center|access-date=April 24, 2024}}</ref> It provides file access, directory operations, security and protection.<ref name=JHU />
The ''virtual file system'', an optional layer, supports multiple concurrent instances of physical file systems, each of which is called a file system implementation.<ref name=IBMKC />
The ''physical file system'' layer provides relatively low-level access to a storage device (e.g. disk). It reads and writes [[Block (data storage)|data blocks]], provides [[Data buffer|buffer]]ing and other [[memory management]] and controls placement of blocks in specific locations on the storage medium. This layer uses [[device driver]]s or [[channel I/O]] to drive the storage device.<ref name=JHU />
== Attributes ==
=== File names ===
{{Main|Filename}}
A '''file name''', or '''filename''', identifies a file to consuming applications and in some cases users.
A file name is unique so that an application can refer to exactly one file for a particular name. If the file system supports directories, then generally file name uniqueness is enforced within the context of each directory. In other words, a storage can contain multiple files with the same name, but not in the same directory.
Most file systems restrict the length of a file name.
Some file systems match file names as [[Case sensitivity|case sensitive]] and others as case insensitive. For example, the names <code>MYFILE</code> and <code>myfile</code> match the same file for case insensitive, but different files for case sensitive.
Most modern file systems allow a file name to contain a wide range of characters from the [[Unicode]] character set. Some restrict characters such as those used to indicate special attributes such as a device, device type, directory prefix, file path separator, or file type.
=== Directories ===
{{Main|Directory (computing)}}
File systems typically support organizing files into '''directories''', also called '''folders''', which segregate files into groups.
This may be implemented by associating the file name with an index in a [[table of contents]] or an [[inode]] in a [[Unix-like]] file system.
Directory structures may be flat (i.e. linear), or allow hierarchies by allowing a directory to contain directories, called subdirectories.
The first file system to support arbitrary hierarchies of directories was used in the [[Multics]] operating system.<ref>{{cite conference|chapter-url=http://www.multicians.org/fjcc4.html|chapter=A General-Purpose File System For Secondary Storage|author=R. C. Daley|author2=P. G. Neumann|title=Proceedings of the November 30--December 1, 1965, fall joint computer conference, Part I on XX - AFIPS '65 (Fall, part I) |year=1965|conference=Fall Joint Computer Conference|publisher=[[AFIPS]]|pages=213–229|doi=10.1145/1463891.1463915|access-date=2011-07-30|doi-access=free}}</ref> The native file systems of Unix-like systems also support arbitrary directory hierarchies, as do, [[Apple Inc.|Apple]]'s [[Hierarchical File System (Apple)|Hierarchical File System]] and its successor [[HFS Plus|HFS+]] in [[classic Mac OS]], the [[File Allocation Table|FAT]] file system in [[MS-DOS]] 2.0 and later versions of MS-DOS and in [[Microsoft Windows]], the [[NTFS]] file system in the [[Windows NT]] family of operating systems, and the ODS-2 (On-Disk Structure-2) and higher levels of the [[Files-11]] file system in [[OpenVMS]].
{{Anchor|METADATA}}
=== Metadata ===
In addition to data, the file content, a file system also manages associated [[metadata]] which may include but is not limited to:
* name
* [[file size|size]] which may be stored as the number of blocks allocated or as a [[byte]] count
* [[system time|when]] created, last accessed, last backed-up
* owner [[user ID|user]] and [[group ID|group]]
* [[file system permissions|access permissions]]
* [[file attribute]]s such as whether the file is read-only, [[executable]], etc.
* [[device file|device type]] (e.g. [[Block devices|block]], [[Character devices|character]], [[Internet socket|socket]], [[subdirectory]], etc.)
A file system stores associated metadata separate from the content of the file.
Most file systems store the names of all the files in one directory in one place—the directory table for that directory—which is often stored like any other file.
Line 47 ⟶ 88:
Additional attributes can be associated on file systems, such as [[NTFS]], [[XFS]], [[ext2]], [[ext3]], some versions of [[Unix File System|UFS]], and [[HFS+]], using [[extended file attributes]]. Some file systems provide for user defined attributes such as the author of the document, the character encoding of a document or the size of an image.
Some file systems allow for different data collections to be associated with one file name. These separate collections may be referred to as ''streams'' or ''forks''. Apple has long used a forked file system on the Macintosh, and Microsoft supports streams in NTFS. Some file systems maintain multiple past revisions of a file under a single file name; the
See [[comparison of file systems#Metadata|comparison of file systems § Metadata]] for details on which file systems support which kinds of metadata.
=== Storage space organization ===
A local file system tracks which areas of storage belong to which file and which are not being used.
When a file system creates a file, it allocates space for data. Some file systems permit or require specifying an initial space allocation and subsequent incremental allocations as the file grows.
To delete a file, the file system records that the file's space is free; available to use for another file.
[[File:100 000-files 5-bytes each -- 400 megs of slack space.png|frame|An example of slack space, demonstrated with 4,096-[[byte]] NTFS clusters: 100,000 files, each five bytes per file, which equal to 500,000 bytes of actual data but require 409,600,000 bytes of disk space to store <!-- The size listing shown in Explorer is oddly doubly-wrong. The example files are 5 bytes each, not 0.1K, and the clusters are a minimum of 4K not 1K.-->]]
A local file system manages storage space to provide a level of reliability and efficiency. Generally, it allocates storage device space in a granular manner, usually multiple physical units (i.e. [[bytes]]). For example, in [[Apple DOS]] of the early 1980s, 256-byte sectors on 140 kilobyte floppy disk used a ''track/sector map''.{{Citation needed|date=September 2012}}
The granular nature results in unused space, sometimes called [[slack space]], for each file except for those that have the rare size that is a multiple of the granular allocation.{{Sfn|Carrier|2005|pp=187–188}} For a 512-byte allocation, the average unused space is 256 bytes. For 64 KB clusters, the average unused space is 32 KB.
Generally, the allocation unit size is set when the storage is configured.
Choosing a relatively small size compared to the files stored, results in excessive access overhead.
Choosing a relatively large size results in excessive unused space.
Choosing an allocation size based on the average size of files expected to be in the storage tends to minimize unusable space.
=== Fragmentation ===
[[File:File system fragmentation.svg|thumb|File systems may become [[File system fragmentation|fragmented]]]]
As a file system creates, modifies and deletes files, the underlying storage representation may become [[File system fragmentation|fragmented]]. Files and the unused space between files will occupy allocation blocks that are not contiguous.
A file becomes fragmented if space needed to store its content cannot be allocated in contiguous blocks. Free space becomes fragmented when files are deleted.<ref>{{cite book|url=https://books.google.com/books?id=dSMJAAAAQBAJ&pg=PA524|title=Embedded Microcomputer Systems: Real Time Interfacing|edition=Third|last=Valvano|first=Jonathan W.|publisher=[[Cengage Learning]]|date=2011|access-date=June 30, 2022|page=524|isbn=978-1-111-42625-5}}</ref>
Fragmentation is invisible to the end user and the system still works correctly. However, this can degrade performance on some storage hardware that works better with contiguous blocks such as [[Hard disk drive#Performance characteristics|hard disk drives]]. Other hardware such as [[Solid-state drive|solid-state drives]] are not affected by fragmentation.
=== Access control ===
<!-- Too many 'see also' links, perhaps these should be moved to the 'See also' sect at the end -->
{{See also|Computer security|Password cracking|Filesystem-level encryption|Encrypting File System}}
A file system often supports access control of data that it manages.
The intent of access control is often to prevent certain users from reading or modifying certain files.
Access control can also restrict access by program in order to ensure that data is modified in a controlled way. Examples include passwords stored in the metadata of the file or elsewhere and [[file permissions]] in the form of permission bits, [[access control list]]s, or [[Capability-based security|capabilities]]. The need for file system utilities to be able to access the data at the media level to reorganize the structures and provide efficient backup usually means that these are only effective for polite users but are not effective against intruders.
<!-- Please don't make this article really big by including all the issues of file security here. Please add it to a file system security article -->
Methods for encrypting file data are sometimes included in the file system. This is very effective since there is no need for file system utilities to know the encryption seed to effectively manage the data. The risks of relying on encryption include the fact that an attacker can copy the data and use brute force to decrypt the data.
===
[[File:Btrfs qgroup screenshot.png|thumb|upright=1.5|Example of qgroup (quota group) of a [[btrfs]] filesystem]]
Some operating systems allow a system administrator to enable [[disk quota]]s to limit a user's use of storage space.
=== Data integrity ===
A file system typically ensures that stored data remains consistent in both normal operations as well as exceptional situations like:
* accessing program neglects to inform the file system that it has completed file access (to close a file)
* accessing program terminates abnormally (crashes)
* media failure
* loss of connection to remote systems
* operating system failure
* system reset ([[warm reboot|soft reboot]])
* power failure ([[hard reboot]])
Recovery from exceptional situations may include updating metadata, directory entries and handling data that was buffered but not written to storage media.
=== Recording ===
A file system might record events to allow analysis of issues such as:
* file or systemic problems and performance
* nefarious access
=== Data access ===
==== Byte stream access ====
Many file systems access data as a stream of [[bytes]]. Typically, to read file data, a program provides a [[memory buffer]] and the file system retrieves data from the medium and then writes the data to the buffer. A write involves the program providing a buffer of bytes that the file system reads and then stores to the medium.
==== Record access ====
Some file systems, or layers on top of a file system, allow a program to define a [[Record (computer science)|record]] so that a program can read and write data as a structure; not an unorganized sequence of bytes.
If a ''fixed length'' record definition is used, then locating the n<sup>th</sup> record can be calculated mathematically, which is relatively fast compared to parsing the data for record separators.
An identification for each record, also known as a key, allows a program to read, write and update records without regard to their ___location in storage. Such storage requires managing blocks of media, usually separating key blocks and data blocks. Efficient algorithms can be developed with pyramid structures for locating records.<ref>{{cite web|url=https://www.researchgate.net/publication/234789457|title=KSAM: A B + -tree-based keyed sequential-access method|work=ResearchGate|access-date=29 April 2016}}</ref>
=== Utilities ===
Typically, a file system can be managed by the user via various utility programs.
Some utilities allow the user to create, configure and remove an instance of a file system. It may allow extending or truncating the space allocated to the file system.
{{Anchor|DENTRY}}
Directory utilities may be used to create, rename and delete ''directory entries'', which are also known as ''dentries'' (singular: ''dentry''),<ref>{{cite book
| last1 = Mohan
| first1 = I. Chandra
| title = Operating Systems
| url = https://books.google.com/books?id=eei_jHVJi3oC
| ___location = Delhi
| publisher = PHI Learning Pvt. Ltd.
| date = 2013
| page = 166
| isbn = 9788120347267
| access-date = 2014-07-27
| quote = The word dentry is short for 'directory entry'. A dentry is nothing but a specific component in the path from the root. They (directory name or file name) provide for accessing files or directories[.]
}}</ref> and to alter metadata associated with a directory. Directory utilities may also include capabilities to create additional links to a directory ([[hard link]]s in [[Unix]]), to rename parent links (".." in [[Unix-like]] operating systems),{{Clarify|date=July 2014}} and to create bidirectional links to files.
File utilities create, list, copy, move and delete files, and alter metadata. They may be able to truncate data, truncate or extend space allocation, append to, move, and modify files in-place. Depending on the underlying structure of the file system, they may provide a mechanism to prepend to or truncate from the beginning of a file, insert entries into the middle of a file, or delete entries from a file. Utilities to free space for deleted files, if the file system provides an undelete function, also belong to this category.
Some file systems defer operations such as reorganization of free space, secure erasing of free space, and rebuilding of hierarchical structures by providing utilities to perform these functions at times of minimal activity. An example is the file system [[defragmentation]] utilities.
Some of the most important features of file system utilities are supervisory activities which may involve bypassing ownership or direct access to the underlying device. These include high-performance backup and recovery, data replication, and reorganization of various data structures and allocation tables within the file system.
=== File system API ===
Utilities, libraries and programs use [[file system API]]s to make requests of the file system. These include data transfer, positioning, updating metadata, managing directories, managing access specifications, and removal.
===
Frequently, retail systems are configured with a single file system occupying the entire [[Computer storage device|storage device]].
Another approach is to [[Disk partitioning|partition]] the disk so that several file systems with different attributes can be used. One file system, for use as browser cache or email storage, might be configured with a small allocation size. This
A third approach, which is mostly used in cloud systems, is to use "[[disk
Having multiple file systems on a single system has the additional benefit that in the event of a corruption of a single
==
=== Disk file systems ===
A ''disk file system'' takes advantages of the ability of disk storage media to randomly address data in a short amount of time. Additional considerations include the speed of accessing data following that initially requested and the anticipation that the following data may also be requested. This permits multiple users (or processes) access to various data on the disk without regard to the sequential ___location of the data. Examples include [[File Allocation Table|FAT]] ([[FAT12]], [[FAT16]], [[FAT32]]), [[exFAT]], [[NTFS]], [[ReFS]], [[Hierarchical File System (Apple)|HFS]] and [[HFS Plus|HFS+]], [[High Performance File System|HPFS]], [[Apple File System|APFS]], [[Unix File System|UFS]], [[ext2]], [[ext3]], [[ext4]], [[XFS]], [[btrfs]], [[Files-11]], [[Veritas File System]], [[VMFS]], [[ZFS]], [[ReiserFS]], [[Novell Storage Services|NSS]] and ScoutFS. Some disk file systems are [[journaling file system]]s or [[versioning file system]]s.
===
[[ISO 9660]] and [[Universal Disk Format]] (UDF) are two common formats that target [[Compact Disc]]s, [[DVD]]s and [[Blu-ray]] [[optical disc|discs]]. [[Mount Rainier (packet writing)|Mount Rainier]] is an extension to UDF supported since 2.6 series of the Linux kernel and since Windows Vista that facilitates rewriting to DVDs.
=== Flash file systems ===
{{Main|Flash file system}}
A ''flash file system'' considers the special abilities, performance and restrictions of [[flash memory]] devices. Frequently a disk file system can use a flash memory device as the underlying storage media, but it is much better to use a file system specifically designed for a flash device.<ref>{{cite book|chapter=18. Storage Alternatives for Mobile Computers|title=Mobile Computing|last1=Douglis|first1=Fred|last2=Cáceres|first2=Ramón|last3=Kaashoek|first3=M. Frans|author3-link=Frans Kaashoek|last4=Krishnan|first4=P.|last5=Li|first5=Kai|author6-link=Kai Li|last6=Marsh|first6=Brian|last7=Tauber|first7=Joshua|publisher=[[USENIX]]|date=1994|volume=353|pages=473–505|isbn=978-0-585-29603-6|doi=10.1007/978-0-585-29603-6_18|s2cid=2441760 }}</ref>
=== Tape file systems ===
A ''tape file system'' is a file system and tape format designed to store files on tape. [[Magnetic tape]]s are sequential storage media with significantly longer random data access times than disks, posing challenges to the creation and efficient management of a general-purpose file system.
In a disk file system there is typically a master file directory, and a map of used and free data regions. Any file additions, changes, or removals require updating the directory and the used/free maps. Random access to data regions is measured in milliseconds so this system works well for disks.
Line 157 ⟶ 244:
IBM has developed a file system for tape called the [[Linear Tape File System]]. The IBM implementation of this file system has been released as the open-source [[Linear Tape File System#IBM Linear Tape File System - Single Drive Edition|IBM Linear Tape File System — Single Drive Edition (LTFS-SDE)]] product. The Linear Tape File System uses a separate partition on the tape to record the index meta-data, thereby avoiding the problems associated with scattering directory entries across the entire tape.
==== Tape formatting ====
Writing data to a tape, erasing, or formatting a tape is often a significantly time-consuming process and can take several hours on large tapes.{{Efn|An LTO-6 2.5 TB tape requires more than 4 hours to write at 160 MB/Sec}} With many data tape technologies it is not necessary to format the tape before over-writing new data to the tape. This is due to the inherently destructive nature of overwriting data on sequential media.
Because of the time it can take to format a tape, typically tapes are pre-formatted so that the tape user does not need to spend time preparing each new tape for use. All that is usually necessary is to write an identifying media label to the tape before use, and even this can be automatically written by software when a new tape is used for the first time.
===Database file systems===
Another concept for file management is the idea of a database-based file system. Instead of, or in addition to, hierarchical structured management, files are identified by their characteristics, like type of file, topic, author, or similar [[metadata (computing)|rich metadata]].<ref>{{cite web|url=
IBM DB2 for i <ref>{{cite web|url=http://www-03.ibm.com/systems/i/software/db2/index.html |title=IBM DB2 for i: Overview |publisher=03.ibm.com |
Some other projects that
* Many [[Web content management system]]s use a [[Database management system|relational DBMS]] to store and retrieve files. For example, [[XHTML]] files are stored as [[XML]] or text fields, while image files are stored as blob fields; [[SQL]] SELECT (with optional [[XPath]]) statements retrieve the files, and allow the use of a sophisticated logic and more rich information associations than "usual file systems
* Very large file systems, embodied by applications like [[Apache Hadoop]] and [[Google File System]], use some ''database file system'' concepts.
===Transactional file systems===
Some programs need to
[[Transaction processing]] introduces
Windows, beginning with Vista, added transaction support to [[NTFS]], in a feature called [[Transactional NTFS]], but its use is now discouraged.<ref>{{cite web|url=http://msdn.microsoft.com/en-us/library/windows/desktop/hh802690(v=vs.85).aspx |title=Alternatives to using Transactional NTFS (Windows) |publisher=Msdn.microsoft.com |date=2013-12-05 |
Ensuring consistency across multiple file system operations is difficult, if not impossible, without file system transactions. [[File locking]] can be used as a [[concurrency control]] mechanism for individual files, but it typically does not protect the directory structure or file metadata. For instance, file locking cannot prevent [[TOCTTOU]] race conditions on symbolic links.
File locking also cannot automatically roll back a failed operation, such as a software upgrade; this requires atomicity.
[[Journaling file system]]s
Data backup systems typically do not provide support for direct backup of data stored in a transactional manner, which makes the recovery of reliable and consistent data sets difficult. Most backup software simply notes what files have changed since a certain time, regardless of the transactional state shared across multiple files in the overall dataset. As a workaround, some database systems simply produce an archived state file containing all data up to that point, and the backup software only backs that up and does not interact directly with the active transactional databases at all. Recovery requires separate recreation of the database from the state file
===Network file systems===
{{Main|Distributed file system}}
A ''network file system'' is a file system that acts as a client for a remote file access protocol, providing access to files on a server.
===Shared disk file systems===
{{Main|Shared disk file system}}
A ''shared disk file system'' is one in which a number of machines (usually servers) all have access to the same external disk subsystem (usually a
=== Special file systems ===
{{anchor|special file system}} Some file systems expose elements of the operating system as files so they can be acted on via the [[file system API]]. This is common in [[Unix-like]] operating systems, and to a lesser extent in other operating systems. Examples include:
{{anchor|device file system}}
* [[devfs]], [[udev]], [[TOPS-10]] expose I/O devices or pseudo-devices as special files
* [[configfs]] and [[sysfs]] expose special files that can be used to query and configure [[Linux]] kernel information
* [[procfs]] exposes process information as special files
===Minimal file system / audio-cassette storage===
When the system needed to write data, the user was notified to press "RECORD" on the cassette recorder, then press "RETURN" on the keyboard to notify the system that the cassette recorder was recording. The system wrote a sound to provide time synchronization, then [[Kansas City standard|modulated sounds]] that encoded a prefix, the data, a [[checksum]] and a suffix. When the system needed to read data, the user was instructed to press "PLAY" on the cassette recorder. The system would ''listen'' to the sounds on the tape waiting until a burst of sound could be recognized as the synchronization. The system would then interpret subsequent sounds as data. When the data read was complete, the system would notify the user to press "STOP" on the cassette recorder. It was primitive, but it
===Flat file systems===
<!-- linked from redirect [[Flat file system]] -->
{{distinguish|Flat file database}}
In a flat file system, there are no [[subdirectory|subdirectories]]; directory entries for all files are stored in a single directory.
When [[floppy disk]] media was first available this type of file system was adequate due to the relatively small amount of data space available. [[CP/M]] machines featured a flat file system, where files could be assigned to one of 16 ''user areas'' and generic file operations narrowed to work on one instead of defaulting to work on all of them. These user areas were no more than special attributes associated with the files
While simple, flat file systems become awkward as the number of files grows and makes it difficult to organize data into related groups of files.
Line 219 ⟶ 309:
A recent addition to the flat file system family is [[Amazon.com|Amazon]]'s [[Amazon S3|S3]], a remote storage service, which is intentionally simplistic to allow users the ability to customize how their data is stored. The only constructs are buckets (imagine a disk drive of unlimited size) and objects (similar, but not identical to the standard concept of a file). Advanced file management is allowed by being able to use nearly any character (including '/') in the object's name, and the ability to select subsets of the bucket's content based on identical prefixes.
== Implementations ==
An [[operating system]] (OS) typically supports one or more file systems. Sometimes an OS and its file system are so tightly interwoven that it is difficult to describe them independently.
An OS typically provides file system access to the user. Often an OS provides [[command line interface]], such as [[Unix shell]], Windows [[Command Prompt]] and [[PowerShell]], and [[DIGITAL Command Language|OpenVMS DCL]]. An OS often also provides [[graphical user interface]] [[file browser]]s such as MacOS [[Finder (software)|Finder]] and Windows [[File Explorer]].
===Unix and Unix-like operating systems===
[[Unix-like]] operating systems create a virtual file system, which makes all the files on all the devices appear to exist in a single hierarchy. This means, in those systems, there is one [[root directory]], and every file existing on the system is located under it somewhere. Unix-like systems can use a [[RAM disk]] or network shared resource as its root directory.
Unix-like systems assign a device name to each device, but this is not how the files on that device are accessed. Instead, to gain access to files on another device, the operating system must first be informed where in the directory tree those files should appear. This process is called [[mount (computing)|mounting]] a file system. For example, to access the files on a [[CD-ROM]], one must tell the operating system "Take the file system from this CD-ROM and make it appear under such-and-such directory
[[Unix-like]] operating systems often include software and tools that assist in the mounting process and provide it new functionality. Some of these strategies have been coined "auto-mounting" as a reflection of their purpose.
* In many situations, file systems other than the root need to be available as soon as the operating system has [[booting|booted]]. All Unix-like systems therefore provide a facility for mounting file systems at boot time. [[System administrator]]s define these file systems in the configuration file [[fstab]] (''vfstab'' in [[Solaris (operating system)|Solaris]]), which also indicates options and mount points.
* In some situations, there is no need to mount certain file systems at [[booting|boot time]], although their use may be desired thereafter. There are some utilities for Unix-like systems that allow the mounting of predefined file systems upon demand.
* Removable media
<!-- supermount definition may be inaccurate -->
<!-- there may be some concepts I {forgot, omitted, did not know, am not creative enough to invent} -->
* Progressive Unix-like systems have also introduced a concept called '''supermounting'''; see, for example, [
* An [[automounter]] will automatically mount a file system when a reference is made to the directory atop which it should be mounted. This is usually used for file systems on network servers, rather than relying on events such as the insertion of media, as would be appropriate for removable media.
===={{Anchor|LINUX}}Linux====
[[Linux]] supports
====Solaris====
[[Solaris (operating system)|Solaris]] in earlier releases defaulted to (non-journaled or non-logging) [[Unix File System|UFS]] for bootable and supplementary file systems. Solaris defaulted to, supported, and extended UFS.
Support for other file systems and significant enhancements were added over time, including [[Veritas Software]] Corp. (
Kernel extensions were added to Solaris to allow for bootable Veritas [[VxFS]] operation. Logging or [[Journaling file system|
[[Logical Volume Management]] allows for spanning a file system across multiple devices for the purpose of adding redundancy, capacity, and/or throughput. Legacy environments in Solaris may use [[Solaris Volume Manager]] (formerly known as [[Solstice DiskSuite]]). Multiple operating systems (including Solaris) may use [[Veritas Volume Manager]]. Modern Solaris based operating systems eclipse the need for
====
[[macOS|macOS (formerly Mac OS X)]] uses the [[Apple File System]] (APFS), which in 2017 replaced a file system inherited from
HFS Plus has three kinds of links: Unix-style [[hard link]]s, Unix-style [[symbolic link]]s, and [[alias (Mac OS)|aliases]]. Aliases are designed to maintain a link to their original file even if they are moved or renamed; they are not interpreted by the file system itself, but by the File Manager code in [[userland (computing)|userland]].
macOS 10.13 High Sierra, which was announced on June 5, 2017, at Apple's WWDC event, uses the [[Apple File System]] on [[solid-state drive]]s.
macOS also supported the [[Unix File System|UFS]] file system, derived from the [[BSD]] Unix Fast File System via [[NeXTSTEP]]. However, as of [[Mac OS X Leopard]], macOS could no longer be installed on a UFS volume, nor can a pre-Leopard system installed on a UFS volume be upgraded to Leopard.<ref>{{cite web|url=http://docs.info.apple.com/article.html?artnum=306516|title=Mac OS X 10.5 Leopard: Installing on a UFS-formatted volume|work=apple.com|date=19 October 2007|access-date=29 April 2016|archive-url=https://web.archive.org/web/20080316033439/http://docs.info.apple.com/article.html?artnum=306516|archive-date=16 March 2008|url-status=dead}}</ref> As of [[Mac OS X Lion]] UFS support was completely dropped.
Newer versions of macOS are capable of reading and writing to the legacy [[File Allocation Table|FAT]] file systems (16 and 32) common on Windows. They are also capable of ''reading'' the newer [[NTFS]] file systems for Windows. In order to ''write'' to NTFS file systems on macOS versions prior to [[Mac OS X Snow Leopard]] third-party software is necessary. Mac OS X 10.6 (Snow Leopard) and later allow writing to NTFS file systems, but only after a non-trivial system setting change (third-party software exists that automates this).<ref>{{cite web|last=OSXDaily|title=How to Enable NTFS Write Support in Mac OS X|url=http://osxdaily.com/2013/10/02/enable-ntfs-write-support-mac-os-x/|access-date=6 February 2014|date=2013-10-02}}</ref>
Finally, macOS supports reading and writing of the [[exFAT]] file system since Mac OS X Snow Leopard, starting from version 10.6.5.<ref name="encase-book">{{cite book|url=https://books.google.com/books?id=c1mezk6uHfIC&q=os+x+exfat+10.6.5&pg=PA79 |title=EnCase Computer Forensics - The Official EnCE: EnCase Certified Examiner |author=Steve Bunting |date=2012-08-14 |publisher=Wiley |access-date=2014-02-07|isbn=9781118219409 }}</ref>
===OS/2===
[[OS/2]] 1.2 introduced the [[High Performance File System]] (HPFS). HPFS supports mixed case file names in different [[code page]]s, long file names (255 characters), more efficient use of disk space, an architecture that keeps related items close to each other on the disk volume, less fragmentation of data, [[Extent (file systems)|extent-based]] space allocation, a [[B+ tree]] structure for directories, and the root directory located at the midpoint of the disk, for faster average access. A [[JFS (file system)|journaled filesystem]] (JFS) was shipped in 1999.
===PC-BSD===
Line 267 ⟶ 363:
===Plan 9===
[[Plan 9 from Bell Labs]] treats everything as a file and accesses all objects as a file would be accessed (i.e., there is no [[ioctl]] or [[mmap]]): networking, graphics, debugging, authentication, capabilities, encryption, and other services are accessed via I/O operations on [[file descriptor]]s.
The [[Inferno (operating system)|Inferno]] operating system shares these concepts with Plan 9.
Line 273 ⟶ 369:
===Microsoft Windows===
[[File:Dir command in Windows Command Prompt.png|thumb|right|300px|Directory listing in a [[Microsoft Windows|Windows]] command shell]]
Windows makes use of the [[File Allocation Table|FAT]], [[NTFS]], [[exFAT]], [[Live File System]] and [[ReFS]] file systems (the last of these is only supported and usable in [[Windows Server 2012]], [[Windows Server 2016]], [[Windows 8]], [[Windows 8.1]], and [[Windows 10]]; Windows cannot boot from it).
Windows uses a ''[[drive letter]]'' abstraction at the user level to distinguish one disk or partition from another. For example, the [[path (computing)|path]]
====FAT====
{{Main|File Allocation Table}}
The family of [[File Allocation Table|FAT]] file systems is supported by almost all operating systems for personal computers, including all versions of [[Microsoft Windows|Windows]] and [[MS-DOS]]/[[PC DOS]], [[OS/2]], and [[DR-DOS]]. (PC DOS is an OEM version of MS-DOS, MS-DOS was originally based on [[Seattle Computer Products|SCP]]'s [[86-DOS]]. DR-DOS was based on [[Digital Research]]'s [[Concurrent DOS]], a successor of [[CP/M-86]].) The FAT file systems are therefore well-suited as a universal exchange format between computers and devices of most any type and age.
The FAT file system traces its roots back to an (incompatible) 8-bit FAT precursor in [[Standalone Disk BASIC]] and the short-lived [[MIDAS (operating system)|MDOS/MIDAS]] project.{{Citation needed|date=September 2012}}
Line 300 ⟶ 396:
{{Main|exFAT}}
[[exFAT]]
exFAT is not backward compatible with FAT file systems such as FAT12, FAT16 or FAT32. The file system is supported with newer Windows systems, such as Windows XP, Windows Server 2003, Windows Vista, Windows 2008, Windows 7, Windows 8,
exFAT is supported in
===OpenVMS===
{{Main|Files-11}}
===MVS
{{Main|MVS#MVS filesystem}}
Prior to the introduction of [[VSAM]], [[OS/360]] systems implemented a hybrid file system. The system was designed to easily support [[Removable media|removable disk packs]], so the information relating to all files on one disk (''volume'' in IBM terminology) is stored on that disk in a [[Flat file database|flat system file]] called the ''[[Volume Table of Contents]]'' (VTOC). The VTOC stores all metadata for the file. Later a hierarchical directory structure was imposed with the introduction of the ''System Catalog'', which can optionally catalog files (datasets) on resident and removable volumes. The catalog only contains information to relate a dataset to a specific volume. If the user requests access to a dataset on an offline volume, and they have suitable privileges, the system will attempt to mount the required volume. Cataloged and non-cataloged datasets can still be accessed using information in the VTOC, bypassing the catalog, if the required volume id is provided to the OPEN request. Still later the VTOC was indexed to speed up access.
===Conversational Monitor System===
{{Main|CMS file system}}
The IBM [[Conversational Monitor System]] (CMS) component of [[VM/370]] uses a separate flat file system for each [[virtual disk]] (''minidisk''). File data and control information are scattered and intermixed. The anchor is a record called the ''Master File Directory'' (MFD), always located in the fourth block on the disk. Originally CMS used fixed-length 800-byte blocks, but later versions used larger size blocks up to 4K. Access to a data record requires two levels of [[indirection]], where the file's directory entry (called a ''File Status Table'' (FST) entry) points to blocks containing a list of addresses of the individual records.
===AS/400 file system===
Data on the AS/400 and its successors consists of system objects mapped into the system virtual address space in a [[single-level store]]. Many types of [[
===Other file systems===
* The Prospero File System is a file system based on the Virtual System Model.<ref>
* [[Flex machine#RSRE FLEX Computer System|RSRE FLEX file system]] - written in [[ALGOL 68]]
* The file system of the [[Michigan Terminal System]] (MTS) is interesting because: (i) it provides "line files" where record lengths and line numbers are associated as metadata with each record in the file, lines can be added, replaced, updated with the same or different length records, and deleted anywhere in the file without the need to read and rewrite the entire file; (ii) using program keys files may be shared or permitted to commands and programs in addition to users and groups; and (iii) there is a comprehensive file locking mechanism that protects both the file's data and its metadata.<ref>
* [[TempleOS]] uses RedSea, a file system made by Terry A. Davis.<ref name="00yK1">{{Cite web|last=Davis|first=Terry A.|date=n.d.|url=http://www.templeos.org/Wb/Doc/Features.html#l1|title=The Temple Operating System|website=www.templeos.org|access-date=March 30, 2017|url-status=dead|archive-url=https://web.archive.org/web/20170331120502/http://www.templeos.org/Wb/Doc/Features.html#l1|archive-date=March 31, 2017}}</ref>
== Limitations ==
=== Design limitations ===
File systems limit [[Comparison of file systems#Limits|storable data capacity]] {{endash}} generally driven by the typical size of storage devices at the time the file system is designed and anticipated into the foreseeable future.
Since storage sizes have increased at near [[exponential growth|exponential]] rate (see [[Moore's law]]), newer storage devices often exceed existing file system limits within only a few years after introduction. This requires new file systems with ever increasing capacity.
With higher capacity, the need for capabilities and therefore complexity increases as well. File system complexity typically varies proportionally with available storage capacity. Capacity issues aside, the file systems of early 1980s [[home computer]]s with 50 KB to 512 KB of storage would not be a reasonable choice for modern storage systems with hundreds of gigabytes of capacity. Likewise, modern file systems would not be a reasonable choice for these early systems, since the complexity of modern file system structures would quickly consume the limited capacity of early storage systems.
===Converting the type of a file system===
It may be advantageous or necessary to have files in a different file system than they currently exist. Reasons include the need for an increase in the space requirements beyond the limits of the current file system. The depth of path may need to be increased beyond the restrictions of the file system. There may be performance or reliability considerations. Providing access to another operating system which does not support the existing file system is another reason.
====In-place conversion====
In some cases conversion can be done in-place, although migrating the file system is more conservative, as it involves a creating a copy of the data and is recommended.<ref name="ms">
====Migrating to a different file system====
Migration has the disadvantage of requiring additional space although it may be faster. The best case is if there is unused space on media which will contain the final file system.
For example, to migrate a FAT32 file system to an ext2 file system
An alternative, when there is not sufficient space to retain the original file system until the new one is created, is to use a work area (such as a removable media). This takes longer but
===Long file paths and long file names===
In [[hierarchical file
Copying files with long names or located in paths of significant depth from one file system to another may cause undesirable results. This depends on how the utility doing the copying handles the discrepancy.
==See also==
{{Div col|colwidth=25em}}
* [[Comparison of file systems]]
* [[Computer data storage]]
* [[Disk quota]]
* [[List of file systems]]
* [[List of Unix
* [[Directory structure]]
* [[
* [[Distributed file system]]
* [[Distributed Data Management Architecture]]
Line 354 ⟶ 464:
* [[File system fragmentation]]
* [[Filename extension]]
* [[Global
* [[Object storage]]
* [[Storage efficiency]]
* [[Synthetic file system]]
* [[Virtual file system]]
{{
==Notes==
Line 368 ⟶ 478:
<!--See http://en.wikipedia.org/wiki/Wikipedia:Footnotes
for an explanation of how to generate footnotes using the <ref(erences/)> tags-->
{{Reflist|30em}}
===Sources===
{{Refbegin|30em}}
* {{cite web
access-date=February 9, 2005|
url=http://jdebp.eu/FGA/os2-disc-and-volume-size-limits.html|
author-first=Jonathan|
author-last=de Boyne Pollard|
title=Disc and volume size limits|
work=Frequently Given Answers|
year=1996}} * {{Cite FTP |
access-date=February 9, 2005|
url=ftp://service.boulder.ibm.com/ps/products/os2/fixes/v4warp/english-us/jr09427/JR09427.TXT| server=IBM| url-status=dead|
title=OS/2 corrective service fix JR09427}}
* {{cite web|
url=
title=Attribute - $EA_INFORMATION (0xD0)|
work=NTFS Information, Linux-NTFS Project}}
* {{cite web
access-date=February 9, 2005|
url=https://linux-ntfs.sourceforge.net/ntfs/attributes/ea.html|
title=Attribute - $EA (0xE0)|
work=NTFS Information, Linux-NTFS Project}}
* {{cite web
access-date=February 21, 2005|
url=https://linux-ntfs.sourceforge.net/ntfs/attributes/standard_information.html|
title=Attribute - $STANDARD_INFORMATION (0x10)|
work=NTFS Information, Linux-NTFS Project}}
* {{cite web|
*
{{Refend}}
Line 408 ⟶ 521:
* {{cite book|first1=Remzi H.|last1=Arpaci-Dusseau|first2=Andrea C.|last2=Arpaci-Dusseau|title=Operating Systems: Three Easy Pieces|publisher=Arpaci-Dusseau Books|year=2014|url=http://www.ostep.org}}
* {{cite book|first=Brian|last=Carrier|title=File System Forensic Analysis|publisher=[[Addison-Wesley]]|year=2005|isbn=0-321-26817-2|url=http://www.digital-evidence.org/fsfa/}}
* {{cite book|first=Helen|last=Custer|title=Inside the Windows NT File System|publisher=[[Microsoft Press]]|year=1994|isbn=1-55615-660-X|url-access=registration|url=https://archive.org/details/insidewindowsntf00cust}}
* {{cite book|first=Dominic|last=Giampaolo|title=Practical File System Design with the Be File System|publisher=Morgan Kaufmann Publishers|year=1999
* {{cite book|first=Kirby|last=McCoy|title=VMS File System Internals|series=VAX - VMS Series|publisher=[[Digital Press]]|year=1990|isbn=1-55558-056-4}}
* {{cite book|first=Stan|last=Mitchell|title=Inside the Windows 95 File System|publisher=[[O'Reilly Media|O'Reilly]]|year=1997|isbn=1-56592-200-X|url=http://oreilly.com/catalog/156592200X}}
* {{cite book|first=Rajeev|last=Nagar|title=Windows NT File System Internals : A Developer's Guide|publisher=[[O'Reilly Media|O'Reilly]]|year=1997|isbn=978-1-56592-249-5|url=
* {{cite book|first=Steve D.|last=Pate|title=UNIX Filesystems: Evolution, Design, and Implementation|publisher=[[John Wiley & Sons|Wiley]]|year=2003|isbn=0-471-16483-6|url=http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471164836.html|archive-date=2013-11-24|access-date=2010-10-17|archive-url=https://web.archive.org/web/20131124021318/http://eu.wiley.com/WileyCDA/WileyTitle/productCd-0471164836.html|url-status=dead}}
* {{cite book|first=Mendel|last=Rosenblum|title=The Design and Implementation of a Log-Structured File System|series=The Springer International Series in Engineering and Computer Science|publisher=Springer|year=1994|isbn=0-7923-9541-7}}
* {{cite book|first1=Mark|last1=Russinovich|first2=David A.|last2=Solomon|first3=Alex|last3=Ionescu|chapter=File Systems|title=Windows Internals|edition=5th|publisher=[[Microsoft Press]]|year=2009|isbn=978-0-7356-2530-
* Prabhakaran, Vijayan (2006). [http://www.cs.wisc.edu/~vijayan/vijayan-thesis.pdf ''IRON File Systems'']. PhD dissertation, University of Wisconsin-Madison.
* {{cite book|first1=Abraham|last1=Silberschatz|first2=Peter Baer|last2=Galvin|first3=Greg|last3=Gagne|chapter=Storage Management|title=Operating System Concepts|edition=7th|publisher=Wiley|year=2004|isbn=0-471-69466-5}}
* {{cite book|first=Andrew S.|last=Tanenbaum|
* {{cite book|first1=Andrew S.|last1=Tanenbaum|
{{Refend}}
Line 426 ⟶ 539:
* [http://linuxgazette.net/102/piszcz.html Benchmarking Filesystems (outdated)] by Justin Piszcz, Linux Gazette 102, May 2004
* [http://linuxgazette.net/122/piszcz.html Benchmarking Filesystems Part II] using kernel 2.6, by Justin Piszcz, Linux Gazette 122, January 2006
* [http://www.debian-administration.org/articles/388 Filesystems (ext3, ReiserFS, XFS, JFS) comparison on Debian Etch] {{Webarchive|url=https://web.archive.org/web/20080913112251/http://www.debian-administration.org/articles/388 |date=2008-09-13 }} 2006
* [http://www.osnews.com/story.php?news_id=69 Interview With the People Behind JFS, ReiserFS & XFS]
* [https://web.archive.org/web/20060313074847/http://www.open-mag.com/features/Vol_18/filesystems/filesystems.htm Journal File System Performance (outdated)]: ReiserFS, JFS, and Ext3FS show their merits on a fast RAID appliance
* [https://web.archive.org/web/20051125112444/http://staff.osuosl.org/~kveton/fs/ Journaled Filesystem Benchmarks (outdated)]: A comparison of ReiserFS, XFS, JFS, ext3 & ext2
* [http://www.osdata.com/system/logical/logical.htm Large List of File System Summaries (most recent update 2006-11-19)]
* [https://web.archive.org/web/20190503084749/http://fsbench.netnation.com/ Linux File System Benchmarks] v2.6 kernel with a stress on CPU usage
*
* [http://www.suse.de/~aj/linux_lfs.html Linux large file support (outdated)]
* [http://www.microsoft.com/whdc/device/storage/LocFileSys.mspx Local Filesystems for Windows]
* [https://web.archive.org/web/20060819215504/http://osdev.berlios.de/osd-fs.html Overview of some filesystems (outdated)]
* [http://www.lrdev.com/lr/unix/sparsefile.html Sparse files support (outdated)]
* {{cite news |title=From BFS to ZFS: past, present, and future of file systems |url=
{{Refend}}
Line 443 ⟶ 556:
{{Wikibooks|Guide to Unix|Explanations/Filesystems and Swap|Filesystems and Swap}}
{{Commons category|File systems}}
*
* [http://filesystems.org/all-projects.html Interesting File System Projects]
{{Computer files}}
{{File systems}}
{{Operating system}}
{{DEFAULTSORT:File System}}
| |||