File system

The file system ( English file system or filesystem ) is a storage organization on a volume such as a data carrier of a computer . Files can be saved, read, changed or deleted. For the user, the file name and internal computer file addresses must be reconciled. Easy retrieval and safe storage are essential tasks of a file system. The order and access system takes the device properties into account and is an elementary component of an operating system .

term

The term “file system” can refer to the entire higher-level directory tree, the directory structure , on the one hand, and to individually integrable file systems, such as partitions , on the other . Since a file system is often used per partition or volume , the term “file system” is often used as a synonym for “partition” - in fact, however, the file system is the content and the partition is only the framework in which the storage space is available is provided.

More than one file system is supported on most operating systems. Each file system must be accommodated on a separately addressable logical volume , such as a partition or an additional data carrier such as a hard drive. The initialization of this logically separate data memory is called formatting . The content of the file system is made accessible by mounting, embedding or English mounting in the running system.

history

Historically, the first punched tape (on film, later on paper) and punched card files are file systems. Like magnetic tape storage devices, they form linear file systems. The drum and hard disk memories, which were later developed for mass storage and fast access, then enabled more complex file systems for the first time through random access to any position in the file system. These file systems offer the option of accessing a file by name. The concept of the file system was abstracted to such an extent that access to files in the network and to devices that are managed virtually as files can also be carried out via file systems. Application programs are thus able to access these different data sources via a standardized interface .

properties

Files in a file system almost always have at least one file name and attributes that provide more detailed information about the file. The file names are stored in directories ; Directories are usually special files. A file name (and thus a file) and the data belonging to the file can be found by the system via such directories . A file system thus forms a namespace . All files (or file-like objects) can be called up via a unique address (file name including path or URI ) - within the file system. The name of a file and other information associated with the saved files are called metadata .

There are specialized file systems for different data carriers (such as magnetic tape , hard disk drives , optical data carriers ( CD , DVD , ...), flash memory , ...).

The file system represents a specific layer of the operating system: All layers above it (rest of the operating system, applications) can access files abstractly using their plain text names. Only with the file system are these abstract information converted into physical addresses ( block number , track, sector, etc.) on the storage medium. In the layer below this, the file system driver communicates with the respective device driver and indirectly with the firmware of the storage system ( drive ). The latter takes on additional organizational tasks, for example the transparent replacement of faulty blocks with reserve blocks.

Organization of mass storage

Mass storage devices such as hard drives, CD-ROMs , and floppy drives are typically block-structured; H. From the operating system perspective, data can only be read or written as a sequence of entire data blocks . A storage device presents the storage medium to the operating system only as a large linear arrangement of many numbered (and above addressable) blocks.

Today a block usually contains 512 (= 2 ⁹ ) or 4096 (= 2 ¹² ) bytes, on optical media (CD-ROM, DVD-ROM) 2048 (= 2 ¹¹ ) bytes. Modern operating systems combine several blocks into a cluster of a fixed size for performance and administrative reasons . Today, clusters with eight or more blocks are common, i.e. 4096 bytes per cluster. The cluster size is generally a power of two (1024, 2048, 4096, 8192, ...).

A file is a defined section of data storage that consists of one or more clusters on the device. Each file is also given a description structure, which can contain the actual size, references to the clusters used and possibly other information such as file type , owner, access rights (metadata).

There are several options for assigning clusters to files.

• The reference of a file consists of the cluster number of the starting cluster and the number of clusters that follow (physically sequential). Disadvantages: when enlarging, the whole file may have to be moved. This complicates file handling and leads to poor performance with many large files. It can happen that a file cannot be saved although there is still enough free space on the data carrier .
• The reference of a file consists of the first cluster number. The cluster number of the following cluster is stored in each cluster of the file. The result is a linked list . Disadvantages: If you do not want to read the file sequentially, for example only the end, the operating system still has to read the entire file to find the end.
• Free assignment of file clusters to subsequent clusters using a table on the mass storage device (example: FAT ). Disadvantages: very large description structure, sequential reading or writing a little slower than ideal, since assignment information is neither bundled nor available with the data.
• Storage of an array of tuples (extent start cluster, extent length) in the description structure of the file. An extent is a sequence of sequential clusters. Today this is implemented in many file systems.

Directories contain file names and references to the respective description structures. Since directories are also storage areas, specially marked files are usually used as directories. The first description structure can contain the output directory.

Another separate area on the storage medium is used to keep records of which blocks or clusters are already occupied and which are still free. A means often used for this is the Block Availability Map (BAM), in which a memory bit is created for each block that shows whether the block is occupied or free. In principle, the BAM contains redundant information and serves the efficiency of administration; If the information stored there is lost, the BAM can be recreated.

In general, the first block is reserved for a so-called boot sector (e.g. master boot record ) that can be used to start up the system. On a storage medium with several partitions , the partition table , which contains organizational data on the partitions, is typically located immediately afterwards . Neither the boot block nor the partition table are part of the actual file system.

Each partition usually contains its own file system that is independent of the data on other partitions; the explanations above relate to the individual partitions which are attached to the partition table one after the other.

boat	Description structures	List of free clusters	Cluster of files and directories
Example of the division of a mass storage device for a simple file system

For reasons of efficiency, i.e. above all to increase the performance / access speed, various strategies have been developed as to how these organizational structures are arranged within the available memory area. Since, for example, there can be any number of subdirectories in many file systems, it is forbidden to reserve fixed spaces for these directory structures from the outset; everything must be organized dynamically. There are also file systems like some from Commodore , which arrange the basic organizational structures such as the root directory and BAM in the middle of the memory area (instead of at the beginning, as is usually the case with others), so that the path that the read / write head takes from there to the actual Data and travel back has to be reduced on average. In general, it can be a strategic approach to physically arrange the actual data and its organizational data as close together as possible.

Access to mass storage

A program accesses the mass storage via the file system. System calls are made available for this under Unix and similar operating systems . The most important system calls are here:

• System calls for directories:
  • mkdir, rmdir
    Creating and deleting a directory
  • opendir, closedir
    Opening and closing a directory
  • readdir
    Reading directory entries
  • chdir
    Change to another directory

• System calls for files:
  • creat, unlink
    Create and delete a file
  • open, close
    Opening and closing a file
  • read, write
    Reading and writing
  • seek
    Repositioning the read / write pointer

In addition, the operating system offers management functions , for example for renaming, copying and moving, creating a file system on a new data carrier, for consistency checks, compression or backup (different depending on the operating system and file system).

The implementation of the system calls of a program is often implemented by the kernel of an operating system and differs between the various file systems. The kernel then translates the accesses into the block operations of the respective mass storage device . (Note: In fact, this only applies to so-called monolithic kernels . On the other hand, systems built on a microkernel or hybrid kernel are designed in such a way that the file system operations do not have to be performed by the kernel itself.)

When a program opens a file using open , the file name is searched for in the directory. The operating system determines the blocks on the mass storage device from the corresponding description structures. If a file is found in the directory, the operating system also receives its description structure and thus the references to the associated clusters and uses this to get to the specific blocks.

With read the program can then access the clusters of the file (and thus the blocks on the mass storage device). If a file becomes larger due to write , a new cluster is taken from the free list if necessary and added to the description structure of the file. The other system calls can also be translated into cluster or block access.

Types of file systems

Linear file systems

The historically first file systems were linear file systems on punched tape or punched card, as well as the magnetic tape systems still used today for backing up data.

Hierarchical file systems

Early file systems ( CP / M , Apple DOS , Commodore DOS ) only had a single directory that contained references to all files on the mass storage device. As the capacity of the storage media grew, it became more and more difficult to keep track of hundreds and thousands of files, so the concept of subdirectories was introduced. A hierarchical file system was developed for the Multics operating system and, after its development was discontinued, was adopted by AT&T Unix Version 1 from 1971. This laid the foundation for most modern file systems, which in the root directory can contain not only regular files but also references to other directories, the subdirectories, with possibly further subdirectories.

This creates a directory structure that is often represented as a directory tree. The hard disk drive C: under Windows , for example, contains files such as boot.ini and ntldr as well as directories such as programs , documents and settings , etc. A directory such as My Documents can then contain subdirectories such as My Pictures or Text . The normal files Brief1.txt and Brief2.txt can then be found in texts .

The directories are also called folders and are, depending on the operating system by backslash (English backslash ) "\" (DOS, Windows, TOS ), slash (English slash ) "/" ( Unix , Linux , macOS , AmigaOS ), point Separate “.” ( OpenVMS ) or colon “:” (older Mac OS versions). Since there is a hierarchy of directories and files, this is called hierarchical file systems. The path through the file system, indicated by directory names separated by separators, is called a path. The Brief1.txt file can be accessed with

• C:\Dokumente und Einstellungen\benutzername\Eigene Dateien\Texte\Brief1.txt (Windows 2000 / XP)
• C:\Users\benutzername\Eigene Dokumente\Brief1.txt (From Windows Vista)
• /Users/benutzername/Texte/Brief1.txt (Mac OS X)
• Macintosh HD:Dokumente:Texte:Brief 1 (Classic Mac OS)
• /home/benutzername/Texte/Brief1.txt (Unix / Linux)
• Laufwerksname:verzeichnis/unterverzeichnis/Brief1.txt (AmigaOS)
• DISK$Laufwerksname:[USERS.benutzername]Brief1.TXT;1 (OpenVMS)

can be accessed. In DOS / Windows there are drive letters followed by a colon, which are placed in front of the paths within the file system. Each data carrier has its own letter, for example usually C: for the first partition of the first hard disk. With Unix there are no drive letters, just a single directory tree. The individual disks are there at certain locations in the tree hooked (command mount ), so that all the disks together result in the aggregate. Windows variants based on Windows NT also work internally with such a tree, but this tree is hidden from the user.

Under AmigaOS a mixture of approaches from DOS and Unix is done. The drives named according to Unix nomenclature are addressed with a colon (df0 :, hda1 :, sda2 :). In addition, logical colon drive names can LIBS:be assigned ASSIGNindependently of the physical volume, as with.

The directory paths of OpenVMS differ greatly from Unix, DOS and Windows paths. First, OpenVMS names the device type, e.g. B. “ DISK$” refers to a local volume. The drive name (up to 255 characters long) is appended and terminated with a colon. The directory part is put in square brackets. The subdirectories are separated by periods, e.g. B. " [USERS.Verzeichnis.Verzeichnis2]". The file name follows at the end of the path, for example " Brief1.TXT;1". The first part is a descriptive name and up to 39 characters long. A period is followed by the three-digit file type, similar to Windows. At the end, the version of the file is added, separated by a semicolon ";".

The term file system often denotes not only the structure and the way in which the data is organized on a data carrier, but also generally the whole tree with several different file systems (hard disk, CD-ROM, ...). To be correct, one would have to speak of a namespace that is formed by different partial namespaces (the file systems of the integrated data carriers), but since this namespace is very file-related, it is often only referred to as the file system.

Network file systems

The system calls such as open , read etc. can also be transmitted to a server via a network . This then accesses its mass storage device and sends the requested information back to the client .

Since the same system calls are used, access from the program and user point of view does not differ from that of the local devices. This is called transparent access because the user does not see the redirection to the other computer, but rather looks directly at the disk of the remote computer - as if through a transparent pane of glass. Special network protocols are available for network file systems.

If a file system, for example in a Storage Area Network (SAN), can be accessed directly by several systems in parallel, one speaks of a global or cluster file system . Here, additional measures must be taken to data loss ( English data corruption ) to avoid by overwriting each other. A metadata server is used for this. All systems forward the metadata access - typically via a LAN - to the metadata server, which carries out these operations such as directory access and block or cluster assignments. The actual data access then takes place via the SAN, as if the file system were connected locally. Since the additional effort ( overhead ) caused by the transfer to the metadata server is hardly significant, especially with large files, a transfer speed similar to that of a directly connected file system can be achieved.

The WebDAV protocol is a special feature, which enables file system access to remote files via HTTP .

Special virtual file systems

The open - read model can also be applied to devices and objects that are normally not addressed via file systems. This makes access to these objects identical to access to normal files, which corresponds to the Unix concept Everything is a file and thus has the advantage of being able to use this data in the same way as configuration files.

With the current Linux kernels (including version 2.6), system and process information can be queried and changed via the virtual proc file system. The virtual file /proc/cpuinfoprovides information about the processor, for example. There are some such pseudo-file systems in Linux. These include u. a. sysfs , usbfs or devpts ; under some BSDs there is a kernfs . All of these file systems only contain purely virtual files with information or devices that are mapped to a "file".

The kernel simulates the existence of a file as it could also exist on a mass storage device.

In contrast, files in ramfs or tmpfs and similar file systems actually exist, but are only kept in the main memory. They are used during the boot phase for speed reasons and for logical and technical reasons .

In addition to Linux, there are also so-called RAM disks for various other operating systems , with which a complete virtual drive is implemented in the main memory, mainly for reasons of speed.

particularities

Many modern file systems have generalized the principle of the file so that you can save not just a sequence of bytes, a so-called stream , but several such sequences ( alternative data streams ) in a file . This makes it possible to edit parts of a file without having to move any other parts that may be very large.

The lack of support for multiple streams is problematic. On the one hand, this manifests itself in the fact that alternative data are lost without warning when transferring to other file systems (ISO 9660, FAT, ext2), and on the other hand in the fact that hardly any tool supports them, which is why the data stored there cannot be easily viewed and virus scanners, for example, overlook viruses stored there.

The fact that the main data stream is not affected by changes to the other streams has advantages for performance, space requirements and data security.

Sparse files , hard links and symbolic links are not only possible under Inode- based file systems .

For mass storage media such as CD-ROM or DVD there are separate file systems that are used across operating systems, especially ISO 9660 , for more see below under Others .

File systems from the Unix area know special device files . Their names are often determined by convention; they can be renamed as desired; For example, as also the keyboard, mouse, and other interfaces specific file name, in this case open , read , write can be accessed, even the memory has a file name /dev/mem. (The Unix philosophy is: " Everything is a file , and if not, it should be a file.")

In other operating systems (such as under MS-DOS version 2.0 or later), there are also special files: COM:, CON:, LPT:, PRN:and others. These devices can be opened in the same way as a file and read and written to using an access number (handle) . Understandably, they don't have a file pointer. In contrast to the block devices (also called "Drives": A:, B:, C:etc.) contain it no files, but behave yourself - with certain restrictions - such as files. These pseudo files have existed since PC DOS 2.0 or MS-DOS 2.0, which was heavily influenced by UNIX. Taking into account the DOS device driver specification, the user is able to write his own device drivers, load them with the DEVICE command and address them using the same pseudo file names. In the past, these special file names were often the cause of security problems, as some applications were not aware of the corresponding names and were therefore not filtered out, but also because the access protection to the associated devices was insufficiently regulated.

In addition, there are file systems that can span multiple underlying storage media (" volumes ") (e.g. the ZFS and Btrfs file systems ), which inherently enable versioning of files (e.g. VMS ) or whose size can be changed at runtime can (e.g. AIX ).

Some file systems offer encryption functions, the scope and security of the functions vary.

Associative file management

These are often incorrectly referred to as database file systems or SQL file systems; these are actually not file systems, but rather information from a file system that is stored in an enhanced form in a database and, in a form that is intuitive for the user, via the virtual file system of the operating system being represented.

Main article: Associative file management

Security aspects

The file system must not lose or accidentally overwrite any data. Two cases in particular involve dangers:

If several tasks are pending at the same time in multitasking, the file system has to keep the individual actions separate so that nothing gets mixed up. If the tasks also address the same file, be it only reading or writing, appropriate locking mechanisms ( locks ) are typically made available or automatically activated in order to avoid conflicts. Simultaneous access from several pages e.g. As a large database file are also the norm, one that besides global barriers that affect the entire file, including those for individual records ( Records can use).

If a drive is currently writing to a storage medium and the operating voltage fails at this point, there is a risk that not only the actual data will be written incompletely, but also that the organizational entries in the directory will no longer be updated correctly. In order to keep this risk at least as small as possible, on the one hand hardware attempts are made to have enough energy buffers (capacitors in the supply) ready so that a work process can still be carried out, on the other hand, the software is designed so that the work steps are as "atomic" as possible are designed, i.e. the sensitive period of time with incomplete data entries is kept as short as possible. If this does not help in extreme cases, there are so-called journaling file systems as a newer development , which keep records of every work step in an additional area of ​​the storage medium, so that what could still be done and what no longer can be reconstructed afterwards.

Flash memories have their own aspects in that they are exposed to wear and tear when they are erased and rewritten, which, depending on the type, only allows around 100,000 to 1,000,000 write cycles. As a rule, it is not possible to delete individual bytes individually, but rather only entire blocks (of varying sizes depending on the model) at once. The file system can be optimized here so that it distributes the write processes as evenly as possible over the entire memory area of ​​the flash module and, for example, does not always start writing at address 0. Keyword: wear leveling algorithms .

The aspect of data security against spying by unauthorized persons is used by file systems that can encrypt all data without other layers of the operating system having to do this.

Another source of danger for the integrity of the data is write actions that are carried out by any software, bypassing the file system, directly to physical addresses on the storage medium. With older operating systems this was easily possible and led to frequent data loss. Newer operating systems can protect these lower levels much more effectively against unauthorized access, so that with the rights of a normal user, direct access to physical media addresses is no longer permitted. If certain diagnostic or repair utilities ( tools ) do require such access, they must have administrator rights.

Life cycle aspects

When migrating files, for example due to a system replacement, files often have to be transferred from one file system to another. This is generally a difficult undertaking because many file systems are functionally incompatible with one another; H. the destination file system cannot hold all files with all attributes that are stored on the source file system. An example of this would be migrating NTFS files with alternate data streams to a file system without support for such streams.

See also

• List of file systems
• Fragmentation (file system)

Web links

Commons : File systems  - collection of images, videos, and audio files

• Comparison and comparison of all file systems (English Wikipedia)
• disktype recognizes the file system type

Linux:

• How do Linux file systems work?
• Linux 4.4 To 4.7 - EXT4 vs. F2FS vs. Btrfs benchmarks (SSD)

Individual evidence

1. ↑ Aeleen fresh: Unix system administration . O'Reilly Germany, 2003, 2: The Unix Philosophy, p. 66, footnote 13 ( full text in Google Book Search): “The term file system thus refers on the one hand to the higher-level directory tree of the system, which includes all hard drive partitions of the system that the user can access (as in» the Unix file system «), On the other hand to the files and directories on the individual hard disk partitions (as in“ Setting up a file system on a hard disk partition ”or“ Mounting the user file system ”). Only from the context does it become clear which of the two meanings of the term is meant. "
2. ↑ Aeleen fresh: Unix system administration . O'Reilly Germany, 2003, 2: The Unix Philosophy, p. 66, footnote 13 ( full text in Google Book Search): “The terms partition and file system are also sometimes used synonymously. Although technically only file systems can be mounted, expressions like "mount a hard drive" or "mount a partition" are common. "
3. ↑ o3one.org