summaryrefslogtreecommitdiffstats
path: root/Documentation/filesystems
diff options
context:
space:
mode:
Diffstat (limited to 'Documentation/filesystems')
-rw-r--r--Documentation/filesystems/f2fs.txt216
-rw-r--r--Documentation/filesystems/fscrypt.rst6
-rw-r--r--Documentation/filesystems/fuse.rst (renamed from Documentation/filesystems/fuse.txt)163
-rw-r--r--Documentation/filesystems/index.rst1
-rw-r--r--Documentation/filesystems/mount_api.txt12
-rw-r--r--Documentation/filesystems/zonefs.txt404
6 files changed, 535 insertions, 267 deletions
diff --git a/Documentation/filesystems/f2fs.txt b/Documentation/filesystems/f2fs.txt
index 3135b80df6da..4eb3e2ddd00e 100644
--- a/Documentation/filesystems/f2fs.txt
+++ b/Documentation/filesystems/f2fs.txt
@@ -235,6 +235,17 @@ checkpoint=%s[:%u[%]] Set to "disable" to turn off checkpointing. Set to "en
hide up to all remaining free space. The actual space that
would be unusable can be viewed at /sys/fs/f2fs/<disk>/unusable
This space is reclaimed once checkpoint=enable.
+compress_algorithm=%s Control compress algorithm, currently f2fs supports "lzo"
+ and "lz4" algorithm.
+compress_log_size=%u Support configuring compress cluster size, the size will
+ be 4KB * (1 << %u), 16KB is minimum size, also it's
+ default size.
+compress_extension=%s Support adding specified extension, so that f2fs can enable
+ compression on those corresponding files, e.g. if all files
+ with '.ext' has high compression rate, we can set the '.ext'
+ on compression extension list and enable compression on
+ these file by default rather than to enable it via ioctl.
+ For other files, we can still enable compression via ioctl.
================================================================================
DEBUGFS ENTRIES
@@ -259,170 +270,6 @@ The files in each per-device directory are shown in table below.
Files in /sys/fs/f2fs/<devname>
(see also Documentation/ABI/testing/sysfs-fs-f2fs)
-..............................................................................
- File Content
-
- gc_urgent_sleep_time This parameter controls sleep time for gc_urgent.
- 500 ms is set by default. See above gc_urgent.
-
- gc_min_sleep_time This tuning parameter controls the minimum sleep
- time for the garbage collection thread. Time is
- in milliseconds.
-
- gc_max_sleep_time This tuning parameter controls the maximum sleep
- time for the garbage collection thread. Time is
- in milliseconds.
-
- gc_no_gc_sleep_time This tuning parameter controls the default sleep
- time for the garbage collection thread. Time is
- in milliseconds.
-
- gc_idle This parameter controls the selection of victim
- policy for garbage collection. Setting gc_idle = 0
- (default) will disable this option. Setting
- gc_idle = 1 will select the Cost Benefit approach
- & setting gc_idle = 2 will select the greedy approach.
-
- gc_urgent This parameter controls triggering background GCs
- urgently or not. Setting gc_urgent = 0 [default]
- makes back to default behavior, while if it is set
- to 1, background thread starts to do GC by given
- gc_urgent_sleep_time interval.
-
- reclaim_segments This parameter controls the number of prefree
- segments to be reclaimed. If the number of prefree
- segments is larger than the number of segments
- in the proportion to the percentage over total
- volume size, f2fs tries to conduct checkpoint to
- reclaim the prefree segments to free segments.
- By default, 5% over total # of segments.
-
- main_blkaddr This value gives the first block address of
- MAIN area in the partition.
-
- max_small_discards This parameter controls the number of discard
- commands that consist small blocks less than 2MB.
- The candidates to be discarded are cached until
- checkpoint is triggered, and issued during the
- checkpoint. By default, it is disabled with 0.
-
- discard_granularity This parameter controls the granularity of discard
- command size. It will issue discard commands iif
- the size is larger than given granularity. Its
- unit size is 4KB, and 4 (=16KB) is set by default.
- The maximum value is 128 (=512KB).
-
- reserved_blocks This parameter indicates the number of blocks that
- f2fs reserves internally for root.
-
- batched_trim_sections This parameter controls the number of sections
- to be trimmed out in batch mode when FITRIM
- conducts. 32 sections is set by default.
-
- ipu_policy This parameter controls the policy of in-place
- updates in f2fs. There are five policies:
- 0x01: F2FS_IPU_FORCE, 0x02: F2FS_IPU_SSR,
- 0x04: F2FS_IPU_UTIL, 0x08: F2FS_IPU_SSR_UTIL,
- 0x10: F2FS_IPU_FSYNC.
-
- min_ipu_util This parameter controls the threshold to trigger
- in-place-updates. The number indicates percentage
- of the filesystem utilization, and used by
- F2FS_IPU_UTIL and F2FS_IPU_SSR_UTIL policies.
-
- min_fsync_blocks This parameter controls the threshold to trigger
- in-place-updates when F2FS_IPU_FSYNC mode is set.
- The number indicates the number of dirty pages
- when fsync needs to flush on its call path. If
- the number is less than this value, it triggers
- in-place-updates.
-
- min_seq_blocks This parameter controls the threshold to serialize
- write IOs issued by multiple threads in parallel.
-
- min_hot_blocks This parameter controls the threshold to allocate
- a hot data log for pending data blocks to write.
-
- min_ssr_sections This parameter adds the threshold when deciding
- SSR block allocation. If this is large, SSR mode
- will be enabled early.
-
- ram_thresh This parameter controls the memory footprint used
- by free nids and cached nat entries. By default,
- 1 is set, which indicates 10 MB / 1 GB RAM.
-
- ra_nid_pages When building free nids, F2FS reads NAT blocks
- ahead for speed up. Default is 0.
-
- dirty_nats_ratio Given dirty ratio of cached nat entries, F2FS
- determines flushing them in background.
-
- max_victim_search This parameter controls the number of trials to
- find a victim segment when conducting SSR and
- cleaning operations. The default value is 4096
- which covers 8GB block address range.
-
- migration_granularity For large-sized sections, F2FS can stop GC given
- this granularity instead of reclaiming entire
- section.
-
- dir_level This parameter controls the directory level to
- support large directory. If a directory has a
- number of files, it can reduce the file lookup
- latency by increasing this dir_level value.
- Otherwise, it needs to decrease this value to
- reduce the space overhead. The default value is 0.
-
- cp_interval F2FS tries to do checkpoint periodically, 60 secs
- by default.
-
- idle_interval F2FS detects system is idle, if there's no F2FS
- operations during given interval, 5 secs by
- default.
-
- discard_idle_interval F2FS detects the discard thread is idle, given
- time interval. Default is 5 secs.
-
- gc_idle_interval F2FS detects the GC thread is idle, given time
- interval. Default is 5 secs.
-
- umount_discard_timeout When unmounting the disk, F2FS waits for finishing
- queued discard commands which can take huge time.
- This gives time out for it, 5 secs by default.
-
- iostat_enable This controls to enable/disable iostat in F2FS.
-
- readdir_ra This enables/disabled readahead of inode blocks
- in readdir, and default is enabled.
-
- gc_pin_file_thresh This indicates how many GC can be failed for the
- pinned file. If it exceeds this, F2FS doesn't
- guarantee its pinning state. 2048 trials is set
- by default.
-
- extension_list This enables to change extension_list for hot/cold
- files in runtime.
-
- inject_rate This controls injection rate of arbitrary faults.
-
- inject_type This controls injection type of arbitrary faults.
-
- dirty_segments This shows # of dirty segments.
-
- lifetime_write_kbytes This shows # of data written to the disk.
-
- features This shows current features enabled on F2FS.
-
- current_reserved_blocks This shows # of blocks currently reserved.
-
- unusable If checkpoint=disable, this shows the number of
- blocks that are unusable.
- If checkpoint=enable it shows the number of blocks
- that would be unusable if checkpoint=disable were
- to be set.
-
-encoding This shows the encoding used for casefolding.
- If casefolding is not enabled, returns (none)
================================================================================
USAGE
@@ -840,3 +687,44 @@ zero or random data, which is useful to the below scenario where:
4. address = fibmap(fd, offset)
5. open(blkdev)
6. write(blkdev, address)
+
+Compression implementation
+--------------------------
+
+- New term named cluster is defined as basic unit of compression, file can
+be divided into multiple clusters logically. One cluster includes 4 << n
+(n >= 0) logical pages, compression size is also cluster size, each of
+cluster can be compressed or not.
+
+- In cluster metadata layout, one special block address is used to indicate
+cluster is compressed one or normal one, for compressed cluster, following
+metadata maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs
+stores data including compress header and compressed data.
+
+- In order to eliminate write amplification during overwrite, F2FS only
+support compression on write-once file, data can be compressed only when
+all logical blocks in file are valid and cluster compress ratio is lower
+than specified threshold.
+
+- To enable compression on regular inode, there are three ways:
+* chattr +c file
+* chattr +c dir; touch dir/file
+* mount w/ -o compress_extension=ext; touch file.ext
+
+Compress metadata layout:
+ [Dnode Structure]
+ +-----------------------------------------------+
+ | cluster 1 | cluster 2 | ......... | cluster N |
+ +-----------------------------------------------+
+ . . . .
+ . . . .
+ . Compressed Cluster . . Normal Cluster .
++----------+---------+---------+---------+ +---------+---------+---------+---------+
+|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
++----------+---------+---------+---------+ +---------+---------+---------+---------+
+ . .
+ . .
+ . .
+ +-------------+-------------+----------+----------------------------+
+ | data length | data chksum | reserved | compressed data |
+ +-------------+-------------+----------+----------------------------+
diff --git a/Documentation/filesystems/fscrypt.rst b/Documentation/filesystems/fscrypt.rst
index 01e909245fcd..bd9932344804 100644
--- a/Documentation/filesystems/fscrypt.rst
+++ b/Documentation/filesystems/fscrypt.rst
@@ -1016,9 +1016,9 @@ astute users may notice some differences in behavior:
- Direct I/O is not supported on encrypted files. Attempts to use
direct I/O on such files will fall back to buffered I/O.
-- The fallocate operations FALLOC_FL_COLLAPSE_RANGE,
- FALLOC_FL_INSERT_RANGE, and FALLOC_FL_ZERO_RANGE are not supported
- on encrypted files and will fail with EOPNOTSUPP.
+- The fallocate operations FALLOC_FL_COLLAPSE_RANGE and
+ FALLOC_FL_INSERT_RANGE are not supported on encrypted files and will
+ fail with EOPNOTSUPP.
- Online defragmentation of encrypted files is not supported. The
EXT4_IOC_MOVE_EXT and F2FS_IOC_MOVE_RANGE ioctls will fail with
diff --git a/Documentation/filesystems/fuse.txt b/Documentation/filesystems/fuse.rst
index 13af4a49e7db..8e455065ce9e 100644
--- a/Documentation/filesystems/fuse.txt
+++ b/Documentation/filesystems/fuse.rst
@@ -1,41 +1,40 @@
+.. SPDX-License-Identifier: GPL-2.0
+==============
+FUSE
+==============
+
Definitions
-~~~~~~~~~~~
+===========
Userspace filesystem:
-
A filesystem in which data and metadata are provided by an ordinary
userspace process. The filesystem can be accessed normally through
the kernel interface.
Filesystem daemon:
-
The process(es) providing the data and metadata of the filesystem.
Non-privileged mount (or user mount):
-
A userspace filesystem mounted by a non-privileged (non-root) user.
The filesystem daemon is running with the privileges of the mounting
user. NOTE: this is not the same as mounts allowed with the "user"
option in /etc/fstab, which is not discussed here.
Filesystem connection:
-
A connection between the filesystem daemon and the kernel. The
connection exists until either the daemon dies, or the filesystem is
umounted. Note that detaching (or lazy umounting) the filesystem
- does _not_ break the connection, in this case it will exist until
+ does *not* break the connection, in this case it will exist until
the last reference to the filesystem is released.
Mount owner:
-
The user who does the mounting.
User:
-
The user who is performing filesystem operations.
What is FUSE?
-~~~~~~~~~~~~~
+=============
FUSE is a userspace filesystem framework. It consists of a kernel
module (fuse.ko), a userspace library (libfuse.*) and a mount utility
@@ -46,50 +45,41 @@ non-privileged mounts. This opens up new possibilities for the use of
filesystems. A good example is sshfs: a secure network filesystem
using the sftp protocol.
-The userspace library and utilities are available from the FUSE
-homepage:
-
- http://fuse.sourceforge.net/
+The userspace library and utilities are available from the
+`FUSE homepage: <http://fuse.sourceforge.net/>`_
Filesystem type
-~~~~~~~~~~~~~~~
+===============
The filesystem type given to mount(2) can be one of the following:
-'fuse'
-
- This is the usual way to mount a FUSE filesystem. The first
- argument of the mount system call may contain an arbitrary string,
- which is not interpreted by the kernel.
+ fuse
+ This is the usual way to mount a FUSE filesystem. The first
+ argument of the mount system call may contain an arbitrary string,
+ which is not interpreted by the kernel.
-'fuseblk'
-
- The filesystem is block device based. The first argument of the
- mount system call is interpreted as the name of the device.
+ fuseblk
+ The filesystem is block device based. The first argument of the
+ mount system call is interpreted as the name of the device.
Mount options
-~~~~~~~~~~~~~
-
-'fd=N'
+=============
+fd=N
The file descriptor to use for communication between the userspace
filesystem and the kernel. The file descriptor must have been
obtained by opening the FUSE device ('/dev/fuse').
-'rootmode=M'
-
+rootmode=M
The file mode of the filesystem's root in octal representation.
-'user_id=N'
-
+user_id=N
The numeric user id of the mount owner.
-'group_id=N'
-
+group_id=N
The numeric group id of the mount owner.
-'default_permissions'
-
+default_permissions
By default FUSE doesn't check file access permissions, the
filesystem is free to implement its access policy or leave it to
the underlying file access mechanism (e.g. in case of network
@@ -97,28 +87,25 @@ Mount options
access based on file mode. It is usually useful together with the
'allow_other' mount option.
-'allow_other'
-
+allow_other
This option overrides the security measure restricting file access
to the user mounting the filesystem. This option is by default only
allowed to root, but this restriction can be removed with a
(userspace) configuration option.
-'max_read=N'
-
+max_read=N
With this option the maximum size of read operations can be set.
The default is infinite. Note that the size of read requests is
limited anyway to 32 pages (which is 128kbyte on i386).
-'blksize=N'
-
+blksize=N
Set the block size for the filesystem. The default is 512. This
option is only valid for 'fuseblk' type mounts.
Control filesystem
-~~~~~~~~~~~~~~~~~~
+==================
-There's a control filesystem for FUSE, which can be mounted by:
+There's a control filesystem for FUSE, which can be mounted by::
mount -t fusectl none /sys/fs/fuse/connections
@@ -130,53 +117,51 @@ named by a unique number.
For each connection the following files exist within this directory:
- 'waiting'
-
- The number of requests which are waiting to be transferred to
- userspace or being processed by the filesystem daemon. If there is
- no filesystem activity and 'waiting' is non-zero, then the
- filesystem is hung or deadlocked.
-
- 'abort'
+ waiting
+ The number of requests which are waiting to be transferred to
+ userspace or being processed by the filesystem daemon. If there is
+ no filesystem activity and 'waiting' is non-zero, then the
+ filesystem is hung or deadlocked.
- Writing anything into this file will abort the filesystem
- connection. This means that all waiting requests will be aborted an
- error returned for all aborted and new requests.
+ abort
+ Writing anything into this file will abort the filesystem
+ connection. This means that all waiting requests will be aborted an
+ error returned for all aborted and new requests.
Only the owner of the mount may read or write these files.
Interrupting filesystem operations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+##################################
If a process issuing a FUSE filesystem request is interrupted, the
following will happen:
- 1) If the request is not yet sent to userspace AND the signal is
+ - If the request is not yet sent to userspace AND the signal is
fatal (SIGKILL or unhandled fatal signal), then the request is
dequeued and returns immediately.
- 2) If the request is not yet sent to userspace AND the signal is not
- fatal, then an 'interrupted' flag is set for the request. When
+ - If the request is not yet sent to userspace AND the signal is not
+ fatal, then an interrupted flag is set for the request. When
the request has been successfully transferred to userspace and
this flag is set, an INTERRUPT request is queued.
- 3) If the request is already sent to userspace, then an INTERRUPT
+ - If the request is already sent to userspace, then an INTERRUPT
request is queued.
INTERRUPT requests take precedence over other requests, so the
userspace filesystem will receive queued INTERRUPTs before any others.
The userspace filesystem may ignore the INTERRUPT requests entirely,
-or may honor them by sending a reply to the _original_ request, with
+or may honor them by sending a reply to the *original* request, with
the error set to EINTR.
It is also possible that there's a race between processing the
original request and its INTERRUPT request. There are two possibilities:
- 1) The INTERRUPT request is processed before the original request is
+ 1. The INTERRUPT request is processed before the original request is
processed
- 2) The INTERRUPT request is processed after the original request has
+ 2. The INTERRUPT request is processed after the original request has
been answered
If the filesystem cannot find the original request, it should wait for
@@ -186,7 +171,7 @@ should reply to the INTERRUPT request with an EAGAIN error. In case
reply will be ignored.
Aborting a filesystem connection
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+================================
It is possible to get into certain situations where the filesystem is
not responding. Reasons for this may be:
@@ -216,7 +201,7 @@ the filesystem. There are several ways to do this:
powerful method, always works.
How do non-privileged mounts work?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+==================================
Since the mount() system call is a privileged operation, a helper
program (fusermount) is needed, which is installed setuid root.
@@ -235,15 +220,13 @@ system. Obvious requirements arising from this are:
other users' or the super user's processes
How are requirements fulfilled?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+===============================
A) The mount owner could gain elevated privileges by either:
- 1) creating a filesystem containing a device file, then opening
- this device
+ 1. creating a filesystem containing a device file, then opening this device
- 2) creating a filesystem containing a suid or sgid application,
- then executing this application
+ 2. creating a filesystem containing a suid or sgid application, then executing this application
The solution is not to allow opening device files and ignore
setuid and setgid bits when executing programs. To ensure this
@@ -275,16 +258,16 @@ How are requirements fulfilled?
of other users' processes.
i) It can slow down or indefinitely delay the execution of a
- filesystem operation creating a DoS against the user or the
- whole system. For example a suid application locking a
- system file, and then accessing a file on the mount owner's
- filesystem could be stopped, and thus causing the system
- file to be locked forever.
+ filesystem operation creating a DoS against the user or the
+ whole system. For example a suid application locking a
+ system file, and then accessing a file on the mount owner's
+ filesystem could be stopped, and thus causing the system
+ file to be locked forever.
ii) It can present files or directories of unlimited length, or
- directory structures of unlimited depth, possibly causing a
- system process to eat up diskspace, memory or other
- resources, again causing DoS.
+ directory structures of unlimited depth, possibly causing a
+ system process to eat up diskspace, memory or other
+ resources, again causing *DoS*.
The solution to this as well as B) is not to allow processes
to access the filesystem, which could otherwise not be
@@ -294,28 +277,27 @@ How are requirements fulfilled?
ptrace can be used to check if a process is allowed to access
the filesystem or not.
- Note that the ptrace check is not strictly necessary to
+ Note that the *ptrace* check is not strictly necessary to
prevent B/2/i, it is enough to check if mount owner has enough
privilege to send signal to the process accessing the
- filesystem, since SIGSTOP can be used to get a similar effect.
+ filesystem, since *SIGSTOP* can be used to get a similar effect.
I think these limitations are unacceptable?
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+===========================================
If a sysadmin trusts the users enough, or can ensure through other
measures, that system processes will never enter non-privileged
-mounts, it can relax the last limitation with a "user_allow_other"
+mounts, it can relax the last limitation with a 'user_allow_other'
config option. If this config option is set, the mounting user can
-add the "allow_other" mount option which disables the check for other
+add the 'allow_other' mount option which disables the check for other
users' processes.
Kernel - userspace interface
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+============================
The following diagram shows how a filesystem operation (in this
-example unlink) is performed in FUSE.
+example unlink) is performed in FUSE. ::
-NOTE: everything in this description is greatly simplified
| "rm /mnt/fuse/file" | FUSE filesystem daemon
| |
@@ -357,12 +339,13 @@ NOTE: everything in this description is greatly simplified
| <fuse_unlink() |
| <sys_unlink() |
+.. note:: Everything in the description above is greatly simplified
+
There are a couple of ways in which to deadlock a FUSE filesystem.
Since we are talking about unprivileged userspace programs,
something must be done about these.
-Scenario 1 - Simple deadlock
------------------------------
+**Scenario 1 - Simple deadlock**::
| "rm /mnt/fuse/file" | FUSE filesystem daemon
| |
@@ -379,12 +362,12 @@ Scenario 1 - Simple deadlock
The solution for this is to allow the filesystem to be aborted.
-Scenario 2 - Tricky deadlock
-----------------------------
+**Scenario 2 - Tricky deadlock**
+
This one needs a carefully crafted filesystem. It's a variation on
the above, only the call back to the filesystem is not explicit,
-but is caused by a pagefault.
+but is caused by a pagefault. ::
| Kamikaze filesystem thread 1 | Kamikaze filesystem thread 2
| |
@@ -410,7 +393,7 @@ but is caused by a pagefault.
| | [lock page]
| | * DEADLOCK *
-Solution is basically the same as above.
+The solution is basically the same as above.
An additional problem is that while the write buffer is being copied
to the request, the request must not be interrupted/aborted. This is
diff --git a/Documentation/filesystems/index.rst b/Documentation/filesystems/index.rst
index 824a3ecbb0ca..386eaad008b2 100644
--- a/Documentation/filesystems/index.rst
+++ b/Documentation/filesystems/index.rst
@@ -47,6 +47,7 @@ Documentation for filesystem implementations.
:maxdepth: 2
autofs
+ fuse
overlayfs
virtiofs
vfat
diff --git a/Documentation/filesystems/mount_api.txt b/Documentation/filesystems/mount_api.txt
index 00ff0cfccfa7..87c14bbb2b35 100644
--- a/Documentation/filesystems/mount_api.txt
+++ b/Documentation/filesystems/mount_api.txt
@@ -427,7 +427,6 @@ returned.
fs_value_is_string, Value is a string
fs_value_is_blob, Value is a binary blob
fs_value_is_filename, Value is a filename* + dirfd
- fs_value_is_filename_empty, Value is a filename* + dirfd + AT_EMPTY_PATH
fs_value_is_file, Value is an open file (file*)
If there is a value, that value is stored in a union in the struct in one
@@ -519,7 +518,6 @@ Parameters are described using structures defined in linux/fs_parser.h.
There's a core description struct that links everything together:
struct fs_parameter_description {
- const char name[16];
const struct fs_parameter_spec *specs;
const struct fs_parameter_enum *enums;
};
@@ -535,19 +533,13 @@ For example:
};
static const struct fs_parameter_description afs_fs_parameters = {
- .name = "kAFS",
.specs = afs_param_specs,
.enums = afs_param_enums,
};
The members are as follows:
- (1) const char name[16];
-
- The name to be used in error messages generated by the parse helper
- functions.
-
- (2) const struct fs_parameter_specification *specs;
+ (1) const struct fs_parameter_specification *specs;
Table of parameter specifications, terminated with a null entry, where the
entries are of type:
@@ -626,7 +618,7 @@ The members are as follows:
of arguments to specify the type and the flags for anything that doesn't
match one of the above macros.
- (6) const struct fs_parameter_enum *enums;
+ (2) const struct fs_parameter_enum *enums;
Table of enum value names to integer mappings, terminated with a null
entry. This is of type:
diff --git a/Documentation/filesystems/zonefs.txt b/Documentation/filesystems/zonefs.txt
new file mode 100644
index 000000000000..935bf22031ca
--- /dev/null
+++ b/Documentation/filesystems/zonefs.txt
@@ -0,0 +1,404 @@
+ZoneFS - Zone filesystem for Zoned block devices
+
+Introduction
+============
+
+zonefs is a very simple file system exposing each zone of a zoned block device
+as a file. Unlike a regular POSIX-compliant file system with native zoned block
+device support (e.g. f2fs), zonefs does not hide the sequential write
+constraint of zoned block devices to the user. Files representing sequential
+write zones of the device must be written sequentially starting from the end
+of the file (append only writes).
+
+As such, zonefs is in essence closer to a raw block device access interface
+than to a full-featured POSIX file system. The goal of zonefs is to simplify
+the implementation of zoned block device support in applications by replacing
+raw block device file accesses with a richer file API, avoiding relying on
+direct block device file ioctls which may be more obscure to developers. One
+example of this approach is the implementation of LSM (log-structured merge)
+tree structures (such as used in RocksDB and LevelDB) on zoned block devices
+by allowing SSTables to be stored in a zone file similarly to a regular file
+system rather than as a range of sectors of the entire disk. The introduction
+of the higher level construct "one file is one zone" can help reducing the
+amount of changes needed in the application as well as introducing support for
+different application programming languages.
+
+Zoned block devices
+-------------------
+
+Zoned storage devices belong to a class of storage devices with an address
+space that is divided into zones. A zone is a group of consecutive LBAs and all
+zones are contiguous (there are no LBA gaps). Zones may have different types.
+* Conventional zones: there are no access constraints to LBAs belonging to
+ conventional zones. Any read or write access can be executed, similarly to a
+ regular block device.
+* Sequential zones: these zones accept random reads but must be written
+ sequentially. Each sequential zone has a write pointer maintained by the
+ device that keeps track of the mandatory start LBA position of the next write
+ to the device. As a result of this write constraint, LBAs in a sequential zone
+ cannot be overwritten. Sequential zones must first be erased using a special
+ command (zone reset) before rewriting.
+
+Zoned storage devices can be implemented using various recording and media
+technologies. The most common form of zoned storage today uses the SCSI Zoned
+Block Commands (ZBC) and Zoned ATA Commands (ZAC) interfaces on Shingled
+Magnetic Recording (SMR) HDDs.
+
+Solid State Disks (SSD) storage devices can also implement a zoned interface
+to, for instance, reduce internal write amplification due to garbage collection.
+The NVMe Zoned NameSpace (ZNS) is a technical proposal of the NVMe standard
+committee aiming at adding a zoned storage interface to the NVMe protocol.
+
+Zonefs Overview
+===============
+
+Zonefs exposes the zones of a zoned block device as files. The files
+representing zones are grouped by zone type, which are themselves represented
+by sub-directories. This file structure is built entirely using zone information
+provided by the device and so does not require any complex on-disk metadata
+structure.
+
+On-disk metadata
+----------------
+
+zonefs on-disk metadata is reduced to an immutable super block which
+persistently stores a magic number and optional feature flags and values. On
+mount, zonefs uses blkdev_report_zones() to obtain the device zone configuration
+and populates the mount point with a static file tree solely based on this
+information. File sizes come from the device zone type and write pointer
+position managed by the device itself.
+
+The super block is always written on disk at sector 0. The first zone of the
+device storing the super block is never exposed as a zone file by zonefs. If
+the zone containing the super block is a sequential zone, the mkzonefs format
+tool always "finishes" the zone, that is, it transitions the zone to a full
+state to make it read-only, preventing any data write.
+
+Zone type sub-directories
+-------------------------
+
+Files representing zones of the same type are grouped together under the same
+sub-directory automatically created on mount.
+
+For conventional zones, the sub-directory "cnv" is used. This directory is
+however created if and only if the device has usable conventional zones. If
+the device only has a single conventional zone at sector 0, the zone will not
+be exposed as a file as it will be used to store the zonefs super block. For
+such devices, the "cnv" sub-directory will not be created.
+
+For sequential write zones, the sub-directory "seq" is used.
+
+These two directories are the only directories that exist in zonefs. Users
+cannot create other directories and cannot rename nor delete the "cnv" and
+"seq" sub-directories.
+
+The size of the directories indicated by the st_size field of struct stat,
+obtained with the stat() or fstat() system calls, indicates the number of files
+existing under the directory.
+
+Zone files
+----------
+
+Zone files are named using the number of the zone they represent within the set
+of zones of a particular type. That is, both the "cnv" and "seq" directories
+contain files named "0", "1", "2", ... The file numbers also represent
+increasing zone start sector on the device.
+
+All read and write operations to zone files are not allowed beyond the file
+maximum size, that is, beyond the zone size. Any access exceeding the zone
+size is failed with the -EFBIG error.
+
+Creating, deleting, renaming or modifying any attribute of files and
+sub-directories is not allowed.
+
+The number of blocks of a file as reported by stat() and fstat() indicates the
+size of the file zone, or in other words, the maximum file size.
+
+Conventional zone files
+-----------------------
+
+The size of conventional zone files is fixed to the size of the zone they
+represent. Conventional zone files cannot be truncated.
+
+These files can be randomly read and written using any type of I/O operation:
+buffered I/Os, direct I/Os, memory mapped I/Os (mmap), etc. There are no I/O
+constraint for these files beyond the file size limit mentioned above.
+
+Sequential zone files
+---------------------
+
+The size of sequential zone files grouped in the "seq" sub-directory represents
+the file's zone write pointer position relative to the zone start sector.
+
+Sequential zone files can only be written sequentially, starting from the file
+end, that is, write operations can only be append writes. Zonefs makes no
+attempt at accepting random writes and will fail any write request that has a
+start offset not corresponding to the end of the file, or to the end of the last
+write issued and still in-flight (for asynchrnous I/O operations).
+
+Since dirty page writeback by the page cache does not guarantee a sequential
+write pattern, zonefs prevents buffered writes and writeable shared mappings
+on sequential files. Only direct I/O writes are accepted for these files.
+zonefs relies on the sequential delivery of write I/O requests to the device
+implemented by the block layer elevator. An elevator implementing the sequential
+write feature for zoned block device (ELEVATOR_F_ZBD_SEQ_WRITE elevator feature)
+must be used. This type of elevator (e.g. mq-deadline) is the set by default
+for zoned block devices on device initialization.
+
+There are no restrictions on the type of I/O used for read operations in
+sequential zone files. Buffered I/Os, direct I/Os and shared read mappings are
+all accepted.
+
+Truncating sequential zone files is allowed only down to 0, in which case, the
+zone is reset to rewind the file zone write pointer position to the start of
+the zone, or up to the zone size, in which case the file's zone is transitioned
+to the FULL state (finish zone operation).
+
+Format options
+--------------
+
+Several optional features of zonefs can be enabled at format time.
+* Conventional zone aggregation: ranges of contiguous conventional zones can be
+ aggregated into a single larger file instead of the default one file per zone.
+* File ownership: The owner UID and GID of zone files is by default 0 (root)
+ but can be changed to any valid UID/GID.
+* File access permissions: the default 640 access permissions can be changed.
+
+IO error handling
+-----------------
+
+Zoned block devices may fail I/O requests for reasons similar to regular block
+devices, e.g. due to bad sectors. However, in addition to such known I/O
+failure pattern, the standards governing zoned block devices behavior define
+additional conditions that result in I/O errors.
+
+* A zone may transition to the read-only condition (BLK_ZONE_COND_READONLY):
+ While the data already written in the zone is still readable, the zone can
+ no longer be written. No user action on the zone (zone management command or
+ read/write access) can change the zone condition back to a normal read/write
+ state. While the reasons for the device to transition a zone to read-only
+ state are not defined by the standards, a typical cause for such transition
+ would be a defective write head on an HDD (all zones under this head are
+ changed to read-only).
+
+* A zone may transition to the offline condition (BLK_ZONE_COND_OFFLINE):
+ An offline zone cannot be read nor written. No user action can transition an
+ offline zone back to an operational good state. Similarly to zone read-only
+ transitions, the reasons for a drive to transition a zone to the offline
+ condition are undefined. A typical cause would be a defective read-write head
+ on an HDD causing all zones on the platter under the broken head to be
+ inaccessible.
+
+* Unaligned write errors: These errors result from the host issuing write
+ requests with a start sector that does not correspond to a zone write pointer
+ position when the write request is executed by the device. Even though zonefs
+ enforces sequential file write for sequential zones, unaligned write errors
+ may still happen in the case of a partial failure of a very large direct I/O
+ operation split into multiple BIOs/requests or asynchronous I/O operations.
+ If one of the write request within the set of sequential write requests
+ issued to the device fails, all write requests after queued after it will
+ become unaligned and fail.
+
+* Delayed write errors: similarly to regular block devices, if the device side
+ write cache is enabled, write errors may occur in ranges of previously
+ completed writes when the device write cache is flushed, e.g. on fsync().
+ Similarly to the previous immediate unaligned write error case, delayed write
+ errors can propagate through a stream of cached sequential data for a zone
+ causing all data to be dropped after the sector that caused the error.
+
+All I/O errors detected by zonefs are notified to the user with an error code
+return for the system call that trigered or detected the error. The recovery
+actions taken by zonefs in response to I/O errors depend on the I/O type (read
+vs write) and on the reason for the error (bad sector, unaligned writes or zone
+condition change).
+
+* For read I/O errors, zonefs does not execute any particular recovery action,
+ but only if the file zone is still in a good condition and there is no
+ inconsistency between the file inode size and its zone write pointer position.
+ If a problem is detected, I/O error recovery is executed (see below table).
+
+* For write I/O errors, zonefs I/O error recovery is always executed.
+
+* A zone condition change to read-only or offline also always triggers zonefs
+ I/O error recovery.
+
+Zonefs minimal I/O error recovery may change a file size and a file access
+permissions.
+
+* File size changes:
+ Immediate or delayed write errors in a sequential zone file may cause the file
+ inode size to be inconsistent with the amount of data successfully written in
+ the file zone. For instance, the partial failure of a multi-BIO large write
+ operation will cause the zone write pointer to advance partially, even though
+ the entire write operation will be reported as failed to the user. In such
+ case, the file inode size must be advanced to reflect the zone write pointer
+ change and eventually allow the user to restart writing at the end of the
+ file.
+ A file size may also be reduced to reflect a delayed write error detected on
+ fsync(): in this case, the amount of data effectively written in the zone may
+ be less than originally indicated by the file inode size. After such I/O
+ error, zonefs always fixes a file inode size to reflect the amount of data
+ persistently stored in the file zone.
+
+* Access permission changes:
+ A zone condition change to read-only is indicated with a change in the file
+ access permissions to render the file read-only. This disables changes to the
+ file attributes and data modification. For offline zones, all permissions
+ (read and write) to the file are disabled.
+
+Further action taken by zonefs I/O error recovery can be controlled by the user
+with the "errors=xxx" mount option. The table below summarizes the result of
+zonefs I/O error processing depending on the mount option and on the zone
+conditions.
+
+ +--------------+-----------+-----------------------------------------+
+ | | | Post error state |
+ | "errors=xxx" | device | access permissions |
+ | mount | zone | file file device zone |
+ | option | condition | size read write read write |
+ +--------------+-----------+-----------------------------------------+
+ | | good | fixed yes no yes yes |
+ | remount-ro | read-only | fixed yes no yes no |
+ | (default) | offline | 0 no no no no |
+ +--------------+-----------+-----------------------------------------+
+ | | good | fixed yes no yes yes |
+ | zone-ro | read-only | fixed yes no yes no |
+ | | offline | 0 no no no no |
+ +--------------+-----------+-----------------------------------------+
+ | | good | 0 no no yes yes |
+ | zone-offline | read-only | 0 no no yes no |
+ | | offline | 0 no no no no |
+ +--------------+-----------+-----------------------------------------+
+ | | good | fixed yes yes yes yes |
+ | repair | read-only | fixed yes no yes no |
+ | | offline | 0 no no no no |
+ +--------------+-----------+-----------------------------------------+
+
+Further notes:
+* The "errors=remount-ro" mount option is the default behavior of zonefs I/O
+ error processing if no errors mount option is specified.
+* With the "errors=remount-ro" mount option, the change of the file access
+ permissions to read-only applies to all files. The file system is remounted
+ read-only.
+* Access permission and file size changes due to the device transitioning zones
+ to the offline condition are permanent. Remounting or reformating the device
+ with mkfs.zonefs (mkzonefs) will not change back offline zone files to a good
+ state.
+* File access permission changes to read-only due to the device transitioning
+ zones to the read-only condition are permanent. Remounting or reformating
+ the device will not re-enable file write access.
+* File access permission changes implied by the remount-ro, zone-ro and
+ zone-offline mount options are temporary for zones in a good condition.
+ Unmounting and remounting the file system will restore the previous default
+ (format time values) access rights to the files affected.
+* The repair mount option triggers only the minimal set of I/O error recovery
+ actions, that is, file size fixes for zones in a good condition. Zones
+ indicated as being read-only or offline by the device still imply changes to
+ the zone file access permissions as noted in the table above.
+
+Mount options
+-------------
+
+zonefs define the "errors=<behavior>" mount option to allow the user to specify
+zonefs behavior in response to I/O errors, inode size inconsistencies or zone
+condition chages. The defined behaviors are as follow:
+* remount-ro (default)
+* zone-ro
+* zone-offline
+* repair
+
+The I/O error actions defined for each behavior is detailed in the previous
+section.
+
+Zonefs User Space Tools
+=======================
+
+The mkzonefs tool is used to format zoned block devices for use with zonefs.
+This tool is available on Github at:
+
+https://github.com/damien-lemoal/zonefs-tools
+
+zonefs-tools also includes a test suite which can be run against any zoned
+block device, including null_blk block device created with zoned mode.
+
+Examples
+--------
+
+The following formats a 15TB host-managed SMR HDD with 256 MB zones
+with the conventional zones aggregation feature enabled.
+
+# mkzonefs -o aggr_cnv /dev/sdX
+# mount -t zonefs /dev/sdX /mnt
+# ls -l /mnt/
+total 0
+dr-xr-xr-x 2 root root 1 Nov 25 13:23 cnv
+dr-xr-xr-x 2 root root 55356 Nov 25 13:23 seq
+
+The size of the zone files sub-directories indicate the number of files
+existing for each type of zones. In this example, there is only one
+conventional zone file (all conventional zones are aggregated under a single
+file).
+
+# ls -l /mnt/cnv
+total 137101312
+-rw-r----- 1 root root 140391743488 Nov 25 13:23 0
+
+This aggregated conventional zone file can be used as a regular file.
+
+# mkfs.ext4 /mnt/cnv/0
+# mount -o loop /mnt/cnv/0 /data
+
+The "seq" sub-directory grouping files for sequential write zones has in this
+example 55356 zones.
+
+# ls -lv /mnt/seq
+total 14511243264
+-rw-r----- 1 root root 0 Nov 25 13:23 0
+-rw-r----- 1 root root 0 Nov 25 13:23 1
+-rw-r----- 1 root root 0 Nov 25 13:23 2
+...
+-rw-r----- 1 root root 0 Nov 25 13:23 55354
+-rw-r----- 1 root root 0 Nov 25 13:23 55355
+
+For sequential write zone files, the file size changes as data is appended at
+the end of the file, similarly to any regular file system.
+
+# dd if=/dev/zero of=/mnt/seq/0 bs=4096 count=1 conv=notrunc oflag=direct
+1+0 records in
+1+0 records out
+4096 bytes (4.1 kB, 4.0 KiB) copied, 0.00044121 s, 9.3 MB/s
+
+# ls -l /mnt/seq/0
+-rw-r----- 1 root root 4096 Nov 25 13:23 /mnt/seq/0
+
+The written file can be truncated to the zone size, preventing any further
+write operation.
+
+# truncate -s 268435456 /mnt/seq/0
+# ls -l /mnt/seq/0
+-rw-r----- 1 root root 268435456 Nov 25 13:49 /mnt/seq/0
+
+Truncation to 0 size allows freeing the file zone storage space and restart
+append-writes to the file.
+
+# truncate -s 0 /mnt/seq/0
+# ls -l /mnt/seq/0
+-rw-r----- 1 root root 0 Nov 25 13:49 /mnt/seq/0
+
+Since files are statically mapped to zones on the disk, the number of blocks of
+a file as reported by stat() and fstat() indicates the size of the file zone.
+
+# stat /mnt/seq/0
+ File: /mnt/seq/0
+ Size: 0 Blocks: 524288 IO Block: 4096 regular empty file
+Device: 870h/2160d Inode: 50431 Links: 1
+Access: (0640/-rw-r-----) Uid: ( 0/ root) Gid: ( 0/ root)
+Access: 2019-11-25 13:23:57.048971997 +0900
+Modify: 2019-11-25 13:52:25.553805765 +0900
+Change: 2019-11-25 13:52:25.553805765 +0900
+ Birth: -
+
+The number of blocks of the file ("Blocks") in units of 512B blocks gives the
+maximum file size of 524288 * 512 B = 256 MB, corresponding to the device zone
+size in this example. Of note is that the "IO block" field always indicates the
+minimum I/O size for writes and corresponds to the device physical sector size.
OpenPOWER on IntegriCloud