Jeffrey Sheldon
A large portion of the functionality managed by the Unix kernel is in the Unix file system. This file system extends well beyond the capabilities of a traditional file system because it supports features such as arbitrary mount points, special file mapping and other functionality. This functionality not only forms a very powerful abstraction for the Unix environment, but also forms a large portion of the kernel. The file system, thus, becomes a strong candidate for a component to be removed from the kernel and made a trusted user mode module.
Due to the number of features managed by the file system, creating a user mode file system process would reduce the complexity of the kernel and thereby increase system reliability by isolating file system faults from the kernel. In theory it could also prevent faults from impacting other user processes which do not use the disk, but since most user processes would be clients of a file system process this isolation may have minimal practical impact. Moving the file system to user mode also opens up a powerful possibility to allow a user mode file system process to make use of another user mode file system which provides support for an entire subtree of the file system. This would allow for simple integration of network file systems, special version control mounts, or even mapping various user functionality to special files by combining file system drivers at run time.
While the flexibility which results from bringing the file system support out of the kernel is appealing, it comes with high costs and actually introduces considerable complexity. Since the final disk or network IO must still travel through the kernel, file access would result in at least one, if not multiple, interprocess requests followed by at least one switch to kernel mode. Such a long chain of data flow would naturally be very costly in terms of processor time. The slow speed could be somewhat mitigated by processor speed increasing faster than the disk, but caching removes most of this effect anyway. Implementing file systems on disk in this manner would also require the kernel to manage raw disk access, thereby forcing the introduction of a system of special permissions for processes that would be prone to security holes. Trusting non-kernel processes to manage the file system would also require exposing the security subsystem to manage file permissions. Additionally any file system security must be managed by the file system to provide persistence. The validity of the security, therefore, depends on a reliable and secure file system.
Moving the file system out of the kernel incurs high costs and risks which outweigh the potential feature gains resulting from a modular set of file system processes. The gains in fault isolation are probably more than offset by an increased potential for faults due to the resulting complex interaction between the kernel and the file system. The file system should, therefore, remain a part of the kernel.