In computing, a distributed file system (DFS) or network file system is any file system that allows access from multiple hosts to files shared via a computer network. This makes it possible for multiple users on multiple machines to share files and storage resources.
Distributed file systems differ in their performance, mutability of content, handling of concurrent writes, handling of permanent or temporary loss of nodes or storage, and their policy of storing content.
Locally managed FOSS Client Written in License Access API High availability Shards Efficient Redundancy Redundancy Granularity Initial release year Memory requirements (GB)Alluxio (Virtual Distributed File System) Java HDFS, FUSE, HTTP/REST, S3 2013Ceph C++ librados (C, C++, Python, Ruby), S3, Swift, FUSE 2010 1 per TB of storageCoda C C 1987 GlusterFS C libglusterfs, FUSE, NFS, SMB, Swift, libgfapi 2005HDFS Java Java and C client, HTTP, FUSE 2005IPFS Go HTTP gateway, FUSE, Go client, Javascript client, command line tool Erasure coding plan: , 2015LizardFS C++ POSIX, FUSE, NFS-Ganesha, Ceph FSAL (via libcephfs) 2013Lustre C POSIX, NFS-Ganesha, NFS, SMB Reed-Solomon in progress: File-level redundancy plan: 2003MinIO Go AWS S3 API, FTP, SFTP 2014MooseFS C POSIX, FUSE Only available in the proprietary version 4.x 2008OpenAFS C Virtual file system, Installable File System 2000OpenIO C Native (Python, C, Java), HTTP/REST, S3, Swift, FUSE (POSIX, NFS, SMB, FTP) 2015 0.5Quantcast File System C C++ client, FUSE (C++ server: MetaServer and ChunkServer are both in C++) 2012RozoFS C, Python FUSE, SMB, NFS, key/value 2011 Tahoe-LAFS Python HTTP (browser or CLI), SFTP, FTP, FUSE via SSHFS, pyfilesystem 2007XtreemFS Java, C++ libxtreemfs (Java, C++), FUSE 2009
Proprietary
Client Written in License Access APIBeeGFS C / C++ FRAUNHOFER FS (FhGFS) EULA, GPLv2 client POSIXCloudianC++ AWS S3, NFS, SMB/CIFS, Rest APIObjectiveFS C POSIX, FUSESpectrum Scale (GPFS) C, C++ POSIX, NFS, SMB, Swift, S3, HDFSMapR-FS C, C++ POSIX, NFS, FUSE, S3, HDFS, CLIIsilon OneFS C/C++ POSIX, NFS, SMB/CIFS, HDFS, HTTP, FTP, SWIFT Object, CLI, Rest APIQumulo C/C++ POSIX, NFS, SMB/CIFS, CLI, S3, Rest APIScality C FUSE, NFS, REST, AWS S3VaultFS C/C++ POSIX, NFS, SMB/CIFS, CLI, S3, Rest API
Remote access Name Run by Access APIAmazon S3 Amazon.com HTTP (REST/SOAP)Google Cloud Storage Google HTTP (REST)SWIFT (part of OpenStack) Rackspace, Hewlett-Packard, others HTTP (REST)Microsoft Azure Microsoft HTTP (REST)IBM Cloud Object StorageIBM (formerly Cleversafe) HTTP (REST)
Comparison Some researchers have made a functional and experimental analysis of several distributed file systems including HDFS, Ceph, Gluster, Lustre and old (1.6.x) version of MooseFS, although this document is from 2013 and a lot of information are outdated (e.g. MooseFS had no HA for Metadata Server at that time).
The cloud based remote distributed storage from major vendors have different APIs and different consistency models.
See also Distributed file system List of file systems, the Distributed file systems section
References