renameat2 flags, OverlayFS, and Docker
overlay2. The compatibility details and limitations below apply to FUSE-mounted Archil disks, including additional
disks attached to a sandbox, and do not apply to its ext4 root volume. See
Linux and container compatibility for sandbox capabilities.
POSIX compatibility details
Archil disks are designed to be broadly compatible with POSIX semantics, even when the underlying data sources are not POSIX compatible, like Amazon S3. When needed, the Archil storage service emulates file operations on data sources. Today, Archil supports the following POSIX features:- All read and write operations, including random writes and appends
- Symbolic links (
ln -s) - Hard Links (
ln) - Atomic renames, including directory renames (
mv)1 - POSIX locks, including
flockandfcntl2 - Anonymous files (unlink after opening)
- File modes
- File ownership
- Extended attributes (with the
--enable-xattrsflag)
archil invalidate-cache.
2.
flock and fcntl locks behave exactly as they do on a local disk for every process on a client, so a disk mounted by a single server or sandbox behaves like a local disk. Locks are not shared between different clients that mount the same disk. In exclusive and shared mode, only one client can write to a file at a time, and you coordinate writers across clients with the checkout/checkin ownership model. In conditional mode, several clients can write to the same file, so coordinate them in your application.
We are actively working on adding support for the following features:
- File ACLs (
system.posix_acl_access) renameat2flags (RENAME_NOREPLACE,RENAME_EXCHANGE,RENAME_WHITEOUT). A rename that passes any of these flags fails withEINVAL, so an Archil disk can’t be used as an overlayfs upper directory today.
Application support
As a result of our broad compatibility with POSIX, Archil is compatible with all applications which run on a local, POSIX file system. This includes:- Version control systems, such as Git (including large repositories,
git clone,git checkout, andgit status), Mercurial, and Subversion - Databases, such as PostgreSQL, MySQL, and SQLite
- Build tools and compilers, such as Make, CMake, GCC, and Clang
- Archive and compression tools, such as
tar,gzip,zip, andrsync - Editors and IDEs, such as Vim, Emacs, and VS Code
- Containers — mount in a container via a bind mount from the host or FUSE inside the container
Limits
- Path length — on a disk synchronized to a data source, a file’s full path can’t exceed the data source’s object-key length limit (1,024 bytes for Amazon S3, including the disk’s bucket prefix). Longer paths fail with
ENAMETOOLONG. - File names — on a disk synchronized to a data source, file and directory names must be valid UTF-8.