When designing a new application for the cloud, picking a storage layer shouldn’t be the hard part. Archil is designed to be the simplest storage solution that delivers good performance for the workloads people actually run in the cloud: applications, databases, model training, data pipelines, and the long tail of stateful systems that need a real file system underneath them. You don’t need to capacity-plan, choose between block storage and file storage, or run a replication or backup tier. Disks are mountable from any server, data lives in S3 in its native object format, and performance is good enough that you can stop thinking about your storage layer and focus on your application. For most workloads, Archil is the right default.Documentation Index
Fetch the complete documentation index at: https://docs.archil.com/llms.txt
Use this file to discover all available pages before exploring further.
When you might want something else
Archil is a network file system, so the cache-miss path always includes a network round-trip. A few cases where a different tool is a better fit:- You need microsecond latency on every operation regardless of working set size. Use local NVMe.
- You already operate a Lustre cluster with HPC tooling and storage admins. Stay on Lustre.
- Your application speaks S3 natively and doesn’t need POSIX or shared access across servers. Just use S3.
How Archil compares to other solutions
The sections below explain how Archil differs from each of the common alternatives in more detail.Block storage (e.g. EBS or Hyperdisk)
Block storage, such as EBS or Hyperdisk, are designed to provide high-performance, low-latency storage for your applications. While they provide full POSIX compatibility, they require manual capacity provisioning in advance, which can be challenging to predict accurately. They also cannot be shared between multiple servers and do not synchronize with object storage like S3. This means each server needs its own dedicated disk and must independently download data from S3, leading to redundant data transfers and higher costs. When you need to scale up your storage, you have to manually provision additional capacity and pay for the full provisioned amount regardless of actual usage.Native FUSE adapters (e.g. S3FS and Mountpoint)
Existing FUSE adapters, such as S3FS and Mountpoint for Amazon S3, are not designed to support full POSIX semantics. For example, Mountpoint for Amazon S3 does not support file locks, random writes, renames, or symbolic links. In order to use these tools, you need to instrument your file applications to understand whether or not they use these unsupported operations, which can be difficult and error-prone. In addition, these tools do not provide shared caching for reference data sets to multiple clients, so large clusters may need to re-download the same data from S3 repeatedly. In comparison, Archil is designed to support all standard POSIX file operations, which means that it’s out-of-the-box compatible with all file applications.Block FUSE adapters (e.g. JuiceFS and ObjectiveFS)
Other disk adapters for S3, such as JuiceFS, ObjectiveFS, or ZeroFS, require that you store data in your S3 bucket in a proprietary block format. This means that you cannot use your existing data sets in S3 with these tools, and these buckets aren’t compatible with any existing S3 workflows that you use. Archil stores data in S3 in its native format, so you can simultaneously use and share the data from both a disk and S3. You can also use Archil to connect to existing data sets in S3.NFS file storage (e.g. Amazon EFS and Amazon S3 Files)
NFS-based file systems, such as Amazon EFS and the newer Amazon S3 Files, are serverless and can be mounted from many servers at once, but the NFS protocol introduces two fundamental limitations that make them unsuitable for many workloads. The first limitation is metadata latency. NFS clients cannot cache the result of mutating metadata operations, so every file creation, rename, or attribute change must round-trip to the server before it can be acknowledged. As a result, metadata-heavy workloads — small files, source trees, package installs, container image extracts, and most interactive applications — are typically an order of magnitude slower on NFS than on local or block storage like EBS. Archil’s custom file protocol allows clients to take exclusive write access to portions of the file system, so creates and writes can be acknowledged locally without a server round-trip. This delivers metadata latency comparable to EBS or local NVMe, even for the small-file workloads that NFS handles poorly. The second limitation is per-client throughput. NFS clients resolve DNS once at mount time and send every subsequent request to a single IP address, which caps a single client at roughly 3 GiB/s regardless of how much network bandwidth the underlying instance has. This is well below the NIC capacity of large GPU instances, making NFS a bottleneck for model training and other high-throughput workloads. Archil’s protocol allows a single client to talk directly to multiple storage servers in parallel — similar to how parallel file systems like Lustre achieve scale-out throughput — so a single client can saturate the NIC of even the largest GPU instances. Archil also stores data in S3 in its native object format, so the same data is accessible through both the file system and the underlying S3 bucket without any proprietary on-disk layout.Shared file storage (e.g. Alluxio and FSx for Lustre)
Some solutions, like Alluxio or FSx for Lustre, require that you either deploy and manage your own infrastructure or that you provision capacity. It’s often difficult to predict the specific storage needs of an application, especially if the application can have unexpected or bursty needs. Archil is a fully managed, serverless solution that automatically scales with your application. You don’t need to provision capacity or deploy infrastructure, and you’re only billed for what you use.To estimate your costs, try the pricing calculator.