Sharing Disks

Archil supports sharing disks between multiple connected users simultaneously. This easily allows you to support workflows where multiple agents are collaborating on the same dataset, or if multiple servers need to have live access to the same data set. Archil delivers the local-like disk performance by matching the semantics of xfs, ext4, or zfs and enabling connected clients to execute file system operations locally. This allows Archil to exceed the performance of comparable NFS or 9p systems, in which file system operations must always be performed on the remote server. Archil requires that clients hold a write-lock, called a “delegation”, in order to operate in this higher-performance mode. In the fullness of time, Archil will dynamically assign and manage delegations for optimal performance. Today, users must manually manage these delegations. Archil supports multiple mount modes, which change how delegations work.

Exclusive mode - By default (archil mount), mounting an Archil disk puts it in exclusive mode. This locks the entire file system for the machine which mounted. That machine will receive local-like performance that matches an EBS volume, but other machines will not be able to access the disk until its unmounted.
Shared mode - If you mount the disk in shared mode (archil mount --shared), no lock is taken out on the disk. By default, the entire disk will appear read-only to the client until it manually requests a delegation (archil checkout <path>). With a delegation, the client will have exclusive write access to the specified <path> and receive local-like performance.
Conditional mode - If you mount the disk in conditional mode (archil mount --conditional), you enable the disk to execute operations remotely, like an NFS or 9p file system. This means that all files (which are not write-locked by another client) are immediately writeable, but the disk performance will be reduced to that of a remote file system. Users in conditional mode can continue to use archil checkout to get exclusive write access and higher performance to subdirectories on the disk.

We intend to make Conditional mode the default for new disks soon. The remainder of this article will describe how Shared mode and delegations work.

Selecting a mode on serverless sandboxes

By default, serverless sandboxes mount in Shared mode (archil mount --shared) to enable multiple sandboxes to easily access the same data. You can change the disk into Exclusive mode by first executing archil checkout /mnt/archil as part of your serverless sandbox command. You can select Conditional mode by using the more verbose serverless sandbox API, like below:

archil.exec({
  data: {
    disk: "dsk-def456",
    conditional: true,
  },
});

In this example, the disk will be mounted to /mnt/archil/data instead of /mnt/archil.

Automatic delegations in shared mode

When mounting in Shared mode, some operations automatically grant a write-lock on the file, to reduce the number of times that you need to call archil checkout. These operations are:

Creating a new file inside of a directory that does not have a delegation
Opening a file that is not covered by a delegation for writing (delegation automatically released on close)
Creating a new directory

These operations will automatically grant a delegation to the client who ran the operation. If you want other clients to have write-access to these resources, you will need to manually call archil checkin on the resource after creation. These rules lead to a common pitfall with Archil in which it’s possible to create a file or directory, but a removal operation will fail with EROFS: Read-only file system. This is because of the automatic delegation granting on file/folder creation. Deleting a file or folder is a directory-level operation, and requires a delegation on the parent directory of the file being removed.

These rules apply only to clients in Shared mode. Clients mounting in Conditional mode will never have delegations automatically issued to them.

Understanding delegations

Each delegation is an exclusive write-lock on a part of an Archil disk for a single client. When you call archil checkout <path>, all files and folders underneath that path will be included as part of the write delegation. This will prevent other write operations, including S3 PutObject operations, from completing against those paths. For example, consider a file system with the following structure:

/
├── group/
│   └── models/
│       ├── model1.txt
│       └── model2.txt
└── data/
    └── data.txt

If one client calls archil checkout group/ to get exclusive write-access to the “group” directory, it will also have exclusive write access to the group/models/ directory and its contents. Other clients will be able to continue to read these files and folders but will not be able to write to them, or call PutObject. Unless it’s mounted in conditional mode, this client would not be able to edit any other part of the disk. Other clients are simultaneously able to call PutObject to add files to the data/ directory or get exclusive write access over that directory by calling archil checkout data/. When your client is operating in conditional mode, write operations are executed on the server-side, and multiple clients can simultaneously read and write to any part of the file system without a delegation.

Releasing delegations

Delegations represent an exclusive write-lock on a part of the Archil disk. As a result, they need to be manually released by clients to signal that they are no longer performing writes to that portion of the disk. You can do this by either unmounting the disk from the server which requested the delegation or by manually calling archil checkin <path>.

Forcing ownership

In some cases, a client may crash or poweroff without cleanly unmounting or calling archil checkin to release a delegation. If this happens, you will not automatically be able to write to the location where the client’s delegation was held. Instead, you will need to force release the delegation. This will cause any in-progress writes from the original client to be lost. This mirrors “force detaching” a block device, such as an EBS or Hyperdisk disk. You can force checkout a delegation by calling archil checkout <path> --force, and you can force mount a disk into exclusive mode by calling archil mount --force. You can also remove delegations using the Archil API, if you don’t have the disk mounted locally:

Typescript

const disk = await archil.getDisk("dsk-abcdef")

for (const delegation of await disk.listDelegations()) {
  // If the client which issued the delegation is no longer
  // connected, revoke the delegation.
  if (delegation.isOrphaned) { 
    await disk.revokeDelegation(delegation)
  }
}

Getting started

Compute

Concepts

Data Sources

Details

Administration

Protocols

Reference

Legal

Selecting a mode on serverless sandboxes

Automatic delegations in shared mode

Understanding delegations

Releasing delegations

Forcing ownership

​Selecting a mode on serverless sandboxes

​Automatic delegations in shared mode

​Understanding delegations

​Releasing delegations

​Forcing ownership

Selecting a mode on serverless sandboxes

Automatic delegations in shared mode

Understanding delegations

Releasing delegations

Forcing ownership