# Best practices Source: https://docs.archil.com/administration/best-practices Recommended ways to structure your Archil accounts, disks, and credentials for production and multi-tenant workloads. This page collects recommendations for running Archil in production — how to structure accounts, disks, and credentials so that workloads and tenants stay isolated, and billing stays manageable as you scale. ## Use a disk as your unit of isolation A disk is Archil's unit of access isolation. Credentials — whether [disk tokens or IAM authorization](/concepts/disk-users) — are scoped to an entire disk, including all of its [branches and checkpoints](/concepts/branches-and-checkpoints). Anyone who can mount a disk can read everything on it, and POSIX permissions are not a security boundary between untrusted users. So when you need to keep workloads or customers from accessing each other's data: * **Give each tenant its own disk** (or set of disks) rather than sharing one disk across tenants, and issue per-tenant, disk-scoped [disk tokens](/concepts/disk-users). * **Don't use branches as a security boundary.** A branch is an alternate history of the same disk, not an isolated disk — it shares the parent disk's credentials. * **Don't rely on POSIX permissions** to separate untrusted tenants on a shared disk. ## Give each agent or job its own fork For AI-agent and CI fan-out workloads, run each agent or job against its own writable [branch](/concepts/branches-and-checkpoints) — or its own disk when the work is untrusted. Branches are copy-on-write and fully independent: each can be mounted on a different VM, writes are isolated from the parent and siblings, and you're only billed for the unique data each fork adds (see [Metering](/concepts/metering)). This gives every agent a clean, isolated working copy without duplicating the base data set. When forks must be isolated from each other for security — for example, one end customer per fork — give each its own disk instead, since a branch shares the parent disk's credentials. ## Structure accounts and billing Use [organizations](/administration/organizations) to group disks by team or environment and to keep billing separate from personal usage. A common setup is a separate **Dev** and **Production** organization, so experimental disks and credentials stay isolated from production data and you can manage access and billing for each independently. If you operate Archil on behalf of your own customers, you can likewise map multiple Archil organizations to your internal boundaries: multiple organizations can be billed under a single contract, so splitting work across organizations — whether by environment or by tenant — doesn't fragment your billing. [Contact us](mailto:support@archil.com) to set up a single contract across multiple organizations. # Pricing & Billing Source: https://docs.archil.com/administration/billing Official Archil pricing: Free, Developer ($10 monthly minimum), Team ($500 monthly minimum), and Enterprise plans, with usage rates for storage ($0.20/GiB-month active), sandbox compute ($0.055/vCPU-hour, $0.020/GiB-hour), serverless tool calls ($0.05 per 1,000), and data transfer. Archil offers Free, Developer, Team, and Enterprise plans. Paid plans have a monthly minimum spend, and metered usage counts toward that minimum. This page and the [pricing page](https://archil.com/pricing) are the official sources for Archil's rates; the pricing page also has the full plan comparison. ## Plans and limits | | Free | Developer | Team | Enterprise | | - | - | - | - | - | | Monthly minimum | \$0 | \$10 | \$500 | Custom | | File systems | 5 | 1,000 | 25,000 | Unlimited | | Maximum size per disk | 10 GB | Unlimited | Unlimited | Unlimited | | Concurrent sandboxes | 5 | 20 | 250 | Unlimited | | Workspaces | 1 | 1 | 3 | Unlimited | Disks on every plan use Archil-managed storage by default, so you don't need a bucket of your own. Free requires no credit card and includes 1,000 serverless tool calls and 1 hour of sandbox compute per month: 1 vCPU-hour and 2 GiB-hours, which is one hour of a sandbox at the default size of 1 vCPU and 2 GiB. A Free sandbox session lasts at most 30 minutes. Its usage limits are hard caps, with no overage. Free does not include bring your own bucket (BYOB) or external mounts. Team customers can add capacity in blocks of 500 concurrent sandboxes, with each additional block costing \$500 per month. ### How the monthly minimum works Your metered usage counts toward your plan's monthly minimum; the minimum is not a subscription fee added to usage. For example, $6 of Developer usage results in a $10 bill, while $18 of usage results in an $18 bill. Enterprise has a custom monthly minimum and custom usage pricing. ## Usage rates These rates apply to Developer and Team. Enterprise pricing is custom, and Free's included usage remains free within its caps. | Resource | Rate | | - | - | | Active storage | \$0.20 / GiB-month | | Archive storage (Archil-managed disks only) | \$0.025 / GB-month | | Serverless tool calls | \$0.05 / 1,000 calls | | Sandbox CPU | \$0.055 / vCPU-hour | | Sandbox memory | \$0.020 / GiB-hour | | Data transfer out of Archil | \$0.07 / GB | | Data transfer within Archil | Free | ### Storage Active storage is the time-weighted average of data in Archil's high-speed storage layer. See [Metering](/concepts/metering) for how active data is measured. Archive storage covers all of the data an Archil-managed disk stores in the bucket Archil operates for it, whether or not that data is active. It takes the place of the object storage bill you would otherwise pay a cloud provider, so it applies only to file systems that do not synchronize to customer-owned buckets. For BYOB, you pay your cloud provider for the data in your bucket instead. Active storage is billed on top of archive storage, for the data in the high-speed layer. For example, an Archil-managed disk that holds 100 GB, of which 5 GiB is active on average over the month, uses $2.50 of archive storage and $1.00 of active storage. ### Serverless tool calls Each `disk.exec()` invocation, `disk.grep()` search request, or SQLite query counts as one call. An `exec` invocation still counts as one call when it runs several Bash, Python, or Node commands; a search request counts as one call regardless of how many workers it uses. Execution is included in the per-call rate, with no additional CPU or memory charge for that call. For example, 100,000 calls use $5 of your monthly minimum and 1 million calls use $50. An `exec` that [asks for more than the default 2 vCPU and 4096 MiB](/compute/serverless-execution#choosing-a-size) counts as several calls: its size as a multiple of the default, by whichever of vCPU and memory is the larger multiple, rounded up. For example, 4 vCPU and 4096 MiB counts as two calls, and 8 vCPU and 16,384 MiB as four. The Free plan runs every `exec` at the default size. Storage and applicable data transfer are metered separately. Model inference is not included. ### Persistent sandboxes and services [Persistent sandboxes](/compute/sandboxes/introduction) and services use the CPU and memory rates above, metered separately. Both count toward your monthly minimum. These rates do not apply to serverless tool calls, which use the per-call rate. Compute is metered on the vCPUs and memory a running sandbox is allocated, sampled each minute, whatever the sandbox actually uses. A sandbox with 1 vCPU and 2 GiB of memory costs \$0.095 an hour. Stopping or pausing a sandbox stops compute charges; its disk data remains subject to [storage pricing](#storage). [Get Sandbox Usage](/api-reference/sandboxes/get-sandbox-usage) returns a sandbox's vCPU-hours, GiB-hours, and their cost since it was created. ### Data transfer On Developer and Team, transfer within Archil is free, including transfer to Archil tool calls and sandboxes. Data leaving Archil, including delivery to external compute, costs \$0.07 per GB. Enterprise transfer pricing is custom. Enterprise customers can use private networking to avoid Archil egress charges. ## Plan features All plans include multi-AZ durability and community support. * **Free:** Archil-managed disks, serverless tool calls, and sandboxes, within the limits above. Sandboxes reach a [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist) of package registries, Git hosts, container registries, OS package mirrors, and model provider APIs, and all other egress is denied. * **Developer, Team, and Enterprise:** BYOB, external mounts, advanced observability, private sandbox images, and custom sandbox [network policies](/compute/sandboxes/firewall), including deny-all egress, domain and CIDR allowlists, and [credential brokering](/compute/sandboxes/firewall#credential-brokering). * **Team and Enterprise:** SOC 2 compliance documentation and a shared Slack support channel. * **Enterprise:** checkpoints and branching for BYOB, private networking via PrivateLink or VPC peering, advanced API key permissions, SSO and SCIM, HIPAA, [BYOC](/reference/byoc) and on-premises deployment, 24/7 support, and an availability SLA. DPA and BAA are available. ### External mounts An external mount is a disk mounted on compute outside Archil, such as your own servers, a Kubernetes cluster, or another sandbox provider like E2B or Daytona. External mounts are available on Developer and above. Usage counts toward Developer's $10 monthly minimum, so a light external workload costs $10 a month. Data read out to external compute is billed as data transfer out of Archil, at \$0.07 per GB on Developer and Team. Transfer to Archil sandboxes and serverless tool calls is free. ### Checkpoints on BYOB disks [Checkpoints and branches](/concepts/branches-and-checkpoints) need Archil's flat-block storage format, which Archil-managed disks use in AWS regions on every plan. On Developer and Team, BYOB disks use the native format instead, which writes each file to your bucket as an ordinary object that any S3 tool can read. Native-format disks can't be checkpointed. Enterprise can create BYOB disks in the flat-block format. On Developer and Team, use an Archil-managed disk for workloads that need checkpoints and branches. ### Networking Free, Developer, and Team use public networking. Enterprise supports public networking, PrivateLink, or VPC peering. [Contact sales](https://cal.com/team/archil/demo) for Enterprise pricing and deployment options. ## Monitoring usage ### Per-disk usage Each disk in the [Archil console](https://console.archil.com) has a **Usage** tab showing a real-time chart of active data over time. You can view usage across multiple time ranges (6 hours, 24 hours, 7 days, 30 days, or all time). ### Account-wide billing The [Billing](https://console.archil.com/billing) page provides a unified view across all disks and regions: * **Current active data** per disk * **Month-to-date average** and **projected monthly cost** per disk and in total * **Past billing cycles** with per-disk breakdowns and links to Stripe invoices ### Programmatic access | Endpoint | Answers | | - | - | | [`GET /api/billing/subscription`](/api-reference/billing/get-subscription) | What plan am I on? The plan, the subscription's status and billing period, any scheduled change, and whether a payment is due. | | [`GET /api/billing/usage`](/api-reference/billing/get-usage) | How much of each resource did I use, and what is it worth at my rates? Quantities per meter (vCPU-hours, GiB-hours, GiB-months, tool calls) per day or per hour, each priced at list, before credits or your base fee. | | [`GET /api/billing/summary`](/api-reference/billing/get-summary) | What does this period's statement come to? The `lines` billing has invoiced or previews for the period, `billedCost`, their sum, and the `credits` you have left. | | [`GET /api/sandboxes/{sid}/usage`](/api-reference/sandboxes/get-sandbox-usage) | How much compute did one sandbox use, and what is it worth at my rates? Its vCPU-hours and GiB-hours since it was created, the rates, and their cost before credits, updated about once a minute while it runs. | A `summary` response for an Enterprise contract with a \$12,500 base fee, 25,000 credits a month and 2,000,000 tool calls included, read mid-period. Money is in dollars. The credits drawn so far are a `credit` line on the statement; what is left of each grant, and when it refills, is `credits`. The tool calls ran past what is included, so billing invoices only the 400,000 beyond it and the grant shows as spent, with no `credit` line of its own: ```json theme={null} { "lines": [ { "type": "fixed", "name": "Enterprise (from 16 Oct 2026 to 16 Nov 2026)", "start": 1792166400, "end": 1794844800, "amount": 12500.00, "invoiceId": "inv_1ab2" }, { "type": "usage", "featureId": "sandbox_vcpu_hours", "name": "Sandbox compute (vCPU-hours)", "quantity": 163545.24, "unit": "vCPU-hours", "start": 1792166400, "end": 1794843000, "amount": 8242.72 }, { "type": "usage", "featureId": "serverless_tool_calls", "name": "Serverless tool calls, 2,400,000 units", "quantity": 2400000, "unit": "calls", "start": 1792166400, "end": 1794843000, "amount": 20.00 }, { "type": "credit", "featureId": "prepaid_credit", "name": "Credits applied", "quantity": 1, "unit": "USD", "start": 1792166400, "end": 1794843000, "amount": -8242.72 } ], "billedCost": 12520.00, "credits": [ { "featureId": "prepaid_credit", "name": "Prepaid credit (USD)", "unit": "USD", "granted": 25000, "remaining": 16757.28, "resetsAt": 1794844800, "expiresAt": null }, { "featureId": "serverless_tool_calls", "name": "Serverless tool calls", "unit": "calls", "granted": 2000000, "remaining": 0, "resetsAt": 1794844800, "expiresAt": null } ], "projected": { "billedCost": 12520.00, "dailyRate": 329.71 } } ``` Usage reaches billing from every region within a few minutes of the end of each reporting interval, so the newest interval can still grow after you read it. Responses are cached for a minute per account. `usage` takes a `timezone`, so "today" resets at your midnight. ## Managing your subscription You can manage your subscription, update payment methods, and view invoices through the **Billing** page in the console. Clicking **Manage Billing** opens the Stripe billing portal. # Organizations Source: https://docs.archil.com/administration/organizations Use Organizations to effectively share disks between members of your team Organizations let you share disks across your team, manage billing separately from personal usage, and delegate disk administration to team members. If you're working solo, you can skip this — personal accounts work for everything. Organizations become useful when multiple people need to create, configure, or mount the same disks. ## Creating an organization You can create an organization by clicking the "Personal account" dropdown in the top right corner of the screen and selecting "Create Organization". When you create an organization, the "slug" for that organization will become the prefix when mounting an organization-owned disk. For example, if you create an organization with the slug "my-org", you can mount an organization-owned disk with the name "my-org/my-disk". ## Managing disks in an organization When you create a disk in the Archil console, it will automatically be owned by the organization that is selected in the top-right corner of the screen. When you view the list of disks in your Archil account, the list will only show disks that are owned by the organization that is selected in the top-right corner of the screen. You can use the dropdown in the top-right corner of the screen to switch between your personal account and any organizations you belong to. Patterns like a disk per tenant, per workspace, or per agent work within your [plan's file system limit](/administration/billing#plans-and-limits). ## Managing members in an organization If you are the administrator of an organization, you can manage the members of the organization by clicking the "Manage" button next to the organization name in dropdown in the top-right corner of the screen, and navigating to the "Members" tab. Members can be assigned either a "Member" or an "Administrator" role. Members with the "Member" role have the ability to view and manage the disks in the organization, including deleting disks and managing the authorization of disk users. Members with the "Administrator" role have the same permissions as the "Member" role, but also have the ability to manage the organization itself, including adding and removing members. # Get Subscription Source: https://docs.archil.com/api-reference/billing/get-subscription GET /api/billing/subscription Returns the account's plan and the state of its subscription: the plan, the subscription's status and billing period, a scheduled change or cancellation if one is pending, payment health (a default payment method on file, an open invoice and when it is retried) and the actions the console offers for it. Accounts that share billing share the subscription and open-invoice information. The plan here is the one the `usage` and `summary` routes price usage under. # Get Summary Source: https://docs.archil.com/api-reference/billing/get-summary GET /api/billing/summary Returns a billing period's statement as billing itself states it: `lines`, each line invoiced for the period or previewed for its end (a fixed fee, a meter's usage billed past its grants, a credit taken off, anything else under its own name), and `billedCost`, their sum, which is what the period invoices or would if it ended now. On the current period also `credits`, the grants the account still draws on, and a projection to the period end at the recent daily pace. Nothing here is derived from the plan's terms: a plan billing can express, this route reports. Money is in dollars to the cent, before tax. A line is before discounts, as billing reports it, and what a reward takes off follows as an `other` line named `Discount`: off each line previewed for the period end, and off each invoice as a whole (its total under its lines' sum), on the span of the invoice's fixed fee when it has one, else of its first line on the period. So the sum is what the period invoiced, or will. # Get Usage Source: https://docs.archil.com/api-reference/billing/get-usage GET /api/billing/usage Returns what the account's meters recorded over a window, for the window as a whole and per interval, in each meter's own unit (such as vCPU-hours, GiB-hours, GiB-months or calls), and what that usage comes to at the account's rates, per meter and in total, before any grant and fixed fee: the cost of what was used, not what will be invoiced, which `/api/billing/summary` reports. The meters are the catalog's metered features, so a meter added there is reported from then on. A meter's rate is the price of the plan item its usage is billed from: the meter's own item, or the credit system whose rate card names it, at the plan's own prices and overrides; a meter the plan does not price has a zero rate, and a tiered item is priced at its first tier. Usage reaches billing once per reporter interval from every region, so an interval's figures can still grow for a few minutes after it ends, longer if a region's reporter is behind. The default window is the seven days (hourly: 24 hours) up to now; a window may be at most 31 days, and is widened to whole intervals. Money is in dollars, to the cent. # Add Disk User Source: https://docs.archil.com/api-reference/disk-users/add-user POST /api/disks/{id}/users Adds an authorized user to a disk. Users can authenticate via: - **token**: A shared token with a nickname and 4-character suffix - **awssts**: AWS STS role assumption with an IAM principal ARN - **s3**: An access key pair for the disk's [S3 API](/protocols/s3-api). The server generates both keys and returns the secret key only in this response. The keys reach the whole disk, read-write unless `readOnly` is set, so keep them on your server and give browsers and other untrusted clients presigned URLs instead. # Remove Disk User Source: https://docs.archil.com/api-reference/disk-users/remove-user DELETE /api/disks/{id}/users/{userType} Removes an authorized user from a disk. # Create Disk Source: https://docs.archil.com/api-reference/disks/create-disk POST /api/disks Creates a new disk with the specified configuration. A default disk token user is automatically generated and returned in the response, so the disk is immediately mountable. The one-time disk token appears in `authorizedUsers[].token` and cannot be retrieved again. (This disk token is scoped to the new disk and is separate from the API key you used to make this request.) To provide your own users instead, pass the deprecated `authMethods` field or call AddDiskUser after creation. # Create Share URL Source: https://docs.archil.com/api-reference/disks/create-share-url POST /api/disks/{id}/share Creates a signed, time-limited URL for downloading a file without authentication. The URL expires after 24 hours by default; pass `expiresIn` to set a custom lifetime up to 7 days. The returned URL points to a public endpoint (`GET /api/shared/{token}`) that verifies the signature, checks expiry, and streams the file with a sensible `Content-Type` and `Content-Disposition`. Range requests are supported. No credentials are needed to open the URL. # Delete Disk Source: https://docs.archil.com/api-reference/disks/delete-disk DELETE /api/disks/{id} Deletes a disk and all associated resources. # Get Disk Source: https://docs.archil.com/api-reference/disks/get-disk GET /api/disks/{id} Returns detailed information about a specific disk. # List Disks Source: https://docs.archil.com/api-reference/disks/list-disks GET /api/disks Returns all disks owned by the authenticated account. # Prefetch File Ranges Source: https://docs.archil.com/api-reference/disks/prefetch-disk POST /api/disks/{id}/prefetch Warms byte ranges of files on a disk into Archil's storage cache, so later reads from any client, serverless exec, or sandbox are served without going to the backing object storage. Each range names one file and an optional `offset` and `length`; a file may appear in several ranges. Paths are relative to the disk root. Ranges must name regular files: a directory is found but warms nothing, so list a directory's files to warm its contents. The response reports one result per range, in request order, and may return before the data is fully fetched. A path that does not exist reports `found: false`. A failed range does not fail the others; retry the ranges whose `error` is `retry`. If the disk is not mounted anywhere, prefetching loads it. Warmed data becomes active data and is metered like data your clients read. # Build Image Source: https://docs.archil.com/api-reference/images/build-image POST /api/images Builds an OCI image for use by sandboxes. Poll [Get Image](/api-reference/images/get-image) until it is `ready`, then pass its `image_id` when creating a sandbox. Repeated requests join an active build. Completed tags rebuild; completed digest references are reused. During a rebuild, sandboxes can use the last successful digest. Submit the request again to retry a failed or unresponsive build. # Get Image Source: https://docs.archil.com/api-reference/images/get-image GET /api/images/{image_id} Returns the image's build status and latest successful digest, if available. Failed builds include a `failure_reason`. Use [Build Image](/api-reference/images/build-image) (`POST /api/images`) to retry interrupted builds. # API Reference Source: https://docs.archil.com/api-reference/introduction Programmatic access to the Archil Control Plane The Archil Control Plane API provides programmatic access to manage disks, mounts, and API keys in the Archil distributed filesystem platform. > [!NOTE] > > **API keys vs. disk tokens.** API keys authenticate requests to this Control Plane API and are scoped to your account. [Disk tokens](/concepts/authorization#token-based-disk-authorization) are a separate kind of credential, scoped to a single disk, used by clients when mounting. The two are not interchangeable. ## Base URL Each region has its own control plane endpoint: | Region | Region ID | Endpoint | | - | - | - | | AWS US East (N. Virginia) | `aws-us-east-1` | `control.green.us-east-1.aws.prod.archil.com` | | AWS EU West (Ireland) | `aws-eu-west-1` | `control.green.eu-west-1.aws.prod.archil.com` | | AWS US West (Oregon) | `aws-us-west-2` | `control.green.us-west-2.aws.prod.archil.com` | | GCP US Central (Iowa) | `gcp-us-central1` | `control.blue.us-central1.gcp.prod.archil.com` | ## Authentication All endpoints require an API key, sent as a bearer token: ``` Authorization: Bearer key-... ``` Use the full key, including its `key-` prefix. The older form without the `Bearer` scheme (`Authorization: key-...`) is still accepted. API keys are specific to the region they were created in, so send requests to that region's control plane host. Create API keys in the [Archil Console](https://console.archil.com) or via the API. ## Response format All responses use a consistent envelope: ```json Success Response theme={null} { "success": true, "data": { ... } } ``` ```json Error Response theme={null} { "success": false, "error": "Error message" } ``` ## HTTP status codes | Code | Description | | - | - | | `200` | Success | | `202` | Accepted; an asynchronous operation started | | `400` | Bad request (validation error) | | `403` | Forbidden (auth failure or access denied) | | `404` | Not found | | `409` | Resource state conflict | | `429` | Capacity or concurrency limit reached | | `500` | Internal server error | | `503` | Feature unavailable or lifecycle transition temporarily blocked | ## Disk ID format Disks are identified by a unique ID in the format: ``` dsk-{16 hex characters} ``` For example: `dsk-0123456789abcdef` ## Getting started > [!NOTE] > > Signing up for Archil is free with no credit card required. [Create your free account](https://console.archil.com) to get an API key and start building. 1. [Create an API key](/api-reference/tokens/create-token) in the console or via API 2. Use the key in the `Authorization` header for all requests 3. [Create your first disk](/api-reference/disks/create-disk) or [persistent sandbox](/api-reference/sandboxes/create-sandbox) # Create Sandbox Source: https://docs.archil.com/api-reference/sandboxes/create-sandbox POST /api/sandboxes Provisions a sandbox VM with the requested shape and an internal, persistent Archil disk backing its root filesystem. This backing disk cannot be disabled or replaced with an existing disk. Use `mounts` to attach additional existing disks at guest paths; omit it or pass an empty list to create a sandbox without additional disks. By default the response reports `pending` after the runtime accepts the start. Set `wait=true` to hold for `running`; if the wait budget expires, the response remains `pending` and startup continues. # Create Sandbox Connection Source: https://docs.archil.com/api-reference/sandboxes/create-sandbox-connection POST /api/sandboxes/{sid}/connections Returns a short-lived WebSocket URL used by the sandbox process API. The URL contains a signed token that expires at `expires_at`; an open WebSocket can remain connected after the token expires. # Create Sandbox Port Token Source: https://docs.archil.com/api-reference/sandboxes/create-sandbox-port-token POST /api/sandboxes/{sid}/port-tokens Creates token-authorized HTTP access to one port. Send the raw token returned by this request in the `X-Archil-Token` request header at the stable `-.` hostname. Public ports accept connections without a token; otherwise a valid token for that sandbox and port is required. The token is not returned again. # Delete Sandbox Source: https://docs.archil.com/api-reference/sandboxes/delete-sandbox DELETE /api/sandboxes/{sid} Permanently deletes a stopped, exited, or failed sandbox and its backing disk. A sandbox with child forks cannot be deleted until those children are deleted, starting with the deepest ones. # Delete Sandbox Port Token Source: https://docs.archil.com/api-reference/sandboxes/delete-sandbox-port-token DELETE /api/sandboxes/{sid}/port-tokens/{token_id} New private connections using the token fail after this request completes; established connections remain open. Expired tokens return not found. # Expose Sandbox Port Source: https://docs.archil.com/api-reference/sandboxes/expose-sandbox-port PUT /api/sandboxes/{sid}/ports/{port} Creates or returns public access to a TCP port in the sandbox. # Fork Sandbox Source: https://docs.archil.com/api-reference/sandboxes/fork-sandbox POST /api/sandboxes/{sid}/fork Checkpoints a running source sandbox and creates an isolated writable disk branch. Forks from running or paused sources resume from the source's CPU, memory, and disk state. Forks from stopped sources cold-boot from the saved disk checkpoint. A running source is resumed after its checkpoint; paused and stopped sources remain inactive. Forked sandboxes may themselves be forked. Passing `checkpoint` forks that saved state without touching the source, so clients can pause once, wait for the checkpoint, and fork it from any number of places. Without `checkpoint` the server pauses a running source inside the request and resumes it afterwards; that path is kept for older clients and is bounded by the request timeout. Sandboxes with additional disk mounts must be stopped before forking. The fork inherits mounts of the same external disks; only the internal root disk is branched, so external disk contents are not isolated. # Get Sandbox Source: https://docs.archil.com/api-reference/sandboxes/get-sandbox GET /api/sandboxes/{sid} # Get Sandbox Network Source: https://docs.archil.com/api-reference/sandboxes/get-sandbox-network GET /api/sandboxes/{sid}/network Returns the last committed network policy for a running, paused, or stopped sandbox. # Get Sandbox Port Token Source: https://docs.archil.com/api-reference/sandboxes/get-sandbox-port-token GET /api/sandboxes/{sid}/port-tokens/{token_id} Returns token metadata without its raw token. Expired tokens return not found. # Get Sandbox Usage Source: https://docs.archil.com/api-reference/sandboxes/get-sandbox-usage GET /api/sandboxes/{sid}/usage Returns the vCPU-hours and GiB-hours a sandbox has been billed for since it was created, and their cost at the account's rates before included allowances and prepaid credit. The figures update about once a minute while the sandbox runs; paused and stopped time is free. Usage stays readable for 90 days after the sandbox is deleted. # List Sandbox Port Tokens Source: https://docs.archil.com/api-reference/sandboxes/list-sandbox-port-tokens GET /api/sandboxes/{sid}/port-tokens Lists unexpired private port token metadata without returning raw tokens. # List Sandbox Ports Source: https://docs.archil.com/api-reference/sandboxes/list-sandbox-ports GET /api/sandboxes/{sid}/ports Lists the explicitly exposed public ports for the sandbox. # List Sandboxes Source: https://docs.archil.com/api-reference/sandboxes/list-sandboxes GET /api/sandboxes All the account's sandboxes, oldest first. # Pause Sandbox Source: https://docs.archil.com/api-reference/sandboxes/pause-sandbox POST /api/sandboxes/{sid}/pause Snapshot the CPU & memory state, then stop the VM. The sandbox reports `pausing` until the snapshot commit marker is durable and the VM exits, then reports `paused`. Attached process connections close, while agent-owned processes remain available for reattachment after resume. A sandbox that is still `pending` must reach `running` before it can be paused. To restore the snapshot, call `/resume`. # Resume Sandbox Source: https://docs.archil.com/api-reference/sandboxes/resume-sandbox POST /api/sandboxes/{sid}/resume Restores a paused sandbox's committed CPU and memory snapshot. Resuming a running sandbox or a pending resume is idempotent. Only a `paused` sandbox can begin a new resume; `stopped`, `exited`, `failed`, `stopping`, `pausing`, or a pending cold start returns 409. # Set Sandbox Timeouts Source: https://docs.archil.com/api-reference/sandboxes/set-sandbox-timeout POST /api/sandboxes/{sid}/timeout Replaces one or both sandbox timeout settings. Updating `timeout` while the sandbox is running resets its hard expiration deadline to that many seconds from now, capped at 24 hours after the current session started. Updating `idle_ttl_seconds` resets the idle countdown when no direct process connection is active; zero disables idle expiry. Omitted settings remain unchanged. Updates are retained across later stops and starts. For an inactive sandbox, they apply to the next powered-on session. Sandboxes that are starting or shutting down, or whose timeout changes concurrently, return 409. # Start Sandbox Source: https://docs.archil.com/api-reference/sandboxes/start-sandbox POST /api/sandboxes/{sid}/start Cold-boots from the persisted configuration and disks. If the sandbox was paused, its memory snapshot is consumed and discarded before the new VM runs; disk state remains. Without mount changes, starting a running sandbox or a pending cold start is idempotent. A sandbox that is stopping, pausing, or pending a resume returns 409. Providing `mounts` replaces the additional disk mounts for this and later sessions and requires an inactive sandbox; omit it to preserve saved mounts. # Stop Sandbox Source: https://docs.archil.com/api-reference/sandboxes/stop-sandbox POST /api/sandboxes/{sid}/stop Gracefully shuts down the guest, then checkpoints its disk before entering `stopped`. The sandbox and any forks from it cold-boot from disk state. Stopping a paused sandbox reuses its existing disk checkpoint. In-flight processes receive the guest shutdown signal. # Unexpose Sandbox Port Source: https://docs.archil.com/api-reference/sandboxes/unexpose-sandbox-port DELETE /api/sandboxes/{sid}/ports/{port} Removes an explicitly exposed public port. An enabled sandbox service publishing the same TCP port remains publicly reachable. # Update Sandbox Network Source: https://docs.archil.com/api-reference/sandboxes/update-sandbox-network PUT /api/sandboxes/{sid}/network Applies the complete replacement policy to a running sandbox, then records it as the current policy. A rejected update leaves the previous policy unchanged. Free-plan accounts cannot update network policies and receive 403 with code sandbox_network_plan_required. For paid accounts, an empty object restores unrestricted egress. Existing network sessions are not terminated; the replacement governs new connections and is retained across later stops and resumes. Sandboxes that are not running, or whose policy changes concurrently, return 409. # Execute Command on Disk Source: https://docs.archil.com/api-reference/serverless-execution/exec-disk POST /api/disks/{id}/exec Launches a Firecracker microVM with the specified disk mounted, runs the given command to completion, shuts down the microVM, and returns the command result. # Execute Command with Multiple Disks Source: https://docs.archil.com/api-reference/serverless-execution/exec-multi POST /api/exec Launches a Firecracker microVM with the supplied set of disks each mounted at its own relative path under `/mnt/archil`, runs the command to completion, and shuts down the microVM. Activation is atomic: every disk mounts or none of them do. Relative paths must be non-empty, non-absolute, and contain no `.` / `..` segments. Mounting two disks at the same relative path is an error. # Search Files (grep) Source: https://docs.archil.com/api-reference/serverless-execution/grep-disk POST /api/disks/{id}/grep Searches files under a directory on the disk for lines matching a regular expression. Listing and matching are fanned out across ephemeral exec microVMs, so the request finishes within the user's time budget regardless of the size of the directory. The user controls search latency and scope with three knobs: - `maxDurationSeconds` is the wall-clock deadline (capped at 30s). - `concurrency` is the maximum number of parallel grep workers. Higher concurrency finishes larger datasets within the deadline, at proportionally more compute. - `maxResults` causes the search to short-circuit after the aggregator has collected this many matches. With `recursive: false` only files directly under `directory` are searched. With `recursive: true` subdirectories are walked breadth-first and grep workers are dispatched as soon as each level finishes listing — listing and matching overlap. Matching runs on ripgrep. Narrow the files searched with `globs`, `types`, and `typesNot`; change matching with `ignoreCase` and `multiline`; and ask for surrounding lines with `contextBefore` and `contextAfter`, which adds `groups` to the result. The matches returned when stopping early on `maxResults`, or on the 32 MiB limit on match and context text, are a sample of whichever workers reported first, not the lexicographically first N. The response surfaces `stoppedReason` so callers can distinguish completion from early termination. # Create API Token Source: https://docs.archil.com/api-reference/tokens/create-token POST /api/tokens Creates a new API token for programmatic access. The full token value is only returned once at creation time. # Delete API Token Source: https://docs.archil.com/api-reference/tokens/delete-token DELETE /api/tokens/{id} Revokes and deletes an API token. # List API Tokens Source: https://docs.archil.com/api-reference/tokens/list-tokens GET /api/tokens Returns all API tokens for the authenticated account. # Sandbox firewall Source: https://docs.archil.com/compute/sandboxes/firewall Network egress filters for sandboxes. Archil's persistent sandboxes support defining a network policy to filter network egress from the sandbox. This is useful when running untrusted workloads, including agents. > [!NOTE] > > Network policies are available on the Developer, Team, and Enterprise plans, including deny-by-default policies and [credential brokering](#credential-brokering). A sandbox created without a policy on these plans has unrestricted egress. > > On the Free plan, every sandbox uses a [managed allowlist](#free-plan-allowlist): egress is denied by default, except to package registries, Git hosts, container registries, OS package mirrors, and model provider APIs. You can't set your own policy on Free. The sandbox firewall operates on both the IP and transport layer, so you can filter both CIDR ranges and domains. To deny all network egress you can set the `default` to `deny`. On top of this, you can apply additional `deny` and `allow` rules to whitelist certain targets. For example: ```typescript theme={null} import * as archil from "disk"; const sandbox = await archil.createSandbox({ network: { egress: { default: "deny", allow: ["192.168.14.0/24", "*.github.com"], } } }); ``` ## Domain filtering Domain rules match hostnames, which the firewall reads from HTTP and TLS traffic on any port. Once a policy has a domain rule, connections that don't carry a hostname, such as SSH or database connections, are blocked unless you allow their IP address or CIDR, even with `default: "allow"`. HTTP/3 is blocked when domain filters or [`drain_on_pause`](#draining-requests-before-pause) selectors are in place. For HTTPS, the firewall reads the server name (SNI) from the TLS ClientHello and passes an allowed connection through still encrypted. For plain HTTP, it checks the `Host` header and the URI authority of every request. If the name is denied, the connection or request is rejected. Plain `allow` and `deny` domain rules do not decrypt TLS, so they need no extra certificate in the sandbox. ## TLS inspection Some rules act on the HTTP requests themselves, so the egress proxy has to decrypt them. It terminates TLS only for hosts that match one of these: * An allow rule with a `transform` that adds headers ([credential brokering](#credential-brokering)). * An allow rule with a `forward_url` ([egress forwarding](#egress-forwarding)). * A [`drain_on_pause`](#draining-requests-before-pause) selector. For those hosts, the proxy decrypts HTTPS connections that offer HTTP in the TLS handshake (or that offer no application protocol on port 443), and checks both the SNI and the authority of each request. It does not decrypt traffic to any other host. When a policy includes one of these rules, Archil installs an "Archil egress" CA certificate into the sandbox so that clients trust the proxy. This happens when you create the sandbox with such a policy or when an [update](#dynamic-updates) adds one, and the CA stays installed across resumes and forks. Commands started after the CA is installed get `SSL_CERT_FILE`, `REQUESTS_CA_BUNDLE`, `CURL_CA_BUNDLE`, and `AWS_CA_BUNDLE` pointing at the system bundle (`/etc/ssl/certs/ca-certificates.crt`), which includes the Archil CA, and `NODE_EXTRA_CA_CERTS` pointing at the Archil CA (`/usr/local/share/ca-certificates/archil-egress.crt`). Values you set yourself take precedence, so a custom bundle must include the Archil CA. ## Free plan allowlist On the Free plan, every sandbox uses a managed policy: egress is denied by default, except to these destinations: * **Package registries** for Python (PyPI), JavaScript (npm, Yarn, jsDelivr, unpkg), Rust (crates.io), Go (the Go module proxy), Ruby (RubyGems), and Java (Maven Central). * **Git hosts**: GitHub, including release and raw file downloads, GitLab, and Bitbucket. * **Container registries**: Docker Hub, GitHub Container Registry, and Quay. * **OS package mirrors** for Debian, Ubuntu, and Alpine. * **Model provider APIs**: Anthropic, OpenAI, Google Gemini, Mistral, xAI, Groq, DeepSeek, Together AI, Fireworks AI, Cohere, and OpenRouter. Archil maintains this list and may change it. A sandbox picks up changes the next time it starts or resumes. `getNetwork` returns the exact hostnames, and the API marks the policy with `managed: true`. The allowlist uses plain domain rules, so it does not decrypt TLS or install the Archil CA. You can't set your own policy on the Free plan. Creating a sandbox with a `network.egress` policy, or updating a sandbox's network, fails with `sandbox_network_plan_required` (HTTP 403). After you upgrade to Developer or above, sandboxes created on Free keep the managed allowlist until you [replace their policy](#dynamic-updates). ## Credential brokering For HTTP egress, we also support defining rules for transforming requests. This means that secret credentials can live outside the sandbox, so untrusted workloads cannot access them directly. For example, you can add API keys to outgoing HTTP requests so your sandbox never sees them: ```typescript theme={null} import * as archil from "disk"; const sandbox = await archil.createSandbox({ network: { egress: { default: "deny", allow: [{ target: "api.github.com", transform: { headers: { Authorization: `Bearer ${process.env.GITHUB_TOKEN}` } } }], } } }); ``` ## Egress forwarding Use `forward_url` on an allow rule to send matching requests to your own gateway. Applications inside the sandbox keep using the original API hostname, while your gateway can apply logging, usage limits, or request filtering before contacting the upstream service. ```typescript theme={null} import * as archil from "disk"; const gatewayToken = process.env.EGRESS_GATEWAY_TOKEN; if (!gatewayToken) throw new Error("Set EGRESS_GATEWAY_TOKEN before creating the sandbox"); const sandbox = await archil.createSandbox({ network: { egress: { default: "deny", allow: [{ target: "api.openai.com", forward_url: "https://gateway.example.com/archil", transform: { headers: { Authorization: `Bearer ${gatewayToken}`, }, }, }], }, }, }); ``` With this policy, a request to `https://api.openai.com/v1/responses?trace=1` is sent to `https://gateway.example.com/archil/v1/responses?trace=1`. Archil appends the original path and query to the gateway URL's path prefix and streams the request body and response. Your gateway decides whether to call the original upstream or return its own response. The `transform` adds your gateway token before forwarding, so the gateway can authenticate the request without exposing the token to code inside the sandbox. Archil also overwrites these headers on forwarded requests: | Header | Value | | - | - | | `archil-forwarded-host` | Original request hostname, such as `api.openai.com`. | | `archil-forwarded-scheme` | Original scheme: `http` or `https`. | | `archil-forwarded-port` | Original port: `80` or `443`. | | `archil-forwarded-path` | Original path and query, such as `/v1/responses?trace=1`. | | `archil-sandbox-id` | ID of the sandbox making the request. | Use these headers to identify the original destination and sandbox. Authenticate with the gateway token; the metadata headers alone do not prove that a request came from Archil. Forwarding works with domain allow rules for HTTP on port 80 and HTTPS on port 443. HTTPS clients must trust the [Archil egress CA](#tls-inspection). Deny rules still take precedence. See [Create Sandbox](/api-reference/sandboxes/create-sandbox) for URL restrictions and [Dynamic Updates](#dynamic-updates) to change an existing sandbox's policy. ## Dynamic Updates A running sandbox's complete network policy can be replaced with `updateNetwork`, and `getNetwork` returns the current policy. A new policy applies to new connections; existing connections stay open. For example you can: 1. initially pull data from an S3 bucket 2. deny all egress while an agent is running 3. run a verifier and upload evaluation results ```typescript theme={null} import * as archil from "disk"; const sandbox = await archil.createSandbox({ network: { egress: { default: "deny", allow: ["data-bucket.s3.us-east-1.amazonaws.com"], } } }); // download data... await sandbox.updateNetwork({ egress: { default: "deny", } }); // run agent... await sandbox.updateNetwork({ egress: { default: "deny", allow: ["result-bucket.s3.us-east-1.amazonaws.com"] } }); // run verifier... ``` ## Draining requests before pause To let in-flight requests to selected hosts finish before a sandbox pauses, list them in `drain_on_pause` when creating the sandbox: ```typescript theme={null} import * as archil from "disk"; const sandbox = await archil.createSandbox({ network: { egress: { default: "allow", drain_on_pause: ["*.anthropic.com", "api.openai.com"], } } }); await sandbox.pause(); ``` Each entry is a lowercase hostname pattern without a scheme, port, or path. `*` matches anywhere, `"*"` selects every host, and a leading `*.` excludes the apex domain. Selectors do not grant network access. When the sandbox pauses, new requests to selected hosts are held and active HTTP(S) responses, including streams, get up to ten minutes to finish before the snapshot is taken. Draining covers ports 80 and 443; clients should retry stale connections after resume. Set selectors at creation, since connections opened before a policy update are not intercepted. # Sandbox images Source: https://docs.archil.com/compute/sandboxes/images Start sandboxes from public or private OCI images and reuse prepared images. Choose an OCI image with the tools and dependencies your workload needs. Archil supports public and private Linux images built for `amd64`. ## Public images For a public image, pass `baseImage` in TypeScript, or `base_image` in Python and REST, when creating a sandbox: ```typescript theme={null} const sandbox = await client.sandboxes.create({ baseImage: "python:3.13" }); ``` You can use Docker Hub shorthand, as above, or a full reference such as `ghcr.io/acme/agent-runtime:v2`. If you omit the image, the sandbox starts from `node:24-bookworm`: Debian 12 with Node.js 24, Python 3, git, curl and a C toolchain. ## Private images For a private image, build it with your registry credentials first. The SDK waits until the image is ready, then you can pass its ID when creating a sandbox: > [tab] TypeScript > > ```typescript theme={null} > import { Archil } from "disk"; > > const client = new Archil({ > apiKey: process.env.ARCHIL_API_KEY, > region: "aws-us-east-1", > }); > const image = await client.images.build({ > source: "ghcr.io/acme/agent-runtime:v2", > registryAuth: { > username: process.env.REGISTRY_USERNAME!, > password: process.env.REGISTRY_TOKEN!, > }, > }); > > const sandbox = await client.sandboxes.create({ imageId: image.imageId }); > ``` > [tab] Python > > ```python theme={null} > import os > from archil import Archil, RegistryAuth > > with Archil(region="aws-us-east-1") as client: > image = client.images.build( > source="ghcr.io/acme/agent-runtime:v2", > registry_auth=RegistryAuth( > username=os.environ["REGISTRY_USERNAME"], > password=os.environ["REGISTRY_TOKEN"], > ), > ) > sandbox = client.sandboxes.create(image_id=image.image_id) > ``` > [tab] REST > > ```bash theme={null} > curl --request POST \ > 'https://control.green.us-east-1.aws.prod.archil.com/api/images' \ > --header "Authorization: Bearer $ARCHIL_API_KEY" \ > --header 'Content-Type: application/json' \ > --data '{ > "source": "ghcr.io/acme/agent-runtime:v2", > "registry_auth": { > "username": "YOUR_REGISTRY_USERNAME", > "password": "YOUR_REGISTRY_TOKEN" > } > }' > ``` > > The response contains `data.image_id`. Set `IMAGE_ID` to that value and poll until `data.status` is `ready` or `failed`: > > ```bash theme={null} > curl "https://control.green.us-east-1.aws.prod.archil.com/api/images/$IMAGE_ID" \ > --header "Authorization: Bearer $ARCHIL_API_KEY" > ``` > > If the build fails, inspect `failure_reason`. Once it is `ready`, create the sandbox: > > ```bash theme={null} > curl --request POST \ > 'https://control.green.us-east-1.aws.prod.archil.com/api/sandboxes?wait=true' \ > --header "Authorization: Bearer $ARCHIL_API_KEY" \ > --header 'Content-Type: application/json' \ > --data "{\"image_id\": \"$IMAGE_ID\"}" > ``` > > If the sandbox response is still `pending`, poll [Get Sandbox](/api-reference/sandboxes/get-sandbox) for `running`. > If it becomes `failed`, inspect `exit_reason`. Use a registry username and a password or token with pull access. Archil does not store these credentials, so supply them each time you build. Once built, the image is available to sandboxes in your account without registry credentials. See [Build Image](/api-reference/images/build-image) and [Get Image](/api-reference/images/get-image) for request options and build failures. ## Reusing and updating images Archil prepares each image for use as a sandbox filesystem. To finish this work before creating sandboxes, build once and reuse the returned image ID. You can also prebuild public images by omitting the registry credentials. If you publish a new version under the same tag, build it again to pick up the change. New sandboxes using that image ID get the latest successful build. While a rebuild is in progress, they can still use the previous successful build. Existing sandboxes keep their original image, including after a restart. To pin an image to a specific version, use a digest reference such as `repository@sha256:...` instead of a tag. # Persistent sandboxes Source: https://docs.archil.com/compute/sandboxes/introduction Run untrusted, AI-generated code in isolated Firecracker microVMs with persistent disks. Archil sandboxes are a complete sandbox platform for running untrusted, AI-generated code. Each sandbox is a dedicated [Firecracker](https://firecracker-microvm.github.io/) microVM with its own Linux kernel, and Archil runs the whole lifecycle: create, pause and resume with memory intact, fork, exposed ports, supervised services, private images, and an egress firewall. You don't need a second sandbox provider. What sets Archil apart is the disk: the root filesystem is a durable Archil disk, so the workspace survives the VM and forks in milliseconds, and any Archil disk you mount is reachable from outside the sandbox. Persistent sandboxes are generally available. They run on the same Firecracker runtime as [serverless execution](/compute/serverless-execution), which serves production agent tool calls, and keep their root filesystems on Archil disks. * **Egress you control:** on Developer, Team, and Enterprise, write your own [firewall](/compute/sandboxes/firewall) with domain and CIDR rules, starting from `default: "deny"` for no network at all. On Free, sandboxes reach only a [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist) of package registries, Git hosts, and model APIs, and everything else is denied. * **Secrets stay outside the sandbox:** with [credential brokering](/compute/sandboxes/firewall#credential-brokering), the egress proxy adds API keys to outgoing requests, so code in the sandbox never sees them (Developer plan and above). * **Private until exposed:** ports and services aren't reachable from the internet until you expose them. * **Isolation:** hardware virtualization (KVM), not shared-kernel containers. Sandboxes can't reach each other, private networks, or cloud metadata. See [Isolation](#isolation). * **No per-command timeout:** a session runs for up to 24 hours, then pauses with its processes intact. See [Setting a time to live](#setting-a-time-to-live). * **Persistent:** the root filesystem lives on an Archil disk, so files and installed packages survive stop, start, and pause. Fork a sandbox to run copies in parallel. * **Any image:** public or private OCI images on x86-64, with 1 to 32 vCPU and 0.25 to 64 GiB of memory. * **Reversible:** sandboxes boot from standard OCI images, and a workspace on a disk synced to your own bucket is stored as ordinary objects that any S3 tool can read without Archil. The same disk mounts in [E2B](/guides/platforms/e2b), [Daytona](/guides/platforms/daytona), and [Modal](/guides/platforms/modal), so you can move compute without moving data. See [Data ownership and portability](/reference/data-ownership). Use sandboxes for agent loops, interactive development environments, and network services such as preview servers or databases. For one-off bash, Python, or Node commands against a disk, use [serverless execution](/compute/serverless-execution). Already running agents on E2B, Daytona, or Vercel Sandbox? You can mount an Archil disk there to keep workspaces portable, then run the same disk in Archil sandboxes when you're ready. See [Platforms](/guides/platforms/e2b). ## Limits and lifecycle at a glance | | | | - | - | | Status | Generally available since September 2026. See the [changelog](/reference/changelog#september-2026). | | Isolation | One Firecracker microVM per sandbox, with its own Linux kernel, isolated by hardware virtualization (KVM). See [Isolation](#isolation). | | Size | 1 to 32 vCPU and 0.25 to 64 GiB of memory, on x86-64. | | Session length | A session pauses after at most 24 hours. Resuming starts a new session. On the Free plan, a session lasts at most 30 minutes. See [Setting a time to live](#setting-a-time-to-live). | | Idle timeout | 30 minutes without a process connection by default, then the sandbox pauses. Set `idleTtlSeconds` to `0` to turn it off. | | Command time limit | None for commands inside a sandbox. One-shot [serverless execution](/compute/serverless-execution#limits) commands stop after 5 minutes. | | Stop | Keeps the root filesystem and mounted disks. Discards memory and processes. | | Pause | Keeps the root filesystem, mounted disks, memory, and running processes. | | Delete | Removes the root filesystem. Mounted disks and their data are kept. | | Network | On Free, egress is limited to a [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist). On Developer and above, you set each sandbox's policy with the [firewall](/compute/sandboxes/firewall), and a sandbox created without one has unrestricted egress. Ports are private until you expose them. | | Pricing | $0.055 per vCPU-hour and $0.020 per GiB-hour on Developer and Team, while the sandbox runs. Stopped and paused sandboxes don't accrue compute charges. Disk storage is billed separately. See [Pricing & Billing](/administration/billing#persistent-sandboxes-and-services). | | Concurrent sandboxes | 5 on Free, 20 on Developer, 250 on Team. See [Plans and limits](/administration/billing#plans-and-limits). | | Regions | Archil's public AWS regions, and [BYOC](/reference/byoc) deployments. See [Region availability](/reference/regions#compute-availability). | For every numeric limit, see [Limits](/reference/limits#sandboxes). > [!NOTE] > > Persistent sandboxes are available in Archil's public AWS regions and in > [BYOC deployments](/reference/byoc) on any cloud (AWS, Google Cloud, or Azure) or bare > metal. See [Region availability](/reference/regions#compute-availability). > [!NOTE] > > Occasional maintenance events may `pause` running sandboxes, and require that you `resume` them to continue working > from the same state. Rarer hardware failures result in sandboxes becoming `failed`, which may incur loss of in-memory > state or writes that haven't yet been persisted to disk. Data already persisted to disk is kept, so starting the > sandbox again boots it from disk. See [What survives a crash](#what-survives-a-crash). You can easily create a persistent sandbox using the TypeScript SDK (or the [Python SDK](/sdks/python#persistent-sandboxes)): ```typescript theme={null} import * as archil from "disk"; const sandbox = await archil.createSandbox({ // Sandbox name, for finding later name: "agent-workspace", // Supports public OCI images baseImage: "python:3.13", // Select the amount of vCPU and memory vcpuCount: 2, memSizeMiB: 4096, }); const execution = await sandbox.run("python --version && pwd"); const result = await execution.wait(); console.log(result.stdout); ``` Archil sandboxes run on x86-64 (`amd64`) and support between 1 and 32 vCPU cores and between 0.25 and 64 GiB of memory. Sandboxes run as the root Linux user (uid: 0, gid: 0). ## One-shot execution or a persistent sandbox Not every task needs a sandbox that persists. [Serverless execution](/compute/serverless-execution) (`disk.exec`) is one-shot, ephemeral execution: each call runs one command in its own Firecracker microVM with your disk mounted, returns the output, and shuts the VM down. Only what the command writes to the disk outlives the call, and there is no sandbox to create, stop, or delete. | | Serverless execution (`disk.exec`) | Persistent sandbox | | - | - | - | | Lifetime | One command, then the microVM shuts down | Runs until you stop, pause, or delete it | | State between calls | Only files written to the disk | Root filesystem and installed packages; memory and processes when paused | | Time limit | 5 minutes per command | Up to 24 hours per session, then it pauses | | Billing | Per call | Per vCPU-hour and GiB-hour | | Good for | Bash and code tools in an agent loop, one-off scripts, fan-out | Agent loops, stateful code interpreters, dev environments, network services | The [quickstart](/getting-started/quickstart) gives an agent a one-shot `bash` tool. The [code interpreter guide](/guides/agents/code-interpreter) keeps a Python kernel alive in a persistent sandbox. For keeping an agent's environment across restarts, see [How to keep an AI agent's environment across sandbox restarts](https://archil.com/article/how-to-keep-an-ai-agent-s-environment-across-sandbox-restarts). ## Isolation Each sandbox runs in a dedicated [Firecracker](https://firecracker-microvm.github.io/) microVM, isolated from the host and from other sandboxes by hardware virtualization (KVM) rather than by containers alone. * Each sandbox boots its own Linux kernel. Code in the sandbox, including code running as root, never runs on the host kernel. * Root inside the VM grants no privileges on the host or in other sandboxes. For how this compares with gVisor and containers, see [What isolation do agent sandboxes actually use?](https://archil.com/article/what-isolation-do-agent-sandboxes-actually-use-firecracker-gvisor-and-containers-compared) You can check these boundaries yourself in a few minutes. [Verify Archil yourself](/details/verify#sandbox-isolation) lists commands to run inside a sandbox. Its latency script reproduces our published disk-operation numbers, such as checkpoint and branch times; it doesn't measure sandbox boot time. ## Linux and container compatibility Sandboxes support running containers inside the VM. Include the tools your workload needs in your [sandbox image](/compute/sandboxes/images); kernel support does not install Docker or other user-space tools. | Capability | Behavior inside a sandbox | | - | - | | Docker and Docker Compose | Supported. Start the Docker daemon inside the sandbox before using it. | | OverlayFS / Docker `overlay2` | Supported on the sandbox's ext4 root filesystem. Keep Docker's data directory (normally `/var/lib/docker`) on that filesystem. | | cgroups | cgroup v2 is mounted at `/sys/fs/cgroup` for managing processes and containers within the VM's allocated resources. | | FUSE | Supported, including additional Archil disk mounts. | | Mount propagation | Linux mount namespaces and shared/slave/private propagation apply within the VM. Configure propagation on the relevant mounts when sharing them with nested containers; mounts do not propagate to the Archil host. | | iptables / nftables | Supported for networking inside the VM, including container networks. Guest rules cannot override the sandbox's [network egress policy](/compute/sandboxes/firewall). | | Nested virtualization | Not supported: `/dev/kvm` is not available inside the sandbox. Docker containers do not require nested virtualization. | The sandbox's root filesystem and an additional FUSE-mounted Archil disk have different filesystem capabilities. An additional Archil disk cannot be used as an OverlayFS upper directory; use it for application data or bind-mount it into a container instead. See [Application Compatibility](/details/support#posix-compatibility-details) for mounted-disk limitations. ## Network egress On Developer, Team, and Enterprise, you set each sandbox's egress policy with the [sandbox firewall](/compute/sandboxes/firewall): deny everything, allow only the domains and CIDR ranges you list, or [broker credentials](/compute/sandboxes/firewall#credential-brokering). A sandbox created without a policy has unrestricted egress. Sandboxes on the Free plan can't reach arbitrary URLs. They can reach only a [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist) of package registries, Git hosts, container registries, OS package mirrors, and model provider APIs, and all other egress is denied. Sandboxes created on Free keep the managed allowlist after an upgrade until you [replace their network policy](/compute/sandboxes/firewall#dynamic-updates). ## Attaching an Archil disk Pass `mounts` when creating a sandbox to give it access to files on an existing Archil disk. The disk must belong to your account and be in the same region as the sandbox: > [tab] TypeScript > > ```typescript theme={null} > import { Archil } from "disk"; > > const client = new Archil({ > apiKey: process.env.ARCHIL_API_KEY, > region: "aws-us-east-1", > }); > const sandbox = await client.sandboxes.create({ > baseImage: "ubuntu:24.04", > mounts: [{ disk: "dsk-0123456789abcdef", path: "/mnt/archil" }], > }); > console.log((await sandbox.exec("ls /mnt/archil")).stdout); > ``` > [tab] Python > > ```python theme={null} > from archil import Archil, SandboxMountSpec > > with Archil(region="aws-us-east-1") as client: > sandbox = client.sandboxes.create( > base_image="ubuntu:24.04", > mounts=[SandboxMountSpec(disk="dsk-0123456789abcdef", path="/mnt/archil")], > ) > print(sandbox.exec("ls /mnt/archil").stdout) > ``` > [tab] REST > > ```bash theme={null} > curl --request POST \ > 'https://control.green.us-east-1.aws.prod.archil.com/api/sandboxes?wait=true' \ > --header "Authorization: Bearer $ARCHIL_API_KEY" \ > --header 'Content-Type: application/json' \ > --data '{ > "base_image": "ubuntu:24.04", > "mounts": [{ "disk_id": "dsk-0123456789abcdef", "path": "/mnt/archil" }] > }' > ``` > > If the response is still `pending`, poll [Get Sandbox](/api-reference/sandboxes/get-sandbox) until it is `running` > before executing commands. If it becomes `failed`, inspect `exit_reason`. Archil handles mount authentication automatically. Deleting a sandbox removes its root filesystem. Mounted disks and their data are preserved. With a single mount, `path` is optional and defaults to `/mnt/archil`. To mount more disks, add an entry for each one with a separate path, such as `/mnt/archil` and `/mnt/models`. Use `subdirectory` to expose only part of a disk. See [Create Sandbox](/api-reference/sandboxes/create-sandbox) for the full mount options and path restrictions. By default, a mount takes exclusive access to the disk. To share it across sandboxes, use `readOnly` (`read_only` in Python and REST) for readers or `conditional` for writers. Coordinate concurrent writes in your application to avoid overwriting each other's changes. See [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents) for a pattern that does this, and [Sharing Disks](/concepts/sharing-disks) for how these modes work. Sandboxes keep their mounts across stop/start and pause/resume. To change them, stop the sandbox and pass a new `mounts` list to [Start Sandbox](/api-reference/sandboxes/start-sandbox). Omit `mounts` to keep the saved list, or pass `[]` to remove all mounts without deleting their disks. For a paused sandbox, `resume()` keeps its mounts; use `start()` to change them, which boots from disk and discards the paused memory state. ## Root filesystem and mounted disks A sandbox stores files in two places, and they persist differently. | | Root filesystem (`/`) | Mounted Archil disk | | - | - | - | | What it is | An ext4 filesystem: your base image plus a copy-on-write layer, stored on an Archil disk that belongs to this sandbox alone | An Archil disk of your own, mounted with FUSE at the `path` you choose | | Survives stop, start, pause, and resume | Yes | Yes | | Survives deleting the sandbox | No | Yes | | When you fork | The fork gets its own copy | The fork mounts the same disk, so changes are shared | | Reachable from outside the sandbox | Only by running commands in the sandbox | Yes: from other sandboxes, [serverless execution](/compute/serverless-execution), [`archil mount`](/mounting/linux), the SDKs, and the [S3 API](/protocols/s3-api) | | Filesystem features | Full ext4, including OverlayFS and Docker's `overlay2` | Archil's [POSIX support](/details/support#posix-compatibility-details); can't be an OverlayFS upper directory | | Good for | Installed packages, toolchains, container images, build caches | Data that other sandboxes or your app use, and data that must outlive the sandbox | Both are durable. For what each keeps after a hardware failure, see [What survives a crash](#what-survives-a-crash). Mounting an Archil disk in another provider's sandbox, such as E2B or Daytona, persists only what is written under the mount path. That sandbox's own root filesystem, including the packages you installed, still goes away with it. In an Archil sandbox, the root filesystem is itself on an Archil disk, so installed packages persist too. ## Passing environment variables Environment variables can be set for the whole sandbox, for a single command, or both. Variables passed when the sandbox is created apply to every command that runs inside it: ```typescript theme={null} const sandbox = await archil.createSandbox({ baseImage: "python:3.13", env: { DATABASE_URL: "postgres://localhost/app", LOG_LEVEL: "info", }, }); ``` Individual processes can add or override their own variables by passing `env` to `run`: ```typescript theme={null} // Runs with DATABASE_URL from the sandbox, and LOG_LEVEL=debug const migration = await sandbox.run("python manage.py migrate", { env: { LOG_LEVEL: "debug" }, }); await migration.wait(); ``` ## Starting and stopping sandboxes A sandbox runs until you shut it down or its [TTL](#setting-a-time-to-live) expires. There are two ways to suspend one, and they differ in how much state they keep. Stopping or pausing a sandbox stops compute charges; retained disk data remains subject to [storage pricing](/administration/billing#storage). Stopping a sandbox sends `SIGTERM` to every running process and shuts down the VM, leaving the disk intact: ```typescript theme={null} await sandbox.stop(); ``` Starting it again boots a fresh VM from that same disk: ```typescript theme={null} await sandbox.start(); ``` Everything written to disk survives, but memory does not, so your application handles shutdown and startup the same way it would on any machine. You can instead pause a VM to snapshot memory alongside the disk, so a resumed sandbox picks up exactly where it left off, with running processes intact: ```typescript theme={null} // Preserve running application state for resume await sandbox.pause(); // Resume the sandbox from the paused state await sandbox.resume(); ``` Deleting a sandbox releases all of its resources and cannot be undone. A sandbox has to be stopped before it can be deleted, unless it has already exited or failed: ```typescript theme={null} await sandbox.stop(); await sandbox.delete(); ``` ## What survives a crash A clean `stop` or `pause` flushes the root filesystem and every mounted Archil disk to Archil storage before the sandbox reaches `stopped` or `paused`, and `pause` also saves memory and running processes there. A paused sandbox doesn't depend on the machine it ran on, so it can resume after a hardware failure. If a sandbox stops without a clean `stop` or `pause`, for example because of a hardware failure, it becomes `failed`: | | Clean `stop` or `pause` | Hardware failure | | - | - | - | | Root filesystem | Kept | Kept as of its last flush. Keep critical data on a mounted disk. | | Mounted Archil disks | Kept | Writes acknowledged by Archil are kept, including every write that `fsync` returned for. | | Memory and processes | Kept by `pause`, discarded by `stop` | Lost | Starting a `failed` sandbox boots it from its root disk. For data you can't lose, write it to a mounted Archil disk and call `fsync`. When `fsync` returns, the write is durable on Archil. Writes are also flushed in the background shortly after they're made. ## Forking a sandbox In some situations, you may want to fork a sandbox into multiple branches from a single point in time. This makes it simple to, for example, start sandboxes from a template without needing to incur startup latency for a slow-running program or run parallel experiments on sandbox state from some point in the past. A sandbox without additional disk mounts can be forked while running, paused, or stopped: ```typescript theme={null} const newSandbox = await sandbox.fork(); ``` A fork checkpoints the sandbox's root disk and creates a [branch](/concepts/branches-and-checkpoints) from that checkpoint. A checkpoint takes about 15 ms and a branch about 35 ms, regardless of disk size. Pausing a running sandbox and booting the fork take additional time. After forking, `newSandbox` will be able to run commands independently of the original sandbox, starting from the exact state of the original sandbox. Forking a running sandbox pauses it while the snapshot is taken and resumes it once the fork is accepted, so attached process connections close; reconnect with `attach`. A paused or stopped sandbox is left as it is, and a fork of a stopped sandbox boots from its disk. ### Forking a sandbox with mounted disks A sandbox with mounted disks must be stopped before forking. Forking a running or paused sandbox copies its memory, and that copy would include the parent's Archil disk clients, with their identity and any [delegations](/concepts/sharing-disks#understanding-delegations) they hold. Two sandboxes would then act as one client. So the fork is refused with `409` and the code `sandbox_fork_mounted`, and a running parent keeps running. A stopped sandbox has no memory to copy: its fork boots from disk and mounts each disk as a new client. To fork a sandbox that has mounted disks: ```typescript theme={null} await sandbox.stop(); // flushes the root filesystem and every mounted disk const fork = await sandbox.fork(); // boots from disk; memory and processes are not copied ``` The fork gets its own root filesystem but mounts the same additional disks as its parent, so changes to those disks are shared. With the default exclusive mount, only one client can have a disk mounted at a time, so the parent can't mount those disks again while the fork has them. To run both sandboxes at once, mount the shared disks with `readOnly` or `conditional` (see [Sharing Disks](/concepts/sharing-disks)), or stop the fork and [start](/api-reference/sandboxes/start-sandbox) it with different `mounts`. To give the fork its own copy of a disk's data, mount a [branch](/concepts/branches-and-checkpoints) of that disk. Separate disks also keep their data isolated. If you fork often, keep the data the fork needs on the root filesystem: a sandbox without mounted disks can be forked while running, with its memory and processes. A sandbox cannot be deleted while it has forks, so delete each fork before the sandbox it was forked from. ## Hosting network services Archil sandboxes support the ability to host network services, including creating per-sandbox preview URLs for CI-like workflows. There are two ways to expose a port for a network service, depending on whether you are inside or outside the sandbox. Outside the sandbox, you can specify a list of ports to expose when creating the sandbox or dynamically expose a port later. ```typescript theme={null} const sandbox = await archil.createSandbox({ ports: [3000] }); const ports = await sandbox.listPorts(); const hostnames = ports.map((port) => port.hostname); console.log(hostnames); const hostname = await sandbox.exposePort(3001); await sandbox.unexposePort(3001); ``` Exposed ports are public: they accept any TCP protocol over TLS with SNI, without authentication. For private access, leave the port unexposed and create a port token. Send the token in the `X-Archil-Token` header; requests without a valid token get `401`. Tokens are checked when a connection opens, so revoking or expiring a token blocks new connections while open ones stay open. Token access supports HTTP/1.1 over HTTPS only, and publicly exposed ports ignore tokens. ```typescript theme={null} const access = await sandbox.createPortToken(8080, { // optionally specify a TTL, up to "8760h"; omit for no expiry ttl: "5m", }); // pass X-Archil-Token when making requests to the hostname const response = await fetch(`https://${access.hostname}`, { headers: { "X-Archil-Token": access.token, } }); await sandbox.deletePortToken(access); ``` Inside the sandbox, `archil services create` runs a command as a supervised service: it starts immediately and restarts if it exits. Service ports are private by default, so reach them by exposing the port or with a port token. To let services publish their own hostnames, create the sandbox with `enable_service_ingress: true` through the [Create Sandbox API](/api-reference/sandboxes/create-sandbox). ```typescript theme={null} const serviceProcess = await sandbox.run( "archil services create web --tcp-port -- ", ); const { stdout } = await serviceProcess.wait(); const service = JSON.parse(stdout); console.log("Server available at: ", service.hostname); ``` For example, in a sandbox with service ingress enabled, the following runs `python3 -m http.server 8080` and returns a stable HTTPS URL that proxies traffic to local port 8080. Without service ingress, the service still runs, but its `hostname` is not routed until you expose the port. ```bash theme={null} $ archil services create web --tcp-port 8080 -- python3 -m http.server 8080 { "after": [], "command": ["python3", "-m", "http.server", "8080"], "enabled": true, "env": {}, "hostname": "8080-.prod-aws-us-west-2-green.archil.app", "name": "web", "state": { "exit_status": null, "pid": null, "restart_count": 0, "result": null, "status": "activating" }, "tcp_port": 8080, "working_dir": null } ``` Traffic to an exposed port or service hostname, or a request with a valid port token, automatically starts a stopped sandbox or resumes a paused one, including one paused by its TTL. All sandbox ports are served over TLS. > [!WARNING] > > Exposed ports and service hostnames are publicly accessible. Use port tokens, or include authentication/authorization > in your service, if you want to expose a private service. ## Choosing a base image Use `baseImage` to start from a public Linux OCI image, such as `python:3.13`. The default is `node:24-bookworm`, which includes Node.js, Python, git and a C toolchain. For private images, build with your registry credentials first, then pass the returned `imageId` when creating the sandbox. See [Sandbox images](/compute/sandboxes/images) for examples and how to reuse builds. ## Setting a time to live A sandbox has a hard timeout and an idle timeout. Both **pause** the sandbox, preserving its disk, memory, and processes for `resume()`. Attached process connections close when it pauses; reconnect after resuming. If the pause snapshot fails, the sandbox is marked `failed`. Stop a paused sandbox before deleting it. | Setting | Range | Default | | - | - | - | | `maxTtlSeconds` | 60–86,400 seconds | 86,400 seconds (24 hours) | | `idleTtlSeconds` | 0–86,400 seconds | 1,800 seconds (30 minutes) | ```typescript theme={null} const sandbox = await archil.createSandbox({ maxTtlSeconds: 3600, idleTtlSeconds: 300, }); await sandbox.setTimeout(7200); await sandbox.setTimeout({ timeoutSeconds: 3600, idleTtlSeconds: 60 }); await sandbox.setTimeout({ idleTtlSeconds: 0 }); ``` Updating the hard timeout resets its deadline from now, capped at 24 hours after the current session started. An idle-only update leaves that deadline unchanged and resets the idle countdown if no process connection is open. Omitted settings stay unchanged. Updates to a paused or stopped sandbox apply to its next powered-on session. Starting or resuming begins a new session with a fresh timeout. Open direct process connections prevent idle expiry. Detached processes and service-port traffic do not keep the idle timer alive, so a sandbox running a server or a background job with nothing attached pauses after 30 minutes by default. Set `idleTtlSeconds` to `0` to turn idle expiry off for unattended jobs or services that must keep running between connections. A process connection to a paused sandbox fails with `409`, so call `resume()` before reconnecting. Activity does not extend the hard timeout. `setTimeout()` can extend it only within the session's 24-hour limit. For REST, use [Set sandbox timeouts](/api-reference/sandboxes/set-sandbox-timeout). On the Free plan, a session lasts at most 30 minutes. A sandbox created without `maxTtlSeconds` gets 30 minutes, and a longer `maxTtlSeconds` or `setTimeout()` is refused with `403` and the code `sandbox_ttl_limit_exceeded`. A sandbox created without `maxTtlSeconds` on Free can also get a shorter session, so that it fits the sandbox compute the plan has left that month. ### Running long jobs For an agent or job that runs for hours: * **Turn off idle expiry.** Create the sandbox with `idleTtlSeconds: 0`, or call `setTimeout({ idleTtlSeconds: 0 })`. Otherwise a detached process, or a service that only receives port traffic, pauses after 30 minutes. * **Plan for the 24-hour pause.** The 24-hour session limit is hard: neither activity nor `setTimeout()` moves it. When it's reached, the sandbox pauses with memory and processes intact, and `resume()` continues them in a new 24-hour session. A paused sandbox runs nothing until it is resumed, so a job that runs longer than 24 hours stops for as long as the sandbox stays paused. Check the sandbox's status with [Get Sandbox](/api-reference/sandboxes/get-sandbox) and call `resume()` when it is `paused`. Traffic to an exposed port or a port token also resumes it (see [Hosting network services](#hosting-network-services)). * **Write results to disk as you go.** If the pause snapshot fails, the sandbox becomes `failed` and memory is lost, but data written to disk is kept. See [What survives a crash](#what-survives-a-crash). ## Opening interactive shells By default, processes run without allocating a pseudo-terminal (pty). This is usually desirable, since it signals to the process that it is not in an interactive terminal. However, some commands, such as those that emit color, full-screen tools like `vim` and `top`, or tools that need interactive input should instead be launched in a pseudo-terminal. Provide terminal dimensions to launch a process in a pseudo-terminal. The process can then interactively receive input and stream output to the caller: ```typescript theme={null} const shell = await sandbox.run("/bin/sh", { terminal: { cols: 120, rows: 40 }, onOutput: ({ data }) => process.stdout.write(data), }); await shell.sendInput("ls -la\n"); await shell.sendInput("exit\n"); const { exitCode } = await shell.wait(); ``` ## Next steps * [Pricing and plan limits](/administration/billing) * [TypeScript SDK](/sdks/typescript#persistent-sandboxes) * [Python SDK](/sdks/python#persistent-sandboxes) * [Sandbox images](/compute/sandboxes/images) * [Create a sandbox](/api-reference/sandboxes/create-sandbox) * [Create a process connection](/api-reference/sandboxes/create-sandbox-connection) * [Fork a sandbox](/api-reference/sandboxes/fork-sandbox) * [Sandbox lifecycle endpoints](/api-reference/sandboxes/pause-sandbox) * [Sandbox firewall](/compute/sandboxes/firewall) # Search files Source: https://docs.archil.com/compute/search-files Archil disks in AWS regions provide a built-in, high-performance search tool that's effective for searching across millions of files on your disk, up to 10x faster than tools like `ripgrep`. ```typescript TypeScript theme={null} import * as archil from "disk"; const d = await archil.getDisk("dsk-abc123"); const { matches } = await d.grep({ directory: "logs", pattern: "ERROR|FATAL", recursive: true, }); ``` ```python Python theme={null} import archil d = archil.get_disk("dsk-abc123") res = d.grep(directory="logs", pattern="ERROR|FATAL", recursive=True) ``` ```bash CLI theme={null} npx disk grep "ERROR|FATAL" dsk-abc123/logs ``` Patterns use [ripgrep's regular expression syntax](https://docs.rs/regex/latest/regex/#syntax), and the options below follow ripgrep's flags, so a search you'd write with `rg` carries over. Teams building the highest-performance AI agents have discovered that agent performance degrades as you give them more tools to select between at each turn. As a result, teams are increasingly moving away from specialized products like vector databases for context and towards lowest-common denominator infrastructure like file storage. However, existing file system tools like `grep` are too slow for searching across large document sets. With Archil's built-in `grep` APIs, you can effectively search across millions of documents on your disk using map-reduce inside of Archil. Each `disk.grep()` request counts as one serverless tool call, regardless of worker count or runtime. See [Pricing & Billing](/administration/billing#serverless-tool-calls). ## Managing concurrency Archil's `grep` API works by performing a massively parallel search across the files stored in the folder that you're searching. Archil automatically manages the concurrency of each search to optimize for search latency. If you'd prefer to control this concurrency manually, you can specify the exact number of workers to use via the `concurrency` option. ```typescript TypeScript theme={null} const { matches } = await d.grep({ directory: "logs", pattern: "ERROR|FATAL", concurrency: 10, }); ``` ## Limiting work For some kinds of search, you do not need an exhaustive set of results. For example, if you're trying to find the first occurrence of an error, you only need one result. You can manage the amount of work that Archil's search performs by specifying either the `maxResults` or `maxDurationSeconds` options. When these options are specified, the search will stop as soon as the result or time limit is reached, regardless of whether the search was exhaustive. Check `stoppedReason` in the result: `completed` means every file was searched, and anything else means the matches may be partial. When a search stops early, its matches are a sample of whichever workers reported first, not the first matches in path order. ## Filtering files To search only some files under the directory, pass `globs`, `types`, or `typesNot`. * `globs` follow ripgrep's `-g` rules, matched against paths relative to `directory`. A leading `!` excludes. A glob without a `/`, like `*.log`, matches a name at any depth; a glob with a `/`, like `src/**/*.ts`, matches the whole relative path. Use `dir/**` for the files under a directory. If you pass any glob without a `!`, files that match none of them are skipped. When several globs match a path, the last one wins. * `types` searches only files of the given [ripgrep file types](https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#manual-filtering-file-types), such as `py`, `ts`, `go`, or `md`. `typesNot` skips them, and takes precedence over `types`. A file that a glob matches is searched or skipped by the glob, whatever its type. ```typescript TypeScript theme={null} const { matches } = await d.grep({ directory: "src", pattern: "TODO", recursive: true, types: ["ts"], globs: ["!**/*.test.ts", "!generated/**"], }); ``` ```python Python theme={null} res = d.grep( directory="src", pattern="TODO", recursive=True, types=["ts"], globs=["!**/*.test.ts", "!generated/**"], ) ``` ```bash CLI theme={null} npx disk grep TODO dsk-abc123/src -t ts -g '!**/*.test.ts' -g '!generated/**' ``` When you search a [workspace](/getting-started/quickstart) at its root, paths start with the mount name. A glob with a `/` must start with a mount name, like `data/src/**`, or with `**/`. Mount names in globs are matched literally: `!logs` skips the `logs` mount, but `!l*` does not. ## Matching options * `ignoreCase` matches regardless of case, like `rg -i`. * `multiline` lets a match span lines, like `rg -U`. A `\n` in the pattern matches a line break, and `.` matches one only after `(?s)`. Files with Windows line endings need `\r?\n`. A multiline match reports its first line as `line` and its last as `endLine`, and its `text` holds every line it covers. Matches on adjacent lines come back as one match. ```typescript TypeScript theme={null} // Find functions whose signature spans several lines const { matches } = await d.grep({ directory: "src", pattern: String.raw`def \w+\(\n(\s+\w+,?\n)+\s*\):`, recursive: true, multiline: true, }); ``` ```python Python theme={null} # Find functions whose signature spans several lines res = d.grep( directory="src", pattern=r"def \w+\(\n(\s+\w+,?\n)+\s*\):", recursive=True, multiline=True, ) ``` ## Context lines Pass `contextBefore` or `contextAfter` (0 to 10 lines each) to get the lines around each match, like `rg -B` and `rg -A`. The result then includes `groups`: runs of consecutive lines from one file, in which each line is marked as a match or as context. Nearby matches share a group, so every line appears once. `matches` still lists the matches on their own. ```typescript TypeScript theme={null} const { groups } = await d.grep({ directory: "logs", pattern: "panic", recursive: true, contextBefore: 3, contextAfter: 3, }); for (const g of groups ?? []) { for (const l of g.lines) { console.log(`${g.file}${l.match ? ":" : "-"}${l.line}: ${l.text}`); } console.log("--"); } ``` ```python Python theme={null} res = d.grep(directory="logs", pattern="panic", recursive=True, context_before=3, context_after=3) for g in res.groups or []: for l in g.lines: print(f"{g.file}{':' if l.match else '-'}{l.line}: {l.text}") print("--") ``` ```bash CLI theme={null} npx disk grep panic dsk-abc123/logs -C 3 ``` ## Result limits * A search returns at most 32 MiB of match and context text. A search that reaches this limit stops with `stoppedReason` set to `max_results`. Context lines count toward the limit, so large context and `maxResults` values multiply the response size. * Each line is truncated at 512 KiB. * Each worker returns at most 4 MiB of output. A batch of files that produces more is cut short, and the search reports `incomplete`. Narrow the search with `globs` or `types` to avoid it. ## Next steps * [TypeScript SDK: searching files](/sdks/typescript#searching-files) — the `disk.grep` API surface * [Python SDK: searching files](/sdks/python#searching-files) — the `Disk.grep` API surface * [API reference: Search Files](/api-reference/serverless-execution/grep-disk) — the raw HTTP endpoint * [Serverless execution](/compute/serverless-execution) — run arbitrary commands when you need more than search # Serverless execution Source: https://docs.archil.com/compute/serverless-execution Run bash commands directly on an Archil file system. Archil's serverless execution makes it simple to run untrusted code like bash, Python, and Node.js directly on the data in your Archil file system **without** needing to manage the lifecycle of a separate sandbox instance. Serverless execution is built on full-featured Linux microVMs that are colocated with your data for maximum security and performance. Each Archil disk can run many commands concurrently, and execution is included in the per-call price. This makes them ideal for tasks such as the bash tool in an AI agent loop. Serverless execution is available in Archil's public AWS regions and in [BYOC deployments](/reference/byoc) on any cloud (AWS, Google Cloud, or Azure) or bare metal. See [Region availability](/reference/regions#compute-availability). If you instead need a sandbox that stays alive in-between calls for the purpose of running commands with long startup times or network services, you can use an Archil [Persistent Sandbox](/compute/sandboxes/introduction). ```typescript TypeScript theme={null} import * as archil from "disk"; const { disk } = await archil.createDisk({ name: "agent-workspace" }); const { stdout } = await disk.exec("git clone --depth 1 https://github.com/pallets/flask.git"); ``` ```python Python theme={null} import archil from archil import S3Mount result = archil.create_disk(name="agent-workspace") res = result.disk.exec("git clone --depth 1 https://github.com/pallets/flask.git") ``` ```bash CLI theme={null} npx disk dsk-abc123 exec "git clone --depth 1 https://github.com/pallets/flask.git" ``` Companies are using Archil's serverless execution for: * **Running untrusted AI-generated code** on a machine that is close to the context that the AI is attempting to work on, without risking that code breaking the origin server. * **Running map-reduce style jobs across large data sets** for data analysis tasks such as full-text search or aggregation. * **Using your disk from a different region** in order to avoid excessive network latency between your application code and the disk, and reduce the amount of data transferred out of Archil. Each `disk.exec()` invocation at the default size counts as one serverless tool call, even when it runs several commands. Execution is included in the per-call price. See [Pricing & Billing](/administration/billing#serverless-tool-calls). ## Working directory and environment Each command starts with the working directory set to where your Archil disk is mounted (`/mnt/archil`). The default runtime image includes most common Unix tools. To run in a different environment, [choose an image](#choosing-an-image), or install additional tools by directing the binaries to your disk. ## Choosing an image By default, commands run in Archil's exec image. To run a command in your own environment, pass `baseImage` (`base_image` in Python) with any public Linux OCI image built for `amd64`, such as `python:3.13`. Docker Hub shorthand works, as does a full reference such as `ghcr.io/acme/agent-runtime:v2`. ```typescript TypeScript theme={null} import * as archil from "disk"; const disk = await archil.getDisk("dsk-abc123"); const { stdout } = await disk.exec("python3 analyze.py", { baseImage: "python:3.13", }); ``` ```python Python theme={null} import archil disk = archil.get_disk("dsk-abc123") res = disk.exec("python3 analyze.py", base_image="python:3.13") ``` ```bash REST theme={null} curl --request POST \ 'https://control.green.us-east-1.aws.prod.archil.com/api/disks/dsk-abc123/exec' \ --header "Authorization: Bearer $ARCHIL_API_KEY" \ --header 'Content-Type: application/json' \ --data '{"command": "python3 analyze.py", "baseImage": "python:3.13"}' ``` The first exec with a new image waits while Archil prepares it; later execs with the same image reuse it. For a private image, or to prepare a public image before the first call, build it with the [images API](/compute/sandboxes/images) and pass the returned `imageId` (`image_id` in Python) instead. The exec runs the image's latest successful build. An image that has not finished its first build is rejected with `409` `image_not_ready`. `baseImage` and `imageId` are mutually exclusive. ```typescript TypeScript theme={null} import { Archil } from "disk"; const client = new Archil({ apiKey: process.env.ARCHIL_API_KEY, region: "aws-us-east-1" }); const image = await client.images.build({ source: "ghcr.io/acme/agent-runtime:v2", registryAuth: { username: process.env.REGISTRY_USERNAME!, password: process.env.REGISTRY_TOKEN!, }, }); const disk = await client.disks.get("dsk-abc123"); const { stdout } = await disk.exec("agent-runtime --version", { imageId: image.imageId }); ``` ```python Python theme={null} import os from archil import Archil, RegistryAuth with Archil(region="aws-us-east-1") as client: image = client.images.build( source="ghcr.io/acme/agent-runtime:v2", registry_auth=RegistryAuth( username=os.environ["REGISTRY_USERNAME"], password=os.environ["REGISTRY_TOKEN"], ), ) disk = client.disks.get("dsk-abc123") res = disk.exec("agent-runtime --version", image_id=image.image_id) ``` Your disks are mounted under `/mnt/archil` in every image. ## Choosing a size Each command runs in a microVM with 2 vCPU and 4096 MiB of memory by default. Pass `vcpuCount` (1 to 32) and `memSizeMib` (256 to 65,536) to size it for the work, such as a build or a data-processing step that needs more memory. In Python, these are `vcpu_count` and `mem_size_mib`. ```typescript TypeScript theme={null} const { stdout } = await disk.exec("make -j8", { vcpuCount: 8, memSizeMib: 16384, }); ``` ```python Python theme={null} res = disk.exec("make -j8", vcpu_count=8, mem_size_mib=16384) ``` A larger command bills as several calls: its size as a multiple of the default, by whichever of vCPU and memory is the larger multiple, rounded up. The example above, 8 vCPU and 16 GiB, is four times the default and bills as four calls. A smaller command still bills as one call. See [Pricing & Billing](/administration/billing#serverless-tool-calls). The Free plan runs every command at the default size; a larger request is rejected with `403` `tool_call_size_limit_exceeded`. ## Restricting network access By default, a command has unrestricted network egress. On the Developer, Team, and Enterprise plans, pass `network` to apply the same egress rules as the [sandbox firewall](/compute/sandboxes/firewall), such as denying everything but a package registry: ```typescript TypeScript theme={null} const { stdout } = await disk.exec("pip install -r requirements.txt && python3 train.py", { baseImage: "python:3.13", network: { egress: { default: "deny", allow: ["pypi.org", "files.pythonhosted.org"] }, }, }); ``` ```python Python theme={null} from archil import SandboxEgressPolicy, SandboxNetwork res = disk.exec( "pip install -r requirements.txt && python3 train.py", base_image="python:3.13", network=SandboxNetwork( egress=SandboxEgressPolicy(default="deny", allow=["pypi.org", "files.pythonhosted.org"]), ), ) ``` Your mounted disks stay reachable under a deny-by-default policy. The policy applies only to that command and is not stored. On the Free plan, a request with `network` is rejected with `403` `network_plan_required`. `baseImage`, `imageId`, `vcpuCount`, `memSizeMib`, and `network` work the same way on multi-disk commands [run with `archil.exec`](#composing-multiple-disks-into-a-single-command). ## Composing multiple disks into a single command Often, users need to run commands on multiple disks at once. For example, if you're building an AI agent which has access to a read-only customer-specific disk and a read-write session-specific disk, and you want your agent to use documents on both disks. To run a command against several disks, you instead pass a map of disks into `archil.exec`. Each disk is automatically mounted to the `/mnt/archil/` path inside the sandbox. For example, in the code below, the `data` disk is mounted to `/mnt/archil/data` and the `logs` disk is mounted to `/mnt/archil/logs`. ```typescript TypeScript theme={null} import * as archil from "disk"; const { stdout } = await archil.exec({ disks: { data: "dsk-abc123", logs: "dsk-def456", }, command: "wc -l /mnt/archil/data/*.csv > /mnt/archil/logs/lines.txt", }); ``` ```python Python theme={null} import archil res = archil.exec( disks={ "data": "dsk-abc123", "logs": "dsk-def456", }, command="wc -l /mnt/archil/data/*.csv > /mnt/archil/logs/lines.txt", ) ``` ## Specifying disk options You may want to run a command in which some disks have reduced permissions, so the code running cannot modify them. Serverless execution supports specifying whether or not a disk should be marked as read-only or whether it should be limited to a subdirectory of the disk. For example, in the code below, the `logs` disk will be mounted to `/mnt/archil/logs`. That directory will be read-only, and code in the sandbox cannnot modify it. It will only display the contents of the `app/logs` subdirectory of the disk. ```typescript TypeScript theme={null} import * as archil from "disk"; const { stdout } = await archil.exec({ disks: { data: "dsk-abc123", logs: { disk: "dsk-def456", subdirectory: "app/logs", readOnly: true, }, }, command: "wc -l /mnt/archil/logs/*.log", }); ``` ```python Python theme={null} import archil from archil import ExecMountSpec res = archil.exec( disks={ "data": "dsk-abc123", "logs": ExecMountSpec("dsk-def456", subdirectory="app/logs", read_only=True), }, command="wc -l /mnt/archil/logs/*.log", ) ``` ## Consistency with other clients Serverless execution works like other Archil clients, with the same read-after-write consistency model. A file written to the disk by one command is immediately visible to the next command that reads it. By default, disks are mounted in shared mode, which means that you'll need to take care to call `archil checkout` before mutating files on the disk. For more details on how to checkout or checkin files in shared mode, see [Sharing disks](/concepts/sharing-disks). ## Limits | Limit | Value | | - | - | | Time per command | 5 minutes | | Command length (input) | 100 KiB | | Output returned | The first 4 MiB of stdout and the first 4 MiB of stderr | | Resources per command | 2 vCPU and 4 GiB of memory by default, up to 32 vCPU and 64 GiB ([choosing a size](#choosing-a-size)) | * After 5 minutes, the call returns `504` with `Exec timed out` and the command is stopped. Writes the command flushed with `fsync` before the timeout are durable; other writes are flushed on a best-effort basis as the command is stopped. For longer work, use a [persistent sandbox](/compute/sandboxes/introduction), which has no per-command timeout. * A command longer than 100 KiB is rejected with `400`. There is no separate stdin. To pass a larger input, write it to the disk first, for example with [`putObject`](/protocols/s3-api), and have the command read the file. * Output past 4 MiB on a stream is dropped, and the stream ends with `[output truncated]`. To keep larger output, write it to a file on the disk and read it with the [getObject API](/protocols/s3-api). * When a command finishes, its writes to the mounted disk are committed before the response returns. If committing fails, the command's exit code is non-zero and stderr says `archil: command output may not have been durably committed`. * Users can execute hundreds of commands in parallel, contact us to have your limits raised to thousands. ## Latency Each call starts a fresh microVM with your disks mounted. Every result reports `timing.queueMs`, the time from the request arriving to the command starting, which includes booting the microVM and mounting the disks. In our measurement from a laptop to a disk in AWS `us-west-2`, a trivial command (`true`) returned in 82 ms at the median end to end, including about 40 ms of internet round trip, with a median `queueMs` of 37 ms. The first call on a newly created disk took longer, 331 ms at the median. See [How fast is a serverless tool call?](https://archil.com/article/how-fast-is-a-serverless-tool-call-for-an-ai-agent) for the full results and a script to reproduce them. These figures are for the default image and size, which start from a pool of ready microVMs. A command with a different [image](#choosing-an-image) or [size](#choosing-a-size) boots its microVM on demand, so its `queueMs` is higher. ## Next steps * [Search files in parallel with `disk.grep`](/compute/search-files) * [Give an agent bash and file tools with the AI SDK](/integrations/ai-sdk) * [TypeScript SDK reference for `disk.exec`](/sdks/typescript#executing-commands) * [Python SDK reference for `Disk.exec`](/sdks/python#executing-commands) * [`disk` CLI reference](/reference/disk-cli) # SQLite Source: https://docs.archil.com/compute/sqlite Archil disks in AWS regions provide a native integration with SQLite that enables you to create and work with an unlimited number of SQLite databases on your disk. This is ideal for AI agents which store their context as a combination of structured and unstructured data. The Archil SQLite integration can be used from multiple clients simultaneously, even for writes. You don't need to manage any resources in order to use the SQLite APIs. Each SQLite query counts as one serverless tool call, with execution included. See [Pricing & Billing](/administration/billing#serverless-tool-calls). The SDK for the Archil SQLite integration is currently available for TypeScript users, in a separate package, which can you install with any Node package manager: ```bash theme={null} npm install disk @archildata/sqlite ``` ## Create and manage SQLite databases To create a new SQLite database on your Archil disk, use the `createDatabase` function and pass in the disk and the name of the database file. You can then use the `getDatabase` function to retrieve the database later. ```ts theme={null} import * as archil from "disk"; import { createDatabase, getDatabase } from "@archildata/sqlite"; const disk = archil.getDisk("dsk-1234"); const db = await createDatabase(disk, "apps/app-1/main.sqlite"); // Retrieve the database on a later request const db = await getDatabase(disk, "apps/app-1/main.sqlite"); ``` `createDatabase` will create a new SQLite database on the disk if it doesn't exist, or return the existing database if it does. `getDatabase` will return an existing database. ## Execute SQL queries You can execute SQL queries against each database using the `read` and `write` methods. The `read` method is used for queries that return data, while the `write` method is used for queries that modify data. ```ts theme={null} const db = await createDatabase(disk, "apps/app-1/main.sqlite"); // Create tables await db.write`CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT, email TEXT)`.run(); // Add users const username = "John Doe"; const email = "john@example.com"; await db.write`INSERT INTO users (name, email) VALUES (${username}, ${email})`.run(); // Read users const users = await db.read`SELECT * FROM users`.all(); ``` > [!NOTE] > > Why is there a difference between `read` and `write`? `write` queries need exclusive write access to the database file, while many `read` queries can be executed in parallel. You can execute multiple queries in a single transaction using the `transaction` method. This ensures that either all queries succeed or none of them are applied. ```ts theme={null} const [johnResult, janeResult] = await db.transaction([ db.write`INSERT INTO users (name, email) VALUES ('John Doe', 'john@example.com')`.run(), db.write`INSERT INTO users (name, email) VALUES ('Jane Doe', 'jane@example.com')`.run() ]); ``` # Authorization Source: https://docs.archil.com/concepts/authorization Configure who is authorized to access the Archil APIs or disk Archil supports three different kinds of resources to authorize access to: * The **Archil disk mount** for mounting the disk on Linux, on macOS, or inside containers. * The **Archil API** for creating Archil disks or sandboxes * The **Archil S3 API** for calling the S3-compatible API from AWS SDKs Mount tokens are scoped to one disk. Control-plane API keys are scoped to an account or organization within a region. They are separate credentials and cannot be used interchangeably. ## Disk mount authorization Each Archil disk manages which servers are authorized to mount the disk using the `archil` CLI or macOS application. Authorizing a server to mount the disk gives that server access to the *entire* file system. Even though Archil respects the POSIX permissions of the files stored on the disk, they should not be used as an isolation boundary for unstrusted workloads running together. Archil supports two ways to authorize a server to mount a disk: 1. **IAM-based authorization** - recommended when you are mounting a disk located in AWS from the same AWS region 2. **Token-based authorization** - recommended when you need the lowest disk mount latency You configure the authorized users for your disk on the Archil console, on the disk details page, by selecting **Connetions**, and then choosing to **Add User**, and selecting either a **Disk Token** or an **IAM User**, or via the [Add Disk User API](/api-reference/disk-users/add-user). ### IAM-based disk authorization IAM-based disk authorization allows you to configure an AWS IAM role or user ARN that is allowed to mount the disk. You can use the `archil utils get-iam-role` CLI command to get the IAM role ARN of the current EC2 instance. You can mount a disk on an IAM-authorized server by running the `archil mount` command without any additional flags. > [!NOTE] > > When using [AWS IAM Roles Anywhere](https://docs.aws.amazon.com/rolesanywhere/latest/userguide/introduction.html), you must set the `AWS_EC2_METADATA_SERVICE_ENDPOINT` and `AWS_REGION` environment variables before mounting. These can be set inline with the mount command or exported in your shell session. ### Token-based disk authorization Token-based disk authorization allows you to create a long-lived bearer token that grants the ability to mount the disk. You mount a disk using a token by setting the `ARCHIL_MOUNT_TOKEN` environment variable and passing it through sudo. ```bash theme={null} export ARCHIL_MOUNT_TOKEN="" sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount /mnt/archil --region ``` > [!NOTE] > > Using `--preserve-env=ARCHIL_MOUNT_TOKEN` passes the disk token through sudo's environment rather than the command line, preventing it from being visible in process listings (`ps aux`). This is more secure than both `sudo -E` (which passes all environment variables) and inline assignment (which exposes the token in the process list). Archil supports the ability to configure tokens to have a either a TTL (allowing mounts up for a period of time) or to be one-time use. When the token expires, it can no longer be used to mount the disk, and read/write operations on the disk from that client will fail. ## API authorization The Archil API allows you to create and manage Archil disks, disk users, and sandboxes -- including grep, SQLite, checkpoint, and S3 APIs. You can authorize to the Archil API using an API key. You can create an API key on the Archil console, by selecting **API Keys** from the sidebar and then clicking **Create API Key**. Archil API keys are scoped to the organization or personal account which created them and a specific Archil region. This means that an API key created for "aws-us-east-1" will not work to manage disks in "aws-us-west-2". API keys have full access to disks in the organization or personal account which created them, including the ability to read data, delete disks, and run sandboxes. ## S3 authorization The Archil API supports S3 APIs such as PutObject, GetObject, DeleteObject, and ListObjectsV2. You only need to create an S3-specific key if you plan on using out-of-the-box AWS S3 SDKs, which require a SigV4 signing access key and secret access key. To create an S3 access key and secret access key, use the Archil console and navigate to the disk details page. Then select **Connetions** from the sidebar, and then click **Add User**. Select **S3** as the user type, and then click **Create User**. You can then use these keys to access your Archil disk through the AWS CLI: ```bash theme={null} export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= aws s3 ls s3://dsk-abc123/ \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` # Branches & Checkpoints Source: https://docs.archil.com/concepts/branches-and-checkpoints Snapshot the state of an Archil disk and fork independent, writable copies from any snapshot. > [!NOTE] > > Branches and checkpoints are generally available in all AWS regions. Checkpoints and branches need Archil's flat-block storage format, which Archil-managed disks use on every plan. Bring your own bucket (BYOB) disks on Developer and Team use the native format, so they can't be checkpointed; flat-block BYOB disks are available on Enterprise. See [Checkpoints on BYOB disks](/administration/billing#checkpoints-on-byob-disks). Branches and checkpoints let you capture the state of an Archil disk at a moment in time and then explore alternative histories without disturbing the original. A **checkpoint** is a named, point-in-time snapshot of the filesystem state on a disk. The filesystem state a checkpoint references is immutable. A **branch** is an independent, writable fork of a disk that starts from a specific checkpoint. Writes on a branch don't affect the source branch. A checkpoint takes about 15 ms and a branch about 35 ms, regardless of disk size. [Forking a sandbox](/compute/sandboxes/introduction#forking-a-sandbox) uses the same operations on the sandbox's root disk. A sandbox with mounted disks must be stopped before forking. Branches form a tree. Every branch points back to the checkpoint on its parent it was forked from. Checkpoints, in turn, always belong to a specific branch (or to the root). > [!TIP] > > If you're familiar with Git, Archil branches and checkpoints map fairly directly to Git branches and Git commits respectively. For example: ![A timeline showing the root disk with checkpoints chk: x and chk: z, branch 1 forked from chk: x with its own checkpoint chk: y, and branch 2 forked from chk: z.](https://mintcdn.com/regattastorage/u3vrhRl3LpYuaOLm/images/branches-and-checkpoints-light.png?fit=max\&auto=format\&n=u3vrhRl3LpYuaOLm\&q=85\&s=e6a0d41cf9ee3e20ae0fbf46e488d42d) Here, the root has two checkpoints: `x` and `z`. Branch 1 was forked from `x` and branch 2 was forked from `z`. Any mutations to the filesystem state on the root after `x` would not appear in branch 1 and vice versa. ## Working with checkpoints Checkpoints are taken from a mounted disk and capture the filesystem state at the time the command runs: ```bash theme={null} # Mount the disk, then take a checkpoint sudo archil mount myorg/mydisk /mnt/archil --region aws-us-east-1 archil checkpoints create /mnt/archil pre-migration ``` Checkpoint names must be unique within the branch they're taken on. Once created, the state a checkpoint points to never changes. To see the checkpoints available on a disk: ```bash theme={null} # Checkpoints on the root archil checkpoints list dsk-0123456789abcdef --region aws-us-east-1 # Checkpoints on a specific branch archil checkpoints list dsk-0123456789abcdef --region aws-us-east-1 --branch experiment-1 ``` The list is rendered oldest-first, so the most recent checkpoint appears at the bottom. ## Working with branches Create a branch by naming the checkpoint you want it to fork from. ```bash theme={null} # Branch from a checkpoint on the root archil branches --region aws-us-east-1 create dsk-0123456789abcdef experiment-1 \ --from-checkpoint pre-migration # Branch from a checkpoint that lives on another branch archil branches --region aws-us-east-1 create dsk-0123456789abcdef experiment-2 \ --from-checkpoint v1 --from-branch experiment-1 ``` To use a branch, mount it with `--branch`: ```bash theme={null} sudo archil mount myorg/mydisk /mnt/branch --region aws-us-east-1 --branch experiment-1 ``` A mounted branch behaves exactly like any other Archil disk. You can read, write, take additional checkpoints from it, and create further branches off of those checkpoints. Writes to a branch are isolated from its parent and from any sibling branches. Because each branch is fully independent, you can mount it on any client, including a different VM from the one using its parent. Two branches of the same disk can also be mounted on separate VMs at the same time. This makes branches a natural fit for fan-out workloads such as giving each agent or CI job its own writable fork. > [!NOTE] > > A branch is not a security boundary. Credentials are scoped to the entire disk, including all of its branches and checkpoints. Anyone who can mount the disk can access every branch and checkpoint on it. To isolate untrusted workloads from each other (for example, one agent or tenant per fork), give each its own disk rather than relying on branches. See [Disk Users](/concepts/disk-users). > [!WARNING] > > Deleting individual branches and checkpoints is currently not supported. We're actively working on this feature, and if you need access to it sooner, please email [support@archil.com](mailto:support@archil.com). For the full set of CLI flags, see [`archil checkpoints`](/reference/archil-cli#checkpoints) and [`archil branches`](/reference/archil-cli#branches) in the CLI reference. For a worked example, see [Running parallel agent attempts from one starting state with copy-on-write branches](https://archil.com/article/running-parallel-agent-attempts-from-one-starting-state-with-copy-on-write-branches). # Data Sources Source: https://docs.archil.com/concepts/data-sources Configure bidirectional synchronization for Archil disks to existing data sets Archil disks use a bucket managed by Archil by default. With bring your own bucket (BYOB), your bucket stores the data and Archil synchronizes with it bidirectionally. Active data is temporarily held in Archil's storage layer in both modes. Connecting a data source is useful when you have an existing data set stored in cloud object storage, like Amazon S3, that you want to expose to agents as a native Linux file system. If, instead, you need to simultaneously use strongly consistent data from both the file system and an S3 API for a net-new data set, we instead recommend that you use Archil's built-in support for the [S3 API](/protocols/s3-api) to access your data. > [!NOTE] > > BYOB disks on Developer and Team use the native format, which stores each file as an ordinary object in your bucket. [Checkpoints and branching](/concepts/branches-and-checkpoints) need the flat-block format, which Archil-managed disks use on every plan and BYOB disks can use on Enterprise. See [Checkpoints on BYOB disks](/administration/billing#checkpoints-on-byob-disks). ## Supported data sources Archil supports a wide variety of data sources out of the box. Each data source has specific configuration requirements and credential setup procedures. For detailed configuration instructions for each supported data source, see the dedicated pages: * [**S3-compatible Object Storage**](/data-sources/s3-object-storage) - including Amazon S3, Google Cloud Storage, Cloudflare R2, and [S3-compatible providers](/data-sources/s3-object-storage#generic-s3-compatible-storage) such as Tigris, MinIO, Wasabi, Backblaze B2, and DigitalOcean Spaces ## Consistency Once a write is committed to Archil, reads that reach the disk server, including the [Archil S3 API](/protocols/s3-api), see it immediately. Mounted clients can continue to serve cached data until their caches refresh. A local write must first be flushed with `fsync` or a background flush before other clients can read it from the server. See [Consistency](/details/consistency). Writes synchronize asynchronously to the backing bucket. Reading that bucket directly can therefore show an older version even after a write is committed to Archil. To reduce the number of operations on backing object storage, Archil automatically combines multiple writes to individual files into a single PutObject call to the data source. Archil copies a file once it has gone 15 seconds without a write, so it usually reaches the data source tens of seconds after its last write; see [Consistency](/details/consistency#from-the-disk-to-the-bucket) for the full timing. Before writes appear in the backing object storage, they are stored durably on Archil's SSD storage servers. Archil redundantly stores data across multiple servers, in multiple Availability Zones, so the failure of any individual node (or an entire Availability Zone) does not result in data loss. See [Data durability](/details/architecture#data-durability) for details. Archil does not offer a way for users to force a synchronization to the backing data source. If you find yourself needing to force synchronizing to the backing bucket, we instead recommend moving S3 API clients to the Archil S3 API, which is strongly consistent with changes in the file system. Synchronization from a data source is bidirectional. To reduce read latencies (often by up to 30x), Archil caches active data from a data source on our high-performance SSD storage servers. As a result, a newly created object on the backing storage system can take minutes to appear in the corresponding directory on disk; see [From the bucket to the disk](/details/consistency#from-the-bucket-to-the-disk). ## Concurrent writes Archil supports concurrently modifying the disk from both the file system and from the origin data source -- as long as these writes occur to different locations on the disk. If you attempt to write to the same file or directory which is being actively modified via the file system, it may result in undefined behavior including data loss or corruption. If you need to concurrently modify the same directory from the file system and the S3 API, we recommend using Archil's built-in S3 API which supports concurrent modification. ## Atomicity Some data sources, such as Amazon S3, do not have atomic operations that mirror each of the possible POSIX file operations (for example, directory renames). When executing these kinds of operations against an Archil disk, you may notice partial results for the operations when reading directly from the underlying data source. Reads that reach the Archil disk server are strongly consistent; mounted clients may serve a cached view. ## Availability If the configured data source becomes unreachable while a disk is mounted, writes are not lost: Archil buffers them durably in the disk's SSD storage cache and flushes them to the data source once it is reachable again. This SSD cache for unacknowledged writes has a fixed size to maintain the durability of the disk, and if you exceed this size, the disk will stop accepting write operations. Reads served from cache are unaffected; a read that misses the cache and requires the data source blocks until it can be satisfied. ## POSIX metadata When Archil synchronizes a disk to a data source, file contents are written as ordinary objects at the matching key. Archil will internally track the POSIX metadata for each file, including its mode bits, ownership information, and any extended attributes. These are not visible by reading the bucket directly. Symbolic links are stored as a text file that contains just the path that the link points to. Hard links are stored as a full copy of the linked data. ## Unicode file names Like most Linux file systems, Archil treats a file name as an exact sequence of bytes and doesn't apply Unicode normalization. On a disk with a data source, each name is written to the bucket as an object key with exactly those bytes. Some object storage providers normalize object keys instead of storing them byte-for-byte. These include [Cloudflare R2](https://developers.cloudflare.com/r2/reference/unicode-interoperability/) and MEGA S4: both convert keys to Unicode Normalization Form C (NFC). Amazon S3 stores keys exactly as written. The same visible text can be encoded in more than one Unicode form. For example, the Korean `폴더` or the accented `é` can be stored precomposed (NFC) or as a sequence of decomposed characters (NFD). On a provider that normalizes keys, names in different forms map to the same object, and the keys the bucket returns can differ from the names Archil wrote. File system operations on affected names, such as renaming a directory, can fail and can stop the disk from accepting writes. > [!WARNING] > > If your data source normalizes Unicode, you're responsible for using NFC file names on the disk. Names that contain > only ASCII characters aren't affected. > > macOS often creates decomposed (NFD) names for non-ASCII characters, for example when you create or rename files > and folders in Finder with Korean, Japanese, or accented Latin characters. Avoid this on disks backed by a provider > that normalizes, or convert names to NFC before writing them. For example, use `unicodedata.normalize("NFC", name)` > in Python, or `name.normalize("NFC")` in JavaScript. On Cloudflare R2, we recommend contacting your Cloudflare account team to opt your account out of Unicode normalization, so that R2 stores keys byte-for-byte. For other S3-compatible providers, check the provider's documentation or support team to find out whether it normalizes object keys. ## Deleting a disk Deleting an Archil disk does not delete the data in its data source. An Archil disk is a synchronized view over the bucket you own — when you delete the disk, Archil stops synchronizing and discards its cached copy, but the objects already written to your data source remain in your bucket, under your control. ## Deleting a data source If you delete the data source while an Archil disk remains connected to it, the associated disk will stop accepting read, write, or mount operations. This process is eventually consistent, and some operations may continue to function after the deletion ofr a data source for a short period of time. # Metering Source: https://docs.archil.com/concepts/metering How Archil measures active data in the high-speed storage layer Archil disks use a high-speed, durable cache to enhance performance when working with data from [data sources](/concepts/data-sources). Active-storage charges measure data stored in this cache layer. ## File data When reading data from a data source, Archil automatically caches it in the high-speed storage layer. Cached data remains active for up to 1 hour after your last access. Writes are buffered in the high-speed storage layer for coalescing before being flushed to the data source. Recently written data remains active for up to 1 hour after it has been flushed. The amount of active data may not exactly match the amount you've read or written. Archil prefetches and caches data ahead of sequential reads, which can result in additional cached data. Rapid overwrites to the same file will also result in additional active data while Archil asynchronously coalesces and flushes the writes. ## File metadata Metadata accesses (e.g., `find`, `ls`, `stat`) do not cause file data to become active and incur no active-storage charges. Metadata writes (e.g., rename, `chmod`) are counted as 32 KiB of active data per operation until the change is propagated to the data source. Taking ownership of a path with [`archil checkout`](/reference/archil-cli#checkout) lets Archil coalesce the many small attribute updates a workload produces into larger chunks before metering them. For example, `git clone`-ing a 15 MB repository into a checked-out directory is metered as roughly fifteen 1 MiB writes rather than the thousands of individual metadata operations the client issues. ## How active storage is measured Archil meters the amount of active data each minute. Active-storage usage is the time-weighted average over the billing period. For example, 100 GiB of active data for half a month is 50 GiB-months. Archive storage is charged separately for file systems that do not synchronize to customer-owned buckets. See [storage pricing](/administration/billing#storage) for details. You can monitor your active data usage in the [Archil console](https://console.archil.com) on each disk's **Usage** tab, or on the [Billing](https://console.archil.com/billing) page for an account-wide view. ## Branches and checkpoints [Branches and checkpoints](/concepts/branches-and-checkpoints) are copy-on-write. Forking a branch or taking a checkpoint does not duplicate the underlying data — you are billed once for the shared base data, plus only the unique data each branch or checkpoint adds. For example, a 100 GiB parent disk with 50 branches that each modify \~5 GiB of unique data is metered as roughly **350 GiB total** — the shared 100 GiB base plus the \~250 GiB of unique per-branch changes — not as 51 full copies (\~5,100 GiB). A checkpoint retains its unique data for as long as it exists; deleting a checkpoint removes that data from your bill. # Sharing Disks Source: https://docs.archil.com/concepts/sharing-disks Archil disks can be shared between multiple servers simultaneously Archil supports sharing disks between multiple connected users simultaneously. This easily allows you to support workflows where multiple agents are collaborating on the same dataset, or if multiple servers need to have live access to the same data set. Archil delivers the local-like disk performance by matching the semantics of xfs, ext4, or zfs and enabling connected clients to execute file system operations locally. This allows Archil to exceed the performance of comparable NFS or 9p systems, in which file system operations must always be performed on the remote server. In conditional mode, any number of clients can write to a disk at the same time, including the same directories and files, with no setup. A client can also check out a path, called a "delegation", to run operations on it locally at local-disk speed. Checkouts are optional. Archil supports three mount modes. When several clients write to one disk, use conditional mode. * **Exclusive mode** - By default ([`archil mount`](/reference/archil-cli#mount)), mounting an Archil disk puts it in exclusive mode. This locks the entire file system for the machine which mounted. That machine will receive local-like performance that matches an EBS volume, but other machines will not be able to write to the disk until it's unmounted. Other machines can still mount the disk with `archil mount --read-only` to read it. * **Shared mode** - If you mount the disk in shared mode (`archil mount --shared`), no lock is taken out on the disk. By default, the entire disk will appear read-only to the client until it manually requests a delegation ([`archil checkout `](/reference/archil-cli#checkout)). With a delegation, the client will have exclusive write access to the specified `` and receive local-like performance. * **Conditional mode** - If you mount the disk in conditional mode (`archil mount --conditional`), you enable the disk to execute operations remotely, like an NFS or 9p file system. This means that all files (which are not write-locked by another client) are immediately writeable, but the disk performance will be reduced to that of a remote file system. Users in conditional mode can continue to use `archil checkout` to get exclusive write access and higher performance to subdirectories on the disk. We intend to make **Conditional mode** the default for new disks soon. The remainder of this article will describe how Shared mode and delegations work, followed by how [Conditional mode](#conditional-mode) works. If several agents or sandboxes need to write to one disk, start with [Sharing a disk between agents](#sharing-a-disk-between-agents). ## Choosing a mode | | Exclusive | Shared | Conditional | | - | - | - | - | | Mount command | `archil mount` | `archil mount --shared` | `archil mount --conditional` | | Clients that can mount at once | One, plus any number of read-only clients | Many | Many | | Where the client can write | Anywhere | Paths it has checked out | Anywhere another client hasn't checked out | | Needs `archil checkout` to write | No | Yes, though some operations check out automatically | No | | Write performance | Local disk | Local disk | Remote file system; local disk on paths it has checked out | | `close` waits for the file's writes to reach the disk | No | No | Yes, except under a delegation it holds | Whatever the mode, a write is durable once `fsync` returns, and another client sees it the next time it opens the file. [Consistency](/details/consistency#between-archil-clients) has the exact rules, including how long a client can serve cached attributes and directory listings. ## Sharing a disk between agents When agents in several sandboxes share one disk, this pattern works on every platform where you run `archil mount`: 1. **Mount in conditional mode in every sandbox** (`archil mount --conditional`). Every sandbox can read and write the disk without `archil checkout`. In [Archil sandboxes](/compute/sandboxes/introduction) and [serverless execution](#selecting-a-mode-on-serverless-execution), set `conditional: true` on the mount instead. 2. **Give each agent its own directory to write,** such as `/mnt/archil/agents//`. Every agent can still read and write every directory, including the same files. If two agents write the same part of a file, the last write to reach the disk wins. 3. **Make the writer's output durable before you hand it off.** When an agent finishes, run `archil sync `, or `fsync` the files it wrote, before it reports that it's done. Once that returns, the files are durable, and any client that opens them reads the new contents. 4. **Hand off through your orchestrator, not the file system.** Pass "the output is ready" through your queue, API, or the agent's tool result. A reader that polls the disk for a new file can keep getting an answer from its cache. 5. **Refresh the reader's cache before it reads.** In the sandbox that picks up the work, run `archil invalidate-cache `, so directory listings and attributes it cached before the handoff don't hide the new files. ```bash theme={null} # Writer, in its own sandbox sudo archil mount "$ARCHIL_DISK_ID" /mnt/archil --region "$ARCHIL_REGION" --conditional mkdir -p /mnt/archil/agents/writer # ... the agent writes its output under /mnt/archil/agents/writer ... sudo archil sync /mnt/archil # returns once every write is durable # ... then tell your orchestrator the output is ready # Reader, in another sandbox, once your orchestrator says the output is ready sudo archil invalidate-cache /mnt/archil cat /mnt/archil/agents/writer/result.json ``` Some variations: * **To have Archil enforce ownership** of an agent's directory, run `archil checkout /mnt/archil/agents/` in that agent's sandbox. Other sandboxes can read the directory but get `EROFS` if they write to it, and the agent gets local-disk speed there. Check it in with `archil checkin` when the agent is done. If the sandbox is killed first, the directory stays locked until you [force the delegation off](#forcing-ownership). * **For many small files,** such as `npm install` or a `git` checkout, check out the directory first. In conditional mode every create, rename, and delete waits for a round trip to the disk; under a checkout they run at local-disk speed. * **Unmount before the sandbox stops.** `archil unmount` sends every buffered write. Writes that haven't reached the disk are lost if the sandbox is killed first. See [Troubleshooting mounts on other platforms](/guides/platforms/troubleshooting). ## Selecting a mode on serverless execution By default, disks in [serverless execution](/compute/serverless-execution) are mounted in **Shared mode** (`archil mount --shared`) to enable multiple sandboxes to easily access the same data. You can change the disk into **Exclusive mode** by first executing `archil checkout /mnt/archil` as part of your serverless execution command. You can select **Conditional mode** by using the more verbose serverless execution API, like below: ```ts theme={null} await archil.exec({ disks: { data: { disk: "dsk-def456", conditional: true, }, }, command: "ls /mnt/archil/data", }); ``` In this example, the disk will be mounted to `/mnt/archil/data` instead of `/mnt/archil`. ## Automatic delegations in shared mode When mounting in **Shared mode**, some operations automatically grant a write-lock on the file, to reduce the number of times that you need to call `archil checkout`. These operations are: * Creating a new file inside of a directory that does not have a delegation * Opening a file that is not covered by a delegation for writing (delegation automatically released on close) * Creating a new directory These operations will automatically grant a delegation to the client who ran the operation. If you want other clients to have write-access to these resources, you will need to manually call [`archil checkin`](/reference/archil-cli#checkin) on the resource after creation. These rules lead to a common pitfall with Archil in which it's possible to create a file or directory, but a removal operation will fail with `EROFS: Read-only file system`. This is because of the automatic delegation granting on file/folder creation. Deleting a file or folder is a directory-level operation, and requires a delegation on the parent directory of the file being removed. > [!NOTE] > > These rules apply only to clients in **Shared mode**. Clients mounting in **Conditional mode** will never have delegations > automatically issued to them. ## Understanding delegations Each delegation is an exclusive write-lock on a part of an Archil disk for a single client. When you call `archil checkout `, all files and folders underneath that path will be included as part of the write delegation. This will prevent other write operations, including [S3 PutObject](/protocols/s3-api) operations, from completing against those paths. For example, consider a file system with the following structure: ``` / ├── group/ │ └── models/ │ ├── model1.txt │ └── model2.txt └── data/ └── data.txt ``` If one client calls `archil checkout group/` to get exclusive write-access to the "group" directory, it will also have exclusive write access to the `group/models/` directory and its contents. Other clients will be able to continue to read these files and folders but will not be able to write to them, or call PutObject. Unless it's mounted in **conditional mode**, this client would not be able to edit any other part of the disk. Other clients are simultaneously able to call PutObject to add files to the `data/` directory or get exclusive write access over that directory by calling `archil checkout data/`. When your client is operating in **conditional mode**, write operations are executed on the server-side, and multiple clients can simultaneously read and write to any part of the file system without a delegation. ## Releasing delegations Delegations represent an exclusive write-lock on a part of the Archil disk. As a result, they need to be manually released by clients to signal that they are no longer performing writes to that portion of the disk. You can do this by either unmounting the disk from the server which requested the delegation or by manually calling `archil checkin `. ## Forcing ownership A client in conditional mode that hasn't checked out a path holds no delegation, so if it crashes there is nothing to release. In some cases, a client may crash or poweroff without cleanly unmounting or calling `archil checkin` to release a delegation. If this happens, you will not automatically be able to write to the location where the client's delegation was held: Archil doesn't release a crashed client's checkouts, or the whole-disk delegation of an exclusive mount, on its own. Mounting again on the same machine doesn't release them either, because every mount is a new client. Instead, you will need to force release the delegation. This will cause any in-progress writes from the original client to be lost. This mirrors "force detaching" a block device, such as an EBS or Hyperdisk disk. You can force checkout a delegation by calling `archil checkout --force`, and you can force mount a disk into exclusive mode by calling `archil mount --force`. You can also remove delegations using the Archil API, if you don't have the disk mounted locally: ```typescript Typescript theme={null} const disk = await archil.getDisk("dsk-abcdef") for (const delegation of await disk.listDelegations()) { // If the client which issued the delegation is no longer // connected, revoke the delegation. if (delegation.isOrphaned) { await disk.revokeDelegation(delegation) } } ``` ## Conditional mode When mounting in **Conditional mode**, a client can write to any file or directory on the disk without calling `archil checkout`, as long as no other client holds a delegation on it. This makes Conditional mode a good fit when several clients need to write to the same disk, such as multiple agents working in a shared workspace. Creating, renaming, and deleting files and directories are executed on the server before the operation returns. Writes to file contents are buffered on the client and sent to the server in the background, like on any Archil mount. Closing the file or calling `fsync` waits for them to reach the disk. Any number of clients in conditional mode can write the same file. If two clients write to the same part of a file, the last write to reach the disk wins. If several clients need to update the same file, coordinate them in your application, or use `archil checkout` to give one client exclusive access to it. A client in conditional mode can still call `archil checkout ` to get exclusive write access and higher performance on part of the disk. While a client holds a delegation, other clients in conditional mode can't write to that path. If a client in conditional mode still has unsent writes to that path when another client checks it out, those writes are lost and its mount becomes read-only; see [Troubleshooting](/reference/troubleshooting#problem-writes-fail-on-a-conditional-mount). # S3 Object Storage Source: https://docs.archil.com/data-sources/s3-object-storage Configure S3-compatible storage providers as data sources for your Archil disk This page applies only when you bring your own bucket (BYOB). By default, Archil disks use Archil-managed storage and need no bucket setup. See [Data sources](/concepts/data-sources). Before Archil can read or write to your bucket, the bucket needs to grant Archil access. This is a one-time setup per bucket, regardless of how you use the disk afterwards (mount, `disk exec`, anything else). The Archil console walks you through this when you create a disk in the UI. This page documents what the console is doing, for setting things up by hand or scripting it. ## Amazon S3 [ Amazon S3](https://aws.amazon.com/s3/) supports two ways to authorize Archil: * **Bucket resource policy** — recommended. Archil's IAM role assumes access to your bucket via a resource policy you attach to the bucket. No long-lived credentials change hands. * **Static IAM credentials** — paste an access key and secret key from an IAM user that has read/write access to the bucket. Simpler to set up, but you're responsible for rotating the credentials. ### Bucket resource policy In the [AWS S3 console](https://console.aws.amazon.com/console/), select your bucket, open the **Permissions** tab, and edit the **Bucket policy**. Add the following statement, substituting `YOUR-BUCKET-NAME` and your disk's session ID for `YOUR-DISK-SESSION-ID` (shown in the [Archil console](https://console.archil.com), e.g. `asi_us-east-1_tl02r28ami6phda70d1pv`): ```json theme={null} { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowArchilAccess", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::904233114299:role/archil-s3.prod.us-east-1" }, "Action": [ "s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Resource": [ "arn:aws:s3:::YOUR-BUCKET-NAME", "arn:aws:s3:::YOUR-BUCKET-NAME/*" ], "Condition": { "StringEquals": { "aws:PrincipalTag/archilSessionId": "YOUR-DISK-SESSION-ID" } } } ] } ``` The `archilSessionId` condition scopes the grant to a single Archil disk, so the bucket is only reachable by that one disk — every other disk in your organization is denied. This is the right default if you point one disk at the bucket. The action list above is the minimum Archil needs: `s3:ListBucket` for bucket existence checks and listings, `s3:GetObject` for reads, `s3:PutObject` for writes (including multipart uploads), and `s3:DeleteObject` for object removal. > [!NOTE] > > Archil doesn't call `AbortMultipartUpload` directly, so it's not required in the policy. However, on rare failure paths a multipart upload can be left incomplete, and the parts continue to incur storage cost until they're cleaned up. We recommend either adding `s3:AbortMultipartUpload` to the action list, or attaching an [S3 lifecycle rule](https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpu-abort-incomplete-mpu-lifecycle-config.html) that aborts incomplete multipart uploads after a few days. The exact `Principal` ARN varies by Archil region — use the table below to find yours, or copy the pre-filled policy from the [Archil console](https://console.archil.com). | Cloud | Region | Principal ARN | | - | - | - | | AWS | us-east-1 | `arn:aws:iam::904233114299:role/archil-s3.prod.us-east-1` | | AWS | us-east-2 (preview) | `arn:aws:iam::787372716928:role/archil-s3.prod.aws.us-east-2` | | AWS | us-west-2 | `arn:aws:iam::795289710871:role/archil-s3.prod.aws.us-west-2` | | AWS | eu-west-1 | `arn:aws:iam::794252461707:role/archil-s3.prod.aws.eu-west-1` | | GCP | us-central1 | `arn:aws:iam::025016447135:role/archil-s3.prod.gcp.us-central1` | #### Authorizing every disk in your organization If you point many disks at the same bucket — for example, one disk per tenant or per workspace — swap the condition to match on `archilAccountId` instead. That's your Archil organization ID (shown in the [Archil console](https://console.archil.com), e.g. `org_3CjfHc6i0vXj24vu0Wpl8twU3nk`) and it's the same for every disk you create: ```json theme={null} "Condition": { "StringEquals": { "aws:PrincipalTag/archilAccountId": "YOUR-ARCHIL-ACCOUNT-ID" } } ``` With this condition every current and future disk in your organization can use the bucket without you having to edit the policy again. The trade-off is that any disk in your organization can reach this bucket, so use it when you control all the disks under that organization. ### Server-side encryption with KMS (SSE-KMS) If your bucket is encrypted with **SSE-S3** (the default `AES256` option), nothing extra is needed — the bucket policy above is sufficient. If your bucket is encrypted with **SSE-KMS** using a customer-managed KMS key, you also need to grant Archil's IAM role permission to use that key. Open the key in the [AWS KMS console](https://console.aws.amazon.com/kms/) (the key's ARN is shown at the top of its detail page), edit the **Key policy**, and add the following statement. Substitute `YOUR-KMS-KEY-ARN`: ```json theme={null} { "Sid": "AllowArchilToUseKey", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::904233114299:role/archil-s3.prod.us-east-1" }, "Action": [ "kms:Decrypt", "kms:GenerateDataKey", "kms:DescribeKey" ], "Resource": "YOUR-KMS-KEY-ARN" } ``` `kms:Decrypt` lets Archil read objects, `kms:GenerateDataKey` lets it write new objects under envelope encryption, and `kms:DescribeKey` is occasionally used during client setup. > [!NOTE] > > SSE-KMS with the AWS-managed `aws/s3` key does **not** work for cross-account access. AWS-managed keys can't be shared outside the account that owns them, so a bucket encrypted with `aws/s3` is unreachable from Archil's role. To use SSE-KMS with Archil, the bucket must use a [customer-managed KMS key](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#customer-cmk). Archil never sets `ServerSideEncryption` headers on its requests, so whatever default encryption you configure on the bucket is what objects land with. You don't need to tell Archil which key you're using. ### Static credentials If you'd rather not edit the bucket policy, create an IAM user with `s3:*` permissions on the bucket and pass its access key ID and secret to Archil when you create the disk: ```typescript TypeScript theme={null} import * as archil from "disk"; await archil.createDisk({ name: "my-disk", mounts: [{ type: "s3", bucketName: "my-bucket", accessKeyId: "AKIA...", secretAccessKey: "...", }], }); ``` ```python Python theme={null} import archil from archil import S3Mount archil.create_disk( name="my-disk", mounts=[S3Mount( bucket_name="my-bucket", access_key_id="AKIA...", secret_access_key="...", )], ) ``` ## Google Cloud Storage [ Google Cloud Storage](https://cloud.google.com/storage) authorizes Archil through static AWS-compatible HMAC credentials. 1. Open the [Google Cloud Storage console](https://console.cloud.google.com/storage/). 2. Click **Settings**, then **Interoperability**. 3. Under **Service account HMAC**, click **Create a key for another service account**. 4. Grant the **Cloud Storage** → **Storage Object Admin** role to the new service account. 5. Record the **Access key** and **Secret** for the new service account HMAC key. Pass them when you create the disk: ```typescript TypeScript theme={null} await archil.createDisk({ name: "my-disk", mounts: [{ type: "gcs", bucketName: "my-bucket", accessKeyId: "GOOG1EABCD1234567890", secretAccessKey: "1234567890abcdef1234567890ABCD/EF", }], }); ``` ```python Python theme={null} from archil import GCSMount archil.create_disk( name="my-disk", mounts=[GCSMount( bucket_name="my-bucket", access_key_id="GOOG1EABCD1234567890", secret_access_key="1234567890abcdef1234567890ABCD/EF", )], ) ``` ## Cloudflare R2 [ Cloudflare R2](https://www.cloudflare.com/r2/) authorizes Archil through static AWS-compatible credentials. 1. Open the [Cloudflare console](https://dash.cloudflare.com/login). 2. Browse to **R2** and click **Manage R2 API Tokens**. 3. Create a new token with **Object Read & Write** permissions. 4. Record the **Access Key ID**, **Secret Access Key**, and the default **endpoint** (looks like `https://.r2.cloudflarestorage.com`). > [!WARNING] > > By default, R2 [normalizes object keys to Unicode NFC](https://developers.cloudflare.com/r2/reference/unicode-interoperability/), but Archil stores file names byte-for-byte. We recommend contacting your Cloudflare account team to opt your account out of Unicode normalization. Otherwise, you're responsible for using NFC file names on the disk. See [Unicode file names](/concepts/data-sources#unicode-file-names). Pass them when you create the disk: ```typescript TypeScript theme={null} await archil.createDisk({ name: "my-disk", mounts: [{ type: "r2", bucketName: "my-bucket", accessKeyId: "R2_ACCESS_KEY_ID", secretAccessKey: "R2_SECRET_ACCESS_KEY", endpoint: "https://.r2.cloudflarestorage.com", }], }); ``` ```python Python theme={null} from archil import R2Mount archil.create_disk( name="my-disk", mounts=[R2Mount( bucket_name="my-bucket", access_key_id="R2_ACCESS_KEY_ID", secret_access_key="R2_SECRET_ACCESS_KEY", bucket_endpoint="https://.r2.cloudflarestorage.com", )], ) ``` ## Generic S3-compatible storage Many providers expose S3-compatible APIs and work with Archil out of the box. To configure one: 1. Get the S3-compatible API endpoint from your provider. 2. Create or obtain access credentials (Access Key ID and Secret Access Key) with read/write/list/delete permissions on the bucket. 3. When creating your disk, select **Generic S3 Compatible** as the data source type and provide the endpoint plus credentials. Archil uses path-style addressing (`/`) for every S3-compatible endpoint, and signs requests for the region `auto` unless you set a bucket region (`bucketRegion`). The provider must accept path-style requests. > [!WARNING] > > Some providers, such as MEGA S4, normalize object keys to Unicode NFC, but Archil stores file names byte-for-byte. If your provider normalizes keys, you're responsible for using NFC file names on the disk. See [Unicode file names](/concepts/data-sources#unicode-file-names). ### Tigris [ Tigris](https://www.tigrisdata.com/) works as a generic S3-compatible data source. Create an access key with read and write access to the bucket in the Tigris console, then use the endpoint `https://t3.storage.dev` and leave the region unset (Archil uses `auto`, which is the region Tigris expects). ```typescript TypeScript theme={null} await archil.createDisk({ name: "my-disk", mounts: [{ type: "s3-compatible", bucketName: "my-bucket", bucketEndpoint: "https://t3.storage.dev", accessKeyId: "tid_...", secretAccessKey: "tsec_...", }], }); ``` Tested with: * [ MinIO](https://min.io/) * [ DigitalOcean Spaces](https://www.digitalocean.com/products/spaces) * [ Wasabi](https://wasabi.com/) * [ Tigris](https://www.tigrisdata.com/) * [ Backblaze B2](https://www.backblaze.com/b2/) (with S3-compatible API) * And many others # Architecture Source: https://docs.archil.com/details/architecture Overview of Archil disk architecture Archil disks provide a POSIX-compatible file system backed by object storage. In managed mode, the backing bucket is managed by Archil. In bring your own bucket (BYOB) mode, the bucket belongs to you. In both modes, active data is temporarily held in Archil's high-performance storage layer, and newly committed writes are durable there while they synchronize to the backing bucket. With Archil's custom file protocol, developers can achieve high-speed access to shared data, even if it requires lots of small files. ## Core components > [card] [Client instances](/mounting/linux) > > Connect to disks using the Archil client > [card] Archil storage system > > High-performance, durable caching layer > [card] [Object storage](/concepts/data-sources) > > A bucket managed by Archil, or a bucket you own The Archil storage layer runs in Archil-managed cloud accounts, or in your own environment with [BYOC](/reference/byoc) on AWS, Google Cloud, Azure, or bare metal. The backing bucket belongs to Archil in managed mode and to you in BYOB mode. ### Archil file protocol The Archil client implements a new, custom storage protocol that enables full POSIX-compliance and high-performance for small-file workloads, such as git. The Archil client supports all Linux distributions. For more information, see the reference documentation for the [Archil client](/reference/archil-cli). On Linux, the client is a FUSE file system: it needs `/dev/fuse` and root, and it does not use NFS or SMB. On macOS, it is an FSKit file system extension. Both speak the same Archil protocol to the storage system over TLS. See [How the client mounts](/mounting/linux#how-the-client-mounts). Applications that can't mount a file system can reach the same disk through the [S3 API](/protocols/s3-api) and the [SDKs](/sdks/typescript). ### Archil storage system Archil disks are hosted on a shared cloud service, the Archil storage system. This centralized, durable cache stores frequently accessed data and recently written data. The cache provides near-infinite storage that automatically expands to the working set size of your application. The Archil storage service is designed to provide sub-millisecond latency for data access. Because the Archil cache is shared across all clients, you can achieve low-latency access to reference data from multiple servers simultaneously. The Archil storage system efficiently translates POSIX-like requests into other protocols, such as Amazon S3, even when those operations (such as rename) are not supported by the underlying data source. Archil disks are always strongly read-after-write consistent for reads that reach the server, including the [Archil S3 API](/protocols/s3-api). Writes are durable as soon as `fsync()` returns. Mounted clients may continue to serve cached data until their caches refresh; see [Consistency](/details/consistency#client-side-caching). Archil also copies writes to the backing bucket in the background. Clients that read that bucket directly see a separate synchronization delay. ### Security Archil encrypts all data at rest (AES-256) and in transit (TLS 1.2 or higher), with access control via IAM roles and scoped API keys. Archil is SOC 2 certified and HIPAA/GDPR compliant. For full details, see [Security & Compliance](/details/security). ### Data durability Archil is designed for **99.999% (5 nines) durability** for newly written data before it is synchronized to object storage, and **99.99% (4 nines) availability**. We can sign an availability SLA for Enterprise customers; [contact us](mailto:sales@archil.com) to discuss it. The 5-nines figure applies as soon as a write reaches the disk, before its copy reaches your data source (see [Consistency](/details/consistency#from-the-disk-to-the-bucket)). Once the copy is in your data source, it also has that data source's own durability. A write that is still buffered in a client hasn't reached the disk yet, and is lost if the client stops before sending it. `fsync`, `archil sync`, and `archil unmount` return only after the client's writes have reached the disk. Archil's caching layer automatically stores data redundantly within multiple Availability Zones in the disk's region. A successful `fsync()` guarantees that the written data has been committed redundantly across these Availability Zones; the 5-nines durability applies from the moment `fsync()` returns, before the data has synchronized to your data source. There is no single point of failure that can cause data loss. An Archil disk is not a single server or cache node — if an individual node, or even an entire Availability Zone, fails before your data has synchronized to the data source, the disk continues serving your data from the remaining Availability Zones and synchronization completes normally. No data is lost, and no action is required on your part. # Client caching Source: https://docs.archil.com/details/caching How the Archil client caches data in memory, and the controls for warming, sizing, and refreshing it. Archil caches your data at two distinct layers: * The **Archil storage system cache** is a durable, multi-Availability-Zone caching layer that runs server-side, between your clients and your [data source](/concepts/data-sources). It's where your active data is durably stored (a successful `fsync()` commits to it) and where it's [metered](/concepts/metering), and it serves data with sub-millisecond latency. See [Architecture](/details/architecture) for how it fits in. * The **client-side cache** is an in-memory cache inside each Archil client — one per mount. It holds recently used data so that repeated access is served locally, without even a round-trip to the storage system. This page is mostly about the **client-side cache**: what it holds, how it's sized and evicted, and the commands you use to warm, refresh, and tune it. To warm the storage system cache without a mounted client, see [Warming the storage system cache with the API](#warming-the-storage-system-cache-with-the-api). ## What the client caches The client maintains several in-memory caches, all populated on demand as your application reads and writes: * **File data** — the contents of files. This is the bulk of the cache. * **File and directory attributes** — metadata such as size, timestamps, and permissions (the result of a `stat`). * **Directory listings** — the entries returned by `readdir` (i.e., `ls`), so repeated traversals of the same directory don't round-trip to the server. * **Extended attributes** — when [xattrs](/reference/archil-cli#mount) are enabled. When your application requests data that isn't in the client cache, the client fetches it from the Archil storage system — which serves it from its durable cache (sub-millisecond) or, on a miss there, from the backing data source (for Amazon S3, typically 10–30 ms) — and then caches the result in memory for future requests. ## Cache size and eviction The client's in-memory data cache is bounded by two settings on [`archil mount`](/reference/archil-cli#mount): * `--max-cache-mb` — the hard ceiling on the client cache size. Default: 1/4 of available RAM or 2048 MiB, whichever is smaller. * `--target-cache-mb` — the size the client aims to keep the cache near, leaving headroom for bursts of new reads. Default: 75% of the maximum. When the cache fills, the client evicts older entries to make room for new data. Two kinds of entries are never evicted: * **Unflushed (dirty) writes** — data you've written that hasn't yet been committed to the storage system is always preserved locally, so eviction can never lose your writes. * **Entries still in use** — anything an application currently has open is kept until it's released. > [!NOTE] > > Because the client cache is bounded, reading or pre-fetching more data than `--max-cache-mb` can hold will evict earlier entries before you use them. Size the cache for your working set when you plan to keep a large data set hot on a client. For example, to give a read-heavy client a larger cache: ```bash theme={null} archil mount /mnt/archil --region aws-us-east-1 \ --max-cache-mb 16384 --target-cache-mb 12288 ``` ## Freshness and consistency Serving reads from the client cache is what lets a client avoid contacting the storage system on every operation, which also means a client can serve stale data after another client changes a file. By default: * Opening a file checks it with the storage system, so reads after `open` see every write that reached the disk before it. * Attributes that `stat` returns are cached for up to 30 seconds. * Directory listings, and the names learned from them, are cached for up to five minutes. Tune this per directory or for the whole mount with [`archil set-cache-expiry`](/reference/archil-cli#set-cache-expiry). For the full model, including when another client's write reaches the disk, see [Consistency](/details/consistency#between-archil-clients). ## Warming the client cache By default the client cache fills lazily as data is read, so the first read of any file on a client pays the fetch latency. When you know which files you'll need — loading a model, starting a database, or running a batch job over a known set of inputs — you can warm them ahead of time so the first read is already local. There are two ways to warm the client cache: * **[`archil prefetch ...`](/reference/archil-cli#prefetch)** — pre-fetch files or directories (recursively) into the cache of an already-mounted disk. The command queues the work and returns immediately; fetching continues in the background. * **[`archil mount --pre-fetch `](/reference/archil-cli#mount)** — warm a comma-separated set of paths automatically as soon as the disk is mounted. ```bash theme={null} # Warm a model directory on an already-mounted disk archil prefetch /mnt/archil models/llama-3 # Or warm it as part of the mount archil mount /mnt/archil --region aws-us-east-1 --pre-fetch models/llama-3 ``` > [!NOTE] > > Pre-fetched data shares the client cache with normal reads, so it counts against `--max-cache-mb` and is subject to the same eviction described above. The client also reads ahead automatically: when it detects sequential reads of a file, it fetches up to 4 MiB beyond them. Change the window with [`mount --read-ahead `](/reference/archil-cli#mount), or pass `0` to disable it. ## Warming the storage system cache with the API On a miss in the storage system cache, the first read of a file waits on the backing data source. To take that wait out of the first read when you know which files a job will need, call [Prefetch File Ranges](/api-reference/disks/prefetch-disk) before the job starts. It warms the files into the storage system cache, so later reads from any client, [serverless exec](/compute/serverless-execution), or [sandbox](/compute/sandboxes/introduction) are served from the cache. It needs no mounted client, so you can call it from a scheduler or control service before the work starts. Each range names one file by its path relative to the disk root, with an optional `offset` and `length` in bytes. A range without them warms the whole file. A request can name up to 1,000 ranges. ```bash theme={null} curl --request POST \ 'https://control.green.us-east-1.aws.prod.archil.com/api/disks/dsk-abc123/prefetch' \ --header "Authorization: Bearer $ARCHIL_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "ranges": [ { "path": "models/llama-3/model.safetensors" }, { "path": "data/2026/events.parquet", "offset": 0, "length": 1048576 } ] }' ``` The response has one result per range, in request order: ```json theme={null} { "success": true, "data": { "results": [ { "found": true, "bytesRequested": 16060522496, "bytesWarmed": 16060522496 }, { "found": true, "bytesRequested": 1048576, "bytesWarmed": 0 } ] } } ``` * `bytesRequested` is the size of the range, and `bytesWarmed` is the part being fetched from the data source; the rest was already cached. The response can return before the fetch finishes. * A path that does not exist reports `found: false`. * Ranges must name files. A directory is found but warms nothing, so list its files and pass each one. * A range that could not be warmed carries `error`: `retry` for a transient failure worth retrying, or `failed`. Other ranges in the request are unaffected. If the disk is not mounted anywhere, the request loads it first. Warmed data is [active data](/concepts/metering#file-data) and is metered like data your clients read. ## Refreshing the client cache To force a client to drop cached entries and re-read from the storage system — for example, when another client has updated files on a [`--shared`](/concepts/sharing-disks) or [`--conditional`](/concepts/sharing-disks#conditional-mode) mount — use [`archil invalidate-cache`](/reference/archil-cli#invalidate-cache). Pass the mount root to refresh everything, or a single file path to refresh just that file. In-flight (dirty) writes are always preserved. ```bash theme={null} # Refresh the whole mount archil invalidate-cache /mnt/archil # Refresh a single file (siblings keep their cache) archil invalidate-cache /mnt/archil/shared-data/report.csv ``` ## Controls at a glance | Goal | Control | | - | - | | Set the maximum client cache size | [`mount --max-cache-mb`](/reference/archil-cli#mount) | | Set the target client cache size | [`mount --target-cache-mb`](/reference/archil-cli#mount) | | Warm files into the client cache after mount | [`prefetch`](/reference/archil-cli#prefetch) | | Warm files into the client cache at mount time | [`mount --pre-fetch`](/reference/archil-cli#mount) | | Warm files into the storage system cache without a client | [Prefetch File Ranges API](/api-reference/disks/prefetch-disk) | | Tune sequential read-ahead | [`mount --read-ahead`](/reference/archil-cli#mount) | | Drop and re-read cached data | [`invalidate-cache`](/reference/archil-cli#invalidate-cache) | | Tune directory-listing cache lifetime | [`set-cache-expiry`](/reference/archil-cli#set-cache-expiry) | # Solution Comparison Source: https://docs.archil.com/details/comparison How Archil compares with EBS, EFS, and S3 FUSE adapters. Archil ran git, npm, and SQLite transaction workloads about 5x to 190x faster than Amazon EFS. When designing a new application for the cloud, picking a storage layer shouldn't be the hard part. Archil is designed to be the simplest storage solution that delivers good performance for the workloads people actually run in the cloud: applications, databases, model training, data pipelines, and the long tail of stateful systems that need a real file system underneath them. You don't need to capacity-plan, choose between block storage and file storage, or run a replication or backup tier. Disks are mountable from any server, disks synced to your bucket keep data in S3 in its native object format, and performance is good enough that you can stop thinking about your storage layer and focus on your application. For most workloads, Archil is the right default. In [our benchmark](#benchmark-against-amazon-efs), Archil ran git, npm, and SQLite transaction workloads about 5x to 190x faster than Amazon EFS. Unlike a block volume, the same disk can be shared across machines, read through S3, and forked. If you stop using Archil, files on a disk synced to your bucket remain there as ordinary S3 objects, readable without Archil. Managed disks export over the S3 API. See [Data ownership and portability](/reference/data-ownership). ## When you might want something else Archil is a network file system, so the cache-miss path always includes a network round-trip. A few cases where a different tool is a better fit: * **You need microsecond latency on every operation regardless of working set size.** Use local NVMe. * **You already operate a Lustre cluster with HPC tooling and storage admins.** Stay on Lustre. * **Your application speaks S3 natively and doesn't need POSIX or shared access across servers.** Just use S3. ## How Archil compares to other solutions The sections below explain how Archil differs from each of the common alternatives in more detail. ### Block storage (e.g. EBS or Hyperdisk) Block volumes such as EBS or Hyperdisk attach to one server at a time, are provisioned and billed by size in advance, and don't synchronize with object storage, so each server keeps its own copy of any data it pulls from S3. An Archil disk is a shared, persistent file system that keeps data in S3 in its native format, and it gives you capabilities a block volume doesn't: * **[Sharing across servers](/concepts/sharing-disks)**: mount the same disk from many servers at once. * **An [S3 API](/protocols/s3-api)**: read and write the same files over S3 from anywhere. * **[Serverless execution](/compute/serverless-execution)**: run commands against the disk without starting a server. * **Elastic capacity**: the disk grows and shrinks with your data, so there's nothing to provision, resize, or pay for while it sits empty. * **[Branches and checkpoints](/concepts/branches-and-checkpoints)**: fork a disk or roll it back to an earlier state. A local block volume is still faster for workloads dominated by small synchronous writes, such as a database that fsyncs every transaction. Archil is faster for cold reads and large sequential I/O. See [Performance tips](/details/performance-tips) to get the most out of your disk. A few features a raw block device provides aren't available on Archil yet, such as [file ACLs and `renameat2` flags](/details/support). ### Native FUSE adapters (e.g. S3FS and Mountpoint) Existing FUSE adapters, such as S3FS and Mountpoint for Amazon S3, are not designed to support full POSIX semantics. For example, Mountpoint for Amazon S3 does not support file locks, random writes, renames, or symbolic links. In order to use these tools, you need to instrument your file applications to understand whether or not they use these unsupported operations, which can be difficult and error-prone. In addition, these tools do not provide shared caching for reference data sets to multiple clients, so large clusters may need to re-download the same data from S3 repeatedly. In comparison, Archil is designed to support all standard POSIX file operations, which means that it's out-of-the-box compatible with all file applications. ### Block FUSE adapters (e.g. JuiceFS and ObjectiveFS) Other disk adapters for S3, such as JuiceFS, ObjectiveFS, or ZeroFS, require that you store data in your S3 bucket in a proprietary block format. This means that you cannot use your existing data sets in S3 with these tools, and these buckets aren't compatible with any existing S3 workflows that you use. Archil stores data in S3 in its native format, so you can simultaneously use and share the data from both a disk and S3. You can also use Archil to connect to existing data sets in S3. For JuiceFS on an agent workspace, see the [agent workspace benchmark](#agent-workspace-benchmark). ### NFS file storage (e.g. Amazon EFS and Amazon S3 Files) NFS-based file systems, such as Amazon EFS and the newer Amazon S3 Files, are serverless and can be mounted from many servers at once, but the NFS protocol introduces two fundamental limitations that make them unsuitable for many workloads. The first limitation is metadata latency. NFS clients cannot cache the result of mutating metadata operations, so every file creation, rename, or attribute change must round-trip to the server before it can be acknowledged. As a result, metadata-heavy workloads such as source trees, package installs, container image extracts, and most interactive applications are typically an order of magnitude slower on NFS than on a local disk. Archil's custom file protocol allows clients to take exclusive write access to portions of the file system, so creates and writes can be acknowledged locally without a server round-trip. The second limitation is per-client throughput. NFS clients resolve DNS once at mount time and send every subsequent request to a single IP address, which caps a single client at roughly 3 GiB/s regardless of how much network bandwidth the underlying instance has. This is well below the NIC capacity of large GPU instances, making NFS a bottleneck for model training and other high-throughput workloads. Archil's protocol allows a single client to talk directly to multiple storage servers in parallel, similar to how parallel file systems like Lustre achieve scale-out throughput, so a single client can saturate the NIC of even the largest GPU instances. Archil also stores data in S3 in its native object format, so the same data is accessible through both the file system and the underlying S3 bucket without any proprietary on-disk layout. #### Benchmark against Amazon EFS Each result is the median of 3 runs from the same host. See the [methodology](#benchmark-methodology) below. | Test | Archil | Amazon EFS | | - | - | - | | git clone, express (214 files) | 0.21 s | 3.9 s | | git clone, react (7,260 files, 1.1 GB repository) | 9.8 s | 81.5 s | | `git status`, react | 0.55 s | 2.6 s | | git checkout of an old tag, react (8,158 files change) | 1.06 s | 46.9 s | | `rm -rf` of the react checkout | 0.14 s | 6.5 s | | `npm ci`, create-next-app (598 packages, \~20k files) | 7.6 s | 86.3 s | | `rm -rf node_modules` | 1.5 s | 83.3 s | | SQLite: 10k single-row transactions, `synchronous=FULL` | 23.9 s | 523 s | | SQLite: 10k single-row transactions, `synchronous=NORMAL` | 2.3 s | 443 s | | SQLite: bulk insert of 1M rows in one transaction | 11.0 s | 1.7 s | | SQLite: 100k random point reads | 7.0 s | over 10 min (stopped) | | Sequential write, 1 MiB blocks, 4 GiB + fsync | 194 MB/s | 590 MB/s | | Sequential read, cold | 1,197 MB/s | 1,020 MB/s | | 4 KiB random read, cold | 4,086 IOPS | 1,298 IOPS | | 4 KiB random write, 30 s + fsync | 953 IOPS | 2,976 IOPS | Amazon EFS was faster for the single large SQLite insert, sequential writes, and 4 KiB random writes. For Amazon S3 Files on an agent workspace, see the [agent workspace benchmark](#agent-workspace-benchmark). To tune Archil for your own workload, see [Performance tips](/details/performance-tips) or email [support@archil.com](mailto:support@archil.com). #### Benchmark methodology The host, versions, and commands for this benchmark are in [File systems on one host](/reference/benchmarks/file-systems#archil-vs-amazon-efs), so you can rerun it from your own host. For longer comparisons, see [Archil vs Amazon S3 Files for agent workspaces](https://archil.com/article/archil-vs-amazon-s3-files-for-agent-workspaces) and [Archil vs Amazon EFS for AI agents](https://archil.com/article/archil-vs-amazon-efs-for-ai-agents). ### Agent workspace benchmark This benchmark compares Archil with Amazon S3 Files and JuiceFS on a model of one heavy user's agent workspace: 205,004 small files (median 10 KB, all under 64 KB), directories of 30,000 and 15,000 entries, and four 250 MiB files, 4.3 GB in total. Every file system was mounted on the same host. 16 threads stand in for concurrent agents. **Cold** means a new client after unmounting, clearing the client's local cache and the page cache, and mounting again, as after a VM restore. **Warm** repeats the same operation right away. See the [methodology](/reference/benchmarks/file-systems#agent-workspace-archil-vs-amazon-s3-files-vs-juicefs). JuiceFS kept data in Amazon S3 and metadata in one of four places: Amazon ElastiCache for Redis, the same Redis volume mounted with `--writeback`, Amazon RDS for PostgreSQL, or a SQLite file on the client's local disk. **Reads**, p50 / p99 in milliseconds unless noted: | Test | Archil | Amazon S3 Files | JuiceFS + Redis | | - | - | - | - | | Small-file read, cold, 16 threads | 7.5 / 15 | 9.3 / 16 | 37 / 121 | | Small-file read, warm, 16 threads | 0.54 / 1.7 | 2.9 / 6.9 | 1.0 / 2.2 | | Small-file read, cold, 1 thread | 8.3 / 14 | 7.7 / 13 | 36 / 165 | | Small-file read, warm, 1 thread | 0.08 / 0.12 | 1.2 / 1.4 | 0.29 / 0.48 | | Read files written before a restore, cold | 1.8 / 3.0 | 6.2 / 11 | 27 / 85 | | `stat`, cold | 3.6 / 18 | 5.1 / 13 | 0.49 / 5.2 | | `stat`, warm | 0.24 / 0.57 | 0.26 / 5.2 | 0.25 / 1.1 | | 250 MiB sequential read, cold (MB/s) | 394 | 144 | 158 | **Scans**, cold / warm in seconds: | Test | Archil | Amazon S3 Files | JuiceFS + Redis | | - | - | - | - | | `grep -rl` over 4,000 files | 20 / 0.24 | 20 / 4.8 | 187 / 0.65 | | List a 30,000-entry directory | 1.77 / 0.16 | 2.1 / 0.007 | 0.08 / 0.07 | | `ls -l` of a 30,000-entry directory | 1.8 / 0.21 | 2.4 / 0.07 | 0.19 / 0.15 | | `find` over the whole workspace | 22 / 22 | 35 / 6.6 | 2.1 / 2.0 | **Writes**, p50 / p99 in milliseconds unless noted: | Test | Archil | Amazon S3 Files | JuiceFS + Redis | | - | - | - | - | | Create, write and close a small file, 16 threads | 0.9 / 28 | 11 / 25 | 31 / 72 | | Then make those 4,800 files durable (s) | 0.04 | 0.01 | 0.01 | | Office save (temp file, fsync, rename), 1 thread | 8.2 / 15 | 30 / 50 | 41 / 131 | | Office save, 8 threads | 60 / 82 | 82 / 100 | 90 / 186 | | Scratch file: create, write 16 KiB, read, delete, 16 threads | 2.9 / 3.7 | 156 / 192 | 174 / 273 | | 250 MiB sequential write including fsync (MB/s) | 501 | 410 | 764 | > [accordion] JuiceFS with Redis writeback, Postgres, and SQLite > > **Reads**, p50 / p99 in milliseconds unless noted: > > | Test | JuiceFS + Redis, writeback | JuiceFS + Postgres | JuiceFS + SQLite | > | - | - | - | - | > | Small-file read, cold, 16 threads | 38 / 131 | 42 / 129 | 30 / 134 | > | Small-file read, warm, 16 threads | 1.0 / 2.1 | 1.6 / 3.7 | 1.3 / 7.1 | > | Small-file read, cold, 1 thread | 42 / 148 | 43 / 138 | 35 / 113 | > | Small-file read, warm, 1 thread | 0.30 / 0.64 | 0.66 / 1.2 | 0.41 / 0.72 | > | Read files written before a restore, cold | 27 / 91 | 28 / 163 | 26 / 99 | > | `stat`, cold | 0.49 / 1.3 | 1.1 / 18 | 1.4 / 6.7 | > | `stat`, warm | 0.25 / 5.2 | 0.03 / 0.48 | 0.26 / 0.93 | > | 250 MiB sequential read, cold (MB/s) | 148 | 160 | 118 | > > **Scans**, cold / warm in seconds: > > | Test | JuiceFS + Redis, writeback | JuiceFS + Postgres | JuiceFS + SQLite | > | - | - | - | - | > | `grep -rl` over 4,000 files | 176 / 0.66 | 197 / 1.6 | 176 / 0.87 | > | List a 30,000-entry directory | 0.08 / 0.07 | 0.63 / 0.60 | 1.16 / 0.74 | > | `ls -l` of a 30,000-entry directory | 0.19 / 0.15 | 0.75 / 0.69 | 1.28 / 0.83 | > | `find` over the whole workspace | 2.1 / 1.9 | 10 / 10 | 8.0 / 7.5 | > > **Writes**, p50 / p99 in milliseconds unless noted: > > | Test | JuiceFS + Redis, writeback | JuiceFS + Postgres | JuiceFS + SQLite | > | - | - | - | - | > | Create, write and close a small file, 16 threads | 2.3 / 11 | 37 / 89 | 33 / 97 | > | Then make those 4,800 files durable (s) | 62.8 | 0.01 | 0.01 | > | Office save (temp file, fsync, rename), 1 thread | 2.2 / 4.5 \* | 58 / 136 | 46 / 123 | > | Office save, 8 threads | 78 / 136 \* | 156 / 237 | 47 / 154 | > | Scratch file: create, write 16 KiB, read, delete, 16 threads | 152 / 262 | 285 / 372 | 38 / 80 | > | 250 MiB sequential write including fsync (MB/s) | 1,634 \* | 700 | 662 | \* With `--writeback`, JuiceFS returns from fsync once data is on the client's local disk and uploads it to S3 later, so a write it has acknowledged can still be lost if the client fails. Clearing the backlog from the 4,800-file burst took 62.8 s. With SQLite, JuiceFS metadata is durable only on the client's local disk. Other file systems were faster in these tests: * **Every JuiceFS configuration:** cold `stat`, cold directory listings, `find`, and the 250 MiB sequential write. * **JuiceFS + Redis:** warm directory listings as well. * **JuiceFS + Postgres:** warm `stat`. * **JuiceFS + SQLite:** a faster p50 for eight concurrent office saves. * **Amazon S3 Files:** warm directory listings, warm `find`, and a slightly faster p50 for single-threaded cold reads. ### Sandbox platforms (e.g. E2B, Daytona, and Vercel Sandbox) Archil is a sandbox platform too. [Archil sandboxes](/compute/sandboxes/introduction) are Firecracker microVMs with pause and resume, fork, ports, and an egress firewall, and their root filesystem is an Archil disk. ### Sandbox provider volumes (e.g. Daytona Volumes) Volumes from a sandbox provider keep files beyond one sandbox, but only within that provider. An Archil disk mounts in Archil sandboxes and over FUSE in sandboxes from any provider that supports it, including [Daytona](/guides/platforms/daytona), [E2B](/guides/platforms/e2b), and [Modal VM sandboxes](/guides/platforms/modal), and your app can read and write it through the [S3 API](/protocols/s3-api) or the SDKs without starting a sandbox. ### Shared file storage (e.g. Alluxio and FSx for Lustre) Some solutions, like Alluxio or FSx for Lustre, require that you either deploy and manage your own infrastructure or that you provision capacity. It's often difficult to predict the specific storage needs of an application, especially if the application can have unexpected or bursty needs. Archil is a fully managed, serverless solution that automatically scales with your application. You don't need to provision capacity or deploy infrastructure. Metered usage counts toward your paid plan's [monthly minimum](/administration/billing#how-the-monthly-minimum-works). *** See the [pricing page](https://archil.com/pricing) for plans and usage rates. # Consistency Source: https://docs.archil.com/details/consistency When other clients and the backing bucket see a write, when a write is durable, and what a crashed client leaves behind Consistency in Archil has two separate boundaries: * **Between Archil clients.** Mounted clients, [serverless execution](/compute/serverless-execution), [sandboxes](/compute/sandboxes/introduction), and the [Archil S3 API](/protocols/s3-api) all read and write one strongly consistent copy of the disk. A mounted client also buffers writes and caches reads, and those two things are the only reasons another client can see an older version of a file. * **Between the disk and its backing bucket.** Archil copies changes to the bucket in the background, and picks up objects written directly to the bucket in the background. Tools that read the bucket directly see changes later than Archil clients do. ![Consistency Model](https://mintcdn.com/regattastorage/ZxDPj9Yr7FlGAuri/images/consistency.svg?fit=max\&auto=format\&n=ZxDPj9Yr7FlGAuri\&q=85\&s=8024a0db1c4872a0b9c728056bb51736) ## Between Archil clients ### The disk is strongly consistent Once a write reaches the disk, every read that reaches the disk returns it, from any client. Renames are atomic: another client's next read of the affected path returns either the old state or the new state, never a partial result. "Strongly consistent" describes the disk, not each client's cache. A mounted client keeps unsent writes in memory and serves reads from its cache when it can. The next two sections give the exact rules for both, so you can tell when another client sees a change. ### When a write reaches the disk The `archil` client buffers writes in memory and starts sending them to the disk in the background about once a second. Nothing waits for that background send. These calls return only after the writes have reached the disk and are [durable](/details/architecture#data-durability): | Call | Waits for | | - | - | | `fsync` or `fdatasync` | The file's writes | | [`archil sync `](/reference/archil-cli#sync) | Every write on the mount | | [`archil checkin `](/reference/archil-cli#checkin) | Writes under the path, then releases the [delegation](/concepts/sharing-disks#understanding-delegations) | | [`archil unmount`](/reference/archil-cli#unmount) | Every write on the mount, then releases every delegation the client holds | | `close` on a [conditional](/concepts/sharing-disks#conditional-mode) mount | The file's writes, unless the client holds a delegation that covers the file, such as a checkout of a directory above it | On exclusive and shared mounts, `close` returns without waiting. There, the file is covered by a delegation, and the client sends its writes at `fsync`, at check-in, at unmount, or in the background. On a shared mount, a file that `open` checked out automatically is checked back in after its last `close`, once its writes are sent, but `close` itself doesn't wait for that. In conditional mode, creating, renaming, and deleting files and directories happen on the disk before the call returns. ### When another client sees it A mounted client answers from its cache when it can. How far that cache can lag depends on what is being read: * **Opening a file** checks the file with the disk. If the file has changed since the client cached it, the client drops the cached contents, so reads after `open` return every write that reached the disk before the `open`. A file that was already open when another client changed it can keep returning cached data; close it and open it again. (Client v0.8.29 and later.) * **File attributes** that `stat` returns, such as size, timestamps, and permissions, are cached for up to 30 seconds. * **Directory listings**, and the names a client learns from them, are cached for up to five minutes by default. Shorten this for one directory or for the whole mount with [`archil set-cache-expiry`](/reference/archil-cli#set-cache-expiry), or set it to `0` to list from the disk every time. * [`archil invalidate-cache `](/reference/archil-cli#invalidate-cache) drops everything the client has cached, so the next read of anything goes to the disk. Writes the client hasn't sent yet are kept. So a reader sees another client's change once two things are true: the writer's call from the table above has returned, and the reader isn't answering from a cache entry that predates it. A client never sees stale data under a path it has checked out, because no other client can write there while it holds the delegation. That is why an exclusive mount, which holds the delegation for the whole disk, behaves like a local disk. See [Client caching](/details/caching) for how the cache is sized and warmed. ### Concurrent writers * **Exclusive and shared mounts** allow one writer per path. A client that writes under a path another client has checked out gets `EROFS: Read-only file system`. * **Conditional mounts** let any number of clients write any path, including the same file. If two clients write the same part of a file, the last write to reach the disk wins. Use `archil checkout` to give one client exclusive access to a path. For a pattern for several agents sharing a disk, see [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents). ## Between the disk and its backing bucket ### From the disk to the bucket Archil copies a changed file to the [backing bucket](/concepts/data-sources) once the file has gone 15 seconds without a write, so it usually appears in the bucket tens of seconds after its last write. A file that keeps changing is still copied: Archil starts copying it no later than 10 minutes after its first write that hasn't reached the bucket. A large backlog of changes takes longer. You can't force a copy or check its progress. If an S3 client needs to see writes as soon as they reach the disk, point it at the [Archil S3 API](/protocols/s3-api) instead of the bucket. The S3 API reads the disk directly, so it is strongly consistent with the file system. ### From the bucket to the disk On a disk that uses your own bucket, objects written directly to the bucket appear on the disk after Archil lists that directory in the bucket again. When a directory is read and its listing is more than five minutes old, Archil lists it again in the background, so a new object usually appears within a few minutes in a directory that is in use. A directory nobody has read for an hour is listed again when it is next read. Mounted clients then apply the caching rules above. If you write the same file from both sides, see [Concurrent writes](/concepts/data-sources#concurrent-writes). ### Durability before the copy The copy to the bucket affects only when the bucket shows a change. A write that has reached the disk is already durable while it waits to be copied. See [Data durability](/details/architecture#data-durability). ## Crashes and forced ownership * **Writes the client hasn't sent are lost** if its process or machine stops, or its sandbox is killed, before they reach the disk. Every write that one of the calls in the table above returned for is durable. Unmount before you stop a machine; see [Troubleshooting mounts on other platforms](/guides/platforms/troubleshooting) for sandboxes. * **Delegations stay in place.** Archil doesn't release the paths a crashed client checked out, or the whole-disk delegation of an exclusive mount, on its own. Other clients can't write there, and an exclusive mount of the disk fails, until you revoke them. Once the client has disconnected, the SDK lists its delegations as orphaned, and you can [release them](/guides/platforms/troubleshooting#release-a-stale-lock). Mounting again on the same machine doesn't help, because every mount is a new client. A conditional mount that hasn't checked out a path holds no delegation, so it leaves nothing behind. * **Forcing ownership loses the holder's unsent writes.** `archil checkout --force` and `archil mount --force` revoke other clients' delegations immediately. Writes those clients hadn't sent to the revoked paths are lost, and their later `fsync` calls return `EIO`. See [Forcing ownership](/concepts/sharing-disks#forcing-ownership). # Performance Source: https://docs.archil.com/details/performance Measured Archil latency and throughput, and how to reproduce every number yourself with a public script. Every latency number on this page was measured with a script you can run on your own account. Nothing here is a design target. See [How we measure](#how-we-measure) and [the latency benchmark](/reference/benchmarks/latency). ## Measured latency Measured on October 27, 2026 from a c7gn.2xlarge in AWS `us-east-1`, against a disk in the same region. | Operation | p50 | p95 | | - | - | - | | Mount a disk | 50 ms | 58 ms | | 4 KiB read, client cache miss | 0.50 ms | 0.75 ms | | 4 KiB write, `O_SYNC` | 2.0 ms | 8.2 ms | | Create a disk | 9.3 ms | 11 ms | | Checkpoint a disk | 13 ms | 19 ms | | Branch from a checkpoint | 33 ms | 36 ms | A client cache miss is served from the file server's [cache](/details/caching). Data that has to come from the data source takes longer, typically 10 to 30 ms for Amazon S3. An `O_SYNC` write returns once the file server has made it durable. Checkpoint and branch times don't depend on disk size. ## Throughput | | Limit | Highest seen on one disk in production | | - | - | - | | Write throughput per disk | 1 GB/s | 620 MB/s | | Read throughput per disk | Unlimited | 5.9 GB/s | The production figures are the highest one-minute averages recorded in production. They are observations, not caps. See [Limits](/reference/limits) for every other limit. ## Compared with Amazon EFS In a [side-by-side benchmark](/details/comparison#benchmark-against-amazon-efs) on one host, Archil ran git, npm, and SQLite transaction workloads about 5x to 190x faster than Amazon EFS. EFS was faster for sequential writes, 4 KiB random writes, and one large SQLite insert. The page lists every result, including the ones Archil lost, and the method behind it. ## Caching If your application requests data that's not in the cache, Archil fetches it from the backing bucket and caches it for future requests. Archil keeps this data in cache for up to 1 hour after the last read or write. Archil may read up to 10 MiB more than requested from the backing bucket. ## How we measure * **Public interfaces only.** The tests use the public API, the released `archil` CLI, and standard tools such as `fio`, `curl`, `git`, and `npm`. No internal access or special builds. * **Same region.** The client runs in the same cloud region as the disk. A cross-region round trip is larger than most of these figures, so run your test where your workload runs. * **Percentiles, not best runs.** Latency rows report p50 and p95 over every sample. Benchmark rows report the median of 3 runs. * **Losses stay in.** When another system wins a test, the result stays in the table. * **Dated and versioned.** Each result lists the date, host, and tool versions, so you can tell when a number is stale. To check our availability yourself, see the public incident and uptime history at [status.archil.com](https://status.archil.com). If your own results differ from ours, email [support@archil.com](mailto:support@archil.com) with your script output. # Performance tips Source: https://docs.archil.com/details/performance-tips Practical ways to get the most out of an Archil disk. ## SQLite * **Group reads into one transaction.** In autocommit mode, every query makes two file-lock calls and a `stat` of the database file. Wrap a group of reads in one transaction, or set `PRAGMA locking_mode=EXCLUSIVE` when a single process owns the database. Inside one transaction, point reads on Archil took 3 to 4 µs each, the same as on a local disk. * **Batch writes into transactions.** Inserting 1M rows in one transaction took 11 s, while 10,000 single-row transactions took 24 s with `synchronous=FULL`. > [!NOTE] > > `synchronous=NORMAL` ran about 10x faster than `FULL` on Archil. In WAL mode, `NORMAL` is > [SQLite's own recommendation](https://www.sqlite.org/pragma.html#pragma_synchronous) for most applications: the database > stays consistent, but the last transactions committed before a machine crash can roll back. Keep `FULL` if every committed > transaction must survive a crash. ## Prefer large sequential I/O Large sequential I/O is a strength on Archil: cold sequential reads ran at about 1.2 GB/s from a single client. Where you can, prefer fewer, larger files and large read and write sizes. ## Dependency installs Installing dependencies directly on a disk works well: `npm ci` installed 598 packages in 7.6 s. Like a local file system, recently written files are staged on the client until an `fsync` or a background flush. When the installed files must be durable before you continue, run [`archil sync`](/reference/archil-cli#sync) on the mount point. See [Ensuring that data is persisted on shutdown](/mounting/linux#ensuring-that-data-is-persisted-on-shutdown). ## Need help with your workload? The Archil team works directly with application teams to tune performance for specific workloads. Email [support@archil.com](mailto:support@archil.com) and we'll help you get the most out of your disk. *** For the full benchmark against Amazon EFS, see [Solution Comparison](/details/comparison#benchmark-against-amazon-efs). # Security & Compliance Source: https://docs.archil.com/details/security How Archil protects your data, and what we commit to contractually Archil supports managed storage and bring your own bucket (BYOB). This page describes where your data lives, how we protect it, and what we're prepared to sign. ## Data ownership * **Bring your own bucket (BYOB):** Your data is stored in a bucket you own. Archil reads from and synchronizes writes to that [data source](/concepts/data-sources) on your behalf. * **Managed storage:** Your data is stored in a bucket managed by Archil. You do not need to configure or operate a bucket. In both modes, active data is temporarily held in Archil's storage layer. Newly committed writes are durable there while they synchronize to object storage. This temporary storage layer is part of the data path even when you bring your own bucket. With BYOB, files are synchronized to your bucket in their native object format, readable with any S3 tool and without Archil. Data already synchronized to your bucket remains there when you stop using Archil. With managed storage, export any data you want to keep before deleting the disk. For how to export data and what you can read without Archil, see [Data ownership and portability](/reference/data-ownership). ## Hosting Archil offers public regions on AWS and Google Cloud. Additional regions are available in preview. [Private deployments](#private-deployment) run on AWS, Google Cloud, Azure, or bare metal. See [Region availability](/reference/regions). Active disk data stays in the cloud and region selected for the disk. A BYOB data source stores the synchronized copy in the bucket and region you configure. Account and billing metadata are described separately in [Subprocessors](/legal/subprocessors). Archil's storage layer is designed for **99.999% (five nines) durability**, achieved through replication across multiple Availability Zones and synchronization to object storage. Archil is designed for **99.99% availability**. We can sign an availability SLA for [Enterprise customers](/administration/billing); [contact us](mailto:sales@archil.com) to discuss it. ## Encryption All data is encrypted at rest and in transit. * **At rest** — Data in Archil's storage layer is encrypted with AES-256. In BYOB mode, data synchronized to your bucket uses that bucket's encryption settings. * **In transit** — Client-to-service connections use TLS 1.2 or higher. Unencrypted connections are rejected. Encryption keys are managed by Archil's key management infrastructure and rotated on a regular schedule. ## SOC 2 Archil undergoes ongoing SOC 2 Type II audits covering security, availability, and confidentiality. Audit reports are available through the [Archil Trust Center](https://security.archil.com/). If you need a copy of our most recent report for your procurement or compliance review, [contact us](mailto:security@archil.com). ## Data protection (GDPR & CCPA) Archil is GDPR and CCPA compliant. We provide **Data Processing Agreements (DPAs)** to [Enterprise plan](/administration/billing) customers. The DPA covers data residency, [subprocessors](/legal/subprocessors), and breach notification commitments. Data residency is respected by default — Archil does not move data across regions without explicit configuration. To request a DPA, [contact us](mailto:security@archil.com). ## HIPAA Archil supports workloads subject to HIPAA requirements. We sign **Business Associate Agreements (BAAs)** with [Enterprise plan](/administration/billing) customers. To request a BAA, [contact us](mailto:security@archil.com). ## Access control Archil provides layered access control to match your environment: * **AWS IAM roles** — Clients authenticate with their existing IAM roles. No additional credentials needed for EC2, Lambda, or ECS. See [Disk Users](/concepts/disk-users). * **Disk tokens** — For non-AWS or cross-region access, a [disk token](/concepts/authorization#token-based-disk-authorization) grants a client mount access to one specific disk. Disk tokens are hashed before storage — Archil never stores them in plaintext. * **API keys** — The [Control Plane API](/api-reference/introduction) uses account-scoped API keys for managing disks, disk tokens, and users programmatically. API keys are separate from disk tokens. * **POSIX permissions** — Standard file permissions (owner, group, other) are enforced on all mounted disks. ## Security testing Archil partners with [Casco Security](https://casco.com/) for monthly penetration testing across our infrastructure. We also run continuous static analysis on every code change. ## Vulnerability disclosure If you discover a security vulnerability in Archil, please report it to [security@archil.com](mailto:security@archil.com). We target acknowledgment within 24 hours. We offer bounties for qualifying vulnerability reports — email us for details. ## Private deployment [Bring Your Own Cloud (BYOC)](/reference/byoc) is available to Enterprise customers. BYOC runs on any cloud, including AWS, Google Cloud, and Azure, as well as on your own bare-metal servers, and can include disks, [persistent sandboxes](/compute/sandboxes/introduction), and [serverless execution](/compute/serverless-execution). Archil fully manages each deployment over an egress-only connection. Telemetry never includes PII or PHI: it is scrubbed inside your cluster before it leaves, file data is never logged, and file paths and names are blocked. Both the file system and storage layers run in your environment, so your disk data never leaves it. See [BYOC](/reference/byoc) for deployment, maintenance, and telemetry details. ## Private networking For organizations requiring private connectivity (e.g., AWS PrivateLink), [contact us](mailto:support@archil.com) to discuss options. # Application Compatibility Source: https://docs.archil.com/details/support Archil is designed for full POSIX-compatibility Archil's FUSE-based file system is designed to be fully compatible with *all* applications which run on a local, POSIX file system. If you discover behavior that you believe deviates from POSIX, please contact us by emailing [support@archil.com](mailto:support@archil.com), and we will address the underlying issue. We are continuously testing our adherence to POSIX, using the same test suite as the XFS file systems. Persistent sandbox root volumes use ext4 and provide standard Linux/POSIX filesystem semantics, including file ACLs, extended attributes, file locks, and atomic renames. They also support Linux `renameat2` flags, OverlayFS, and Docker `overlay2`. The compatibility details and limitations below apply to FUSE-mounted Archil disks, including additional disks attached to a sandbox, and do not apply to its ext4 root volume. See [Linux and container compatibility](/compute/sandboxes/introduction#linux-and-container-compatibility) for sandbox capabilities. ## POSIX compatibility details Archil disks are designed to be broadly compatible with POSIX semantics, even when the underlying data sources are not POSIX compatible, like Amazon S3. When needed, the [Archil storage service](/details/architecture) emulates file operations on data sources. Today, Archil supports the following POSIX features: * All read and write operations, including random writes and appends * Symbolic links (`ln -s`) * Hard Links (`ln`) * Atomic renames, including directory renames (`mv`)[1](#note-rename) * POSIX locks, including `flock` and `fcntl`[2](#note-locks) * Anonymous files (unlink after opening) * File modes * File ownership * Extended attributes (with the `--enable-xattrs` flag) **1.** A rename is atomic — a reader on another client sees either the old name or the new name, never a partial result. Archil's storage system is fully consistent; a client's cached view can lag briefly and refreshes with [`archil invalidate-cache`](/reference/archil-cli#invalidate-cache).\ **2.** `flock` and `fcntl` locks behave exactly as they do on a local disk for every process on a client, so a disk mounted by a single server or sandbox behaves like a local disk. In exclusive and shared mode, only one client can write to a file at a time, and you coordinate writers across clients with the [checkout/checkin ownership model](/concepts/sharing-disks). In [conditional mode](/concepts/sharing-disks#conditional-mode), several clients can write to the same file, so coordinate them in your application. **We are actively working on adding support for the following features:** * File ACLs (`system.posix_acl_access`) * `renameat2` flags (`RENAME_NOREPLACE`, `RENAME_EXCHANGE`, `RENAME_WHITEOUT`). A rename that passes any of these flags fails with `EINVAL`, so an Archil disk can't be used as an overlayfs upper directory today. ## Application support As a result of our broad compatibility with POSIX, Archil is compatible with all applications which run on a local, POSIX file system. This includes: * **Version control systems**, such as Git (including large repositories, `git clone`, `git checkout`, and `git status`), Mercurial, and Subversion * **Databases**, such as PostgreSQL, MySQL, and SQLite * **Build tools and compilers**, such as Make, CMake, GCC, and Clang * **Archive and compression tools**, such as `tar`, `gzip`, `zip`, and `rsync` * **Editors and IDEs**, such as Vim, Emacs, and VS Code * **Containers** — [mount in a container](/mounting/containers) via a bind mount from the host or FUSE inside the container ## Limits * **Path length** — on a disk synchronized to a data source, a file's full path can't exceed the data source's object-key length limit (1,024 bytes for Amazon S3, including the disk's bucket prefix). Longer paths fail with `ENAMETOOLONG`. * **File names** — on a disk synchronized to a data source, file and directory names must be valid UTF-8. # Introduction Source: https://docs.archil.com/getting-started/introduction Archil provides the infrastructure that makes it simple to connect real-world AI applications to their data. Out of the box, Archil provides everything that you need to build production-scale stateful agents or agent platforms for your organization. You can use Archil to: * **Run untrusted AI-generated code in sandboxes.** Each [persistent sandbox](/compute/sandboxes/introduction) is a Firecracker microVM with its own Linux kernel, a persistent root disk, and [controlled egress](/compute/sandboxes/firewall). For one-off commands against a disk, use [serverless execution](/compute/serverless-execution). * **Build and give agents access to petabyte-scale context**, using our [infinite, shared disks](/concepts/sharing-disks). Disks use Archil-managed storage by default, and you can optionally synchronize them with your own bucket in sources like [Amazon S3 object storage](/data-sources/s3-object-storage). * **Share artifacts between agents and humans** with a strongly consistent [REST API](/protocols/s3-api) that allows you to easily embed file watching and uploading into any website. ```typescript Vercel AI SDK theme={null} import { generateText } from "ai"; import * as archil from "disk"; import { createDiskTools } from "@archildata/ai-sdk"; const disk = await archil.getDisk('disk-1234'); const result = await generateText({ model: "anthropic/claude-sonnet-4.6", prompt: "What files do you have access to?", tools: createDiskTools(disk), }); ``` ```typescript Archil Typescript SDK theme={null} import * as archil from "disk"; const myDisk = await archil.getDisk('disk-1234'); // Easily hand off between bash and REST // await myDisk.exec('curl google.com > google.txt'); console.log(await myDisk.getObject('google.txt')) ``` If you're building agents, the easiest way to get started with Archil is with one of our integrations with popular agent frameworks like [AI SDK](/integrations/ai-sdk), [eve](/integrations/eve), [Mastra](https://www.npmjs.com/package/@archildata/mastra), or [LangChain](https://www.npmjs.com/package/@archildata/langchain). For more control, use our [TypeScript SDK](/sdks/typescript) or [Python SDK](/sdks/python) directly. ## Features Archil's storage primitives support a rich set data management features to make it simple to build complex, agentic workflows. * [Branches and checkpointing](/concepts/branches-and-checkpoints) make it possible to have simultaneous agents run on the same data set, and pick the best output. * [Native SQLite support](/compute/sqlite) allows you to easily create databases on your file system, and serverlessly read and write to them from anywhere. * [Immutable log support](/protocols/s3-api#appending) simplifies how you build log-like abstractions on your disk. Archil runs its own [sandboxes](/compute/sandboxes/introduction) on Firecracker microVMs. If your agents already run elsewhere, the same disks mount in other sandbox providers (such as [Daytona](/guides/platforms/daytona), [E2B](/guides/platforms/e2b), and [Superserve](https://docs.superserve.ai/storage/archil)), so workspaces stay portable, and can be mounted from [Kubernetes clusters](/reference/csi-driver) or provisioned with [Terraform](/reference/terraform). ## At a glance * **Storage:** disks use Archil-managed storage by default, so you don't need a bucket of your own. Bringing your own bucket is optional and available on paid plans. See [Data sources](/concepts/data-sources). * **Durability:** designed for 99.999% durability for newly written data, which is stored across multiple Availability Zones once `fsync` returns, and 99.99% availability. See [Data durability](/details/architecture#data-durability). * **Regions:** disks run in `aws-us-east-1`, `aws-us-west-2`, `aws-eu-west-1`, and `gcp-us-central1`. Sandboxes and serverless execution run in the AWS regions. [BYOC](/reference/byoc) deployments run everything on any cloud or bare metal. See [Region availability](/reference/regions). * **Your data, your exit:** disks synced to your own bucket store each file as an ordinary object that any S3 tool can read without Archil. Managed disks export with any S3 client over the [S3 API](/protocols/s3-api). See [Data ownership and portability](/reference/data-ownership). * **SDKs:** TypeScript ([`disk`](/sdks/typescript) on npm) and Python ([`archil`](/sdks/python) on PyPI), both covering disks, serverless execution, and sandboxes, plus adapters for agent frameworks. See [SDKs](/sdks/overview). * **Plans:** Free (no credit card), Developer ($10 monthly minimum), Team ($500 monthly minimum), and Enterprise (custom). ## Pricing Start on Free with included storage, serverless tool calls, and sandbox compute. No credit card is required. See [Pricing & Billing](/administration/billing) for rates, plan limits, and examples. # Quickstart Source: https://docs.archil.com/getting-started/quickstart This page will walk you through how to set up an Archil disk and build a real agent with the [AI SDK](https://ai-sdk.dev) in less than 5 minutes that uses real tools, like `bash`, against your data. You can follow along within the [Free plan limits](/administration/billing#plans-and-limits). For this tutorial, let's build an agent that writes a storybook for us, creates a nicely formatted PDF document of our book, and delivers an audit-log of every tool that it used to create the book -- **without the use of a separate sandbox**. This quickstart uses one-shot [serverless execution](/compute/serverless-execution): the agent's `bash` tool runs each command in a fresh microVM, for up to 5 minutes. If your agent needs a long-lived machine, use a [persistent sandbox](/compute/sandboxes/introduction). Its status, session and idle limits, pricing, and what persists on stop are summarized in [Limits and lifecycle at a glance](/compute/sandboxes/introduction#limits-and-lifecycle-at-a-glance). > [!NOTE] > > This tutorial does not walk through mounting the Archil file system. If you already have > a server you want to attach Archil to, see [Mount on Linux (FUSE)](/mounting/linux) (or [macOS](/mounting/macos) / [in a container](/mounting/containers)). **1. Create an API key** Sign in to [the Archil console](https://console.archil.com/api-keys), create an API key from the **API keys** page, and export it alongside a key for your model provider: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export ANTHROPIC_API_KEY=sk-ant-... ``` Create your API key in the region closest to where your users are, for this tutorial, you need to use one of the Archil AWS regions where sandboxes are offered (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`); see [Region availability](/reference/regions). **2. Install dependencies** In your project directory, install the required dependencies for this tutorial: ```bash theme={null} npm install ai @ai-sdk/anthropic disk @archildata/ai-sdk ``` > [!NOTE] > > The Archil TypeScript SDK is the `disk` package. Working in Python instead? Install the [Python SDK](/sdks/python) with `pip install archil`. **3. Create your agent's workspace** Archil provides the file system workspace for agents to store files, manipulate context, and run complex commands -- while providing concurrent access to that workspace from your web application for humans. You can create a file system for each agent, within your [plan's file system limit](/administration/billing#plans-and-limits). Make one for our tutorial agent: ```bash npx theme={null} npx disk create tutorial-agent-disk ``` ```typescript Typescript theme={null} import * as archil from "disk"; const { disk: agentDisk } = await archil.createDisk({ name: "tutorial-agent-disk" }); ``` The result includes a token that you can use to mount the disk directly. You do not need it for this tutorial. > [!NOTE] > > By default, the disk uses Archil-managed storage, so you don't need a bucket of your own. > If you have an existing S3 or GCS bucket you'd like the agent to work against, you can specify it as a [data source](/concepts/data-sources) > when creating your disk -- see [S3 Object Storage](/data-sources/s3-object-storage). When specifying a data source, your disk > will mirror the contents of your bucket. **4. Build the agent** Let's build our agent to now write us a short story. We use a [ToolLoopAgent](https://ai-sdk.dev/docs/reference/ai-sdk-core/tool-loop-agent) from the AI SDK to iteratively call tools until we accomplish our goal. We'll go ahead and give the agent some instructions to use Archil and connect our disk as a set of tools in [conditional mode](/concepts/sharing-disks). ```typescript theme={null} import { anthropic } from "@ai-sdk/anthropic"; import { ToolLoopAgent } from "ai"; import { createDiskTools } from "@archildata/ai-sdk"; // Mount the disk as a workspace, opting the agent into our new, // conditional mode semantics. const workspace = archil.workspace({ workspace: { disk: agentDisk, conditional: true }, }); const agent = new ToolLoopAgent({ model: anthropic("claude-sonnet-5"), instructions: [ "You are a writing assistant. Save your work as you go.", "Everything happens in /mnt/archil/workspace — use that path for bash and for the file tools.", "Each bash command runs in a fresh microVM, so anything you install belongs in the workspace if you want it next time.", ].join(" "), tools: createDiskTools(workspace), }); await agent.generate({ prompt: "Write a short story, format it nicely and save it as story.pdf", }); ``` That's the whole integration. `createDiskTools` automatically gives the agent a set of tools to manipulate data on the disk without a separate sandbox which includes: read/write/delete files, run bash, glob, and grep. You can view your story in the [Archil console](https://console.archil.com/), or by downloading the contents of the file: ```typescript theme={null} import { writeFile } from "node:fs/promises"; await writeFile("story.pdf", await workspace.getObject("workspace/story.pdf")); ``` **5. Record an audit log** Next, let's extend our agent to make it ready for production by building an audit log of each tool call that was made while it ran. We can easily build an append-only log on Archil by using the `AppendObject` function of our extended [S3 API](/protocols/s3-api#appending). We can use the [onToolExecutionEnd](https://ai-sdk.dev/docs/ai-sdk-core/lifecycle-callbacks#ontoolexecutionend) lifecycle hook in the AI SDK to record each tool call as it happens. Replace the `agent.generate` line in your code with the following: ```typescript theme={null} const auditPath = `.audit/${Date.now()}.jsonl`; await agent.generate({ prompt: "Write a short story, format it nicely and save it as story.pdf", onToolExecutionEnd: async ({ toolCall, toolOutput, toolExecutionMs }) => { const entry = { tool: toolCall.toolName, input: toolCall.input, ms: Math.round(toolExecutionMs), result: toolOutput.type === "tool-result" ? toolOutput.output : toolOutput.error, }; await agentDisk.appendObject(auditPath, `${JSON.stringify(entry)}\n`); }, }); ``` Now, you can view your audit log by reading the contents of the `auditPath` file. ```typescript theme={null} const audit = new TextDecoder().decode(await agentDisk.getObject(auditPath)); for (const line of audit.trim().split("\n")) { const { tool, ms, input } = JSON.parse(line); console.log(`${tool.padEnd(11)} ${String(ms).padStart(6)}ms ${JSON.stringify(input).slice(0, 60)}`); } ``` Which will output the audit log entries to the console that look like this: ``` run_bash 202ms {"command":"which pandoc wkhtmltopdf weasyprint pdflatex ens... run_bash 3571ms {"command":"pip install --quiet reportlab 2>&1 | tail -20; e... write_file 55ms {"path":"/mnt/archil/workspace/story.txt","content":"The Lig... run_bash 3241ms {"command":"cd /mnt/archil/workspace && pip install --quiet... ``` You now know the basics of how to build agents with Archil with persistent disks that can scale to any size, have multiple agents work in parallel, and provide strong-consistency for human clients who might interact using a web application via our [S3-compatible API](/protocols/s3-api). ## Next steps > [card] [AI SDK integration](/integrations/ai-sdk) > > Mount several disks as one workspace, and scope what the agent can reach. > [card] [Serverless execution](/compute/serverless-execution) > > The runtime behind `run_bash` — fan out across microVMs, compose disks, no lifecycle to manage. > [card] [Parallel agents](/guides/agents/parallel-agents) > > Fork a sandbox per agent, run them in parallel, and keep the best result. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Keep a sandbox warm across tool calls when the agent needs a long-lived machine. > [card] [Mount to a Linux server](/mounting/linux) > > Use the same disk as a POSIX filesystem from your own machines. > [card] [Pricing](https://archil.com/pricing) > > View Archil pricing and plan details. # Agentic observability Source: https://docs.archil.com/guides/agents/agentic-observability Write application logs straight to an Archil disk, then have an AI agent triage errors on a schedule and build dashboards on request. Dashboards and alerts only cover the questions someone thought to ask in advance. An agent can ask new ones, as long as it can read the logs. In this guide, your services write their logs as plain JSON files straight to an Archil disk, and an agent gets read-only access to them. You'll build two things on top: * **A triage job** that runs every few minutes. Deterministic SQL finds anomalies, a decision model that costs a fraction of a cent per run picks the ones that matter, and a frontier model investigates only those. Quiet runs don't call a model at all. * **Dashboards on request.** Ask a question in plain English and get back a link to a page of charts that answers it, plus a script that rebuilds the page without the model. There's no log pipeline to run: no collector, no ingest service, no index. Apart from Archil and the model, the tools are open source: | Job | Project | What it does here | | - | - | - | | Query | [DuckDB](https://duckdb.org) | Runs SQL directly over the log files, with no import step | | Chart | [Vega-Lite](https://vega.github.io/vega-lite/) | Renders dashboard charts to static SVG | | Agent loop | [AI SDK](https://ai-sdk.dev) | Runs the model with Archil's file, grep, and bash tools | ```mermaid theme={null} flowchart LR S[Your services] -->|one file per service, hour, and host| L[(app-logs disk)] L -->|read-only| D[Detect: SQL] D -->|anomalies| C[Classify: decision model] C -->|real problems| I[Investigate: agent] L -->|read-only| B[Dashboard agent] I -->|reports, notes| W[(obs-agent disk)] B -->|dashboards| W W -->|share link| Y[You or Slack] ``` This works well on Archil for a few reasons: * **Logs are ordinary files,** so the agent uses `grep` and SQL, which it already knows well, instead of a vendor query language. * **Queries run next to the data.** The agent's commands run in [serverless microVMs](/compute/serverless-execution) that mount the disk, so logs are never downloaded to where the agent runs. In our runs, a DuckDB query across every log file returned in 237 ms. * **The agent can't change the logs.** It mounts the log disk read-only and keeps its notes, reports, and dashboards on a second disk. * **Every instance writes to the same disk.** In [conditional mode](/concepts/sharing-disks), any number of hosts can write to one disk at the same time, and each line is readable by other clients as soon as it's written. ## Prerequisites Export your Archil API key, a region that offers sandboxes (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [AI Gateway](https://vercel.com/ai-gateway) key for the model. Then install the dependencies: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 export AI_GATEWAY_API_KEY=... npm init -y && npm pkg set type=module npm install disk @archildata/ai-sdk ai zod tsx ``` Save each file in this guide in your project directory, and run scripts with `npx tsx .ts`. ## Write logs to a disk You'll run a small demo shop in a [persistent sandbox](/compute/sandboxes/introduction) that mounts the log disk and writes to it directly. In production, you mount the disk wherever your services run; see [Run it in production](#run-it-in-production). ### The log handler `archil_logging.py` is a standard Python `logging` handler. It writes each record as one JSON line to `///.ndjson`, with hours in UTC: * **One directory per hour,** so a query for a time window only reads the hours it needs. * **One file per host.** When many hosts write to one disk in conditional mode and two of them write the same part of a file, the last write wins. Giving every writer its own file avoids that. The host name comes from the `INSTANCE` environment variable, or the machine's hostname. * **Every line has `ts`, `level`, `service`, `host`, and `msg`,** plus whatever fields you pass, so the agent can rely on them. * **Writes happen on a background thread.** `archil_logger` puts records on a queue with Python's `QueueHandler`, and a `QueueListener` thread writes them out. In our runs, an append took microseconds, but closing the previous hour's file waits for its writes to reach the disk in conditional mode. The queue keeps waits like that off your request path. ```python archil_logging.py theme={null} import atexit import json import logging import os import queue from datetime import datetime, timezone from logging.handlers import QueueHandler, QueueListener class HourlyJsonHandler(logging.Handler): """Appends each record as one JSON line to ////.ndjson, in UTC.""" def __init__(self, root, service, host): super().__init__() self.root, self.service, self.host = root, service, host self.path, self.file = None, None def emit(self, record): ts = datetime.fromtimestamp(record.created, timezone.utc) path = f"{self.root}/{self.service}/{ts:%Y-%m-%d/%H}/{self.host}.ndjson" if path != self.path: if self.file: self.file.close() os.makedirs(os.path.dirname(path), exist_ok=True) self.file, self.path = open(path, "a"), path entry = { "ts": ts.isoformat(timespec="milliseconds"), "level": record.levelname, "service": self.service, "host": self.host, "msg": record.getMessage(), **getattr(record, "fields", {}), } self.file.write(json.dumps(entry) + "\n") self.file.flush() def archil_logger(service, root="/mnt/logs", host=None): """A logger whose records are written to the disk by a background thread, so logging never blocks on storage.""" records = queue.Queue(-1) handler = HourlyJsonHandler(root, service, host or os.environ.get("INSTANCE") or os.uname().nodename) listener = QueueListener(records, handler) listener.start() atexit.register(listener.stop) # write out queued records on exit logger = logging.getLogger(service) logger.addHandler(QueueHandler(records)) logger.setLevel(logging.INFO) return logger ``` ### The demo services `shop.py` stands in for three services: `api`, `checkout`, and `worker`. Every api request has a `request_id` that also appears in the checkout service's log line, so the agent can follow a request across services. The worker fails about 8% of `send_receipt` jobs with an SMTP error. This is the kind of steady background noise that a useful triage job should not page you for. Two flag files let you cause trouble when you're ready: `/tmp/break-checkout` makes checkout's payment calls time out, and `/tmp/heic-uploads` makes the worker fail on HEIC images, a new error that doesn't affect anyone's order. ```python shop.py theme={null} import os import random import time import uuid from logging import ERROR, INFO from archil_logging import archil_logger BREAK_FLAG = "/tmp/break-checkout" HEIC_FLAG = "/tmp/heic-uploads" api, checkout, worker = (archil_logger(name) for name in ["api", "checkout", "worker"]) def log(logger, level, msg, **fields): logger.log(level, msg, extra={"fields": fields}) def pay(request_id): order_id = f"ord_{uuid.uuid4().hex[:10]}" amount = random.choice([1299, 2499, 4999, 8900]) if os.path.exists(BREAK_FLAG) and random.random() < 0.35: ms = random.randint(5000, 5100) log(checkout, ERROR, "payment authorization failed", request_id=request_id, order_id=order_id, amount_cents=amount, provider="payments-gateway", duration_ms=ms, error="upstream timeout after 5000ms (pool=payments-gateway, in_use=64/64)") return 502, ms ms = round(random.lognormvariate(5.0, 0.3)) log(checkout, INFO, "payment authorized", request_id=request_id, order_id=order_id, amount_cents=amount, provider="payments-gateway", duration_ms=ms) return 200, ms while True: path = random.choices(["/", "/search", "/cart", "/checkout"], weights=[4, 3, 2, 1])[0] request_id = f"req_{uuid.uuid4().hex[:12]}" if path == "/checkout": status, ms = pay(request_id) ms += random.randint(5, 20) else: status, ms = 200, round(random.lognormvariate(3, 0.5)) log(api, ERROR if status >= 500 else INFO, "request", method="POST" if path == "/checkout" else "GET", path=path, status=status, duration_ms=ms, request_id=request_id, user_id=f"u_{random.randint(1, 5000)}") if random.random() < 0.05: job = random.choice(["send_receipt", "sync_inventory", "resize_image"]) if job == "send_receipt" and random.random() < 0.08: log(worker, ERROR, "job failed", job=job, attempt=1, error="smtp 550: mailbox unavailable") elif job == "resize_image" and os.path.exists(HEIC_FLAG) and random.random() < 0.5: log(worker, ERROR, "job failed", job=job, attempt=1, error="unsupported image format: heic (thumbnail skipped)") else: log(worker, INFO, "job done", job=job, duration_ms=round(random.lognormvariate(4, 0.7))) time.sleep(random.uniform(0.01, 0.05)) ``` ### Start everything The setup script creates two disks: `app-logs` for the logs and `obs-agent` for the agent's tools and output. It then starts the sandbox and runs the shop as a supervised [service](/compute/sandboxes/introduction#hosting-network-services). The sandbox mounts `app-logs` in conditional mode, the mode you'd use for a fleet of hosts. `idleTtlSeconds: 0` keeps it running when no one is connected. Finally, the script installs the agent's tools on `obs-agent`: the DuckDB command-line binary and Vega. With them, this small script renders a Vega-Lite chart to SVG: ```javascript vl2svg.mjs theme={null} // Usage: node vl2svg.mjs < chart.vl.json > chart.svg import { readFileSync } from "node:fs"; import * as vega from "vega"; import { compile } from "vega-lite"; const spec = JSON.parse(readFileSync(0, "utf8")); const view = new vega.View(vega.parse(compile(spec).spec), { renderer: "none" }); process.stdout.write(await view.toSVG()); ``` ```typescript setup.ts theme={null} import * as archil from "disk"; import { readFile } from "node:fs/promises"; const { disk: logs } = await archil.createDisk({ name: "app-logs" }); const { disk: agent } = await archil.createDisk({ name: "obs-agent" }); // The application host. Its services write their logs straight to the disk. const sandbox = await archil.createSandbox({ name: "shop", baseImage: "python:3.13", vcpuCount: 1, memSizeMiB: 1024, idleTtlSeconds: 0, mounts: [{ disk: logs, path: "/mnt/logs", conditional: true }], }); for (const file of ["shop.py", "archil_logging.py"]) { await sandbox.files.uploadFile(await readFile(file), `/root/${file}`); } await sandbox.exec("archil-sandbox services create shop --env INSTANCE=shop-1 -- python3 /root/shop.py"); // The agent's workspace: DuckDB for SQL, and Vega to render charts as SVG. await agent.putObject("bin/vl2svg.mjs", await readFile("vl2svg.mjs")); const res = await archil.exec({ disks: { agent: { disk: agent, conditional: true } }, command: [ "cd agent", "curl -sfL -o /tmp/duckdb.zip https://github.com/duckdb/duckdb/releases/download/v1.5.6/duckdb_cli-linux-amd64.zip", "unzip -oq /tmp/duckdb.zip -d bin", "npm install --no-audit --no-fund --silent vega@6.4.0 vega-lite@6.5.0", "bin/duckdb --version", ].join(" && "), }); if (res.exitCode !== 0) throw new Error(res.stderr); console.log(res.stdout); ``` ``` v1.5.6 (Variegata) 069cc9f9b5 ``` The shop starts logging immediately. After a minute, check that the lines are arriving. This command runs DuckDB in a serverless microVM with both disks mounted read-only: ```typescript query.ts theme={null} import * as archil from "disk"; const [logs] = await archil.listDisks({ name: "app-logs" }); const [agent] = await archil.listDisks({ name: "obs-agent" }); const { stdout, timing } = await archil.exec({ disks: { logs: { disk: logs, readOnly: true }, agent: { disk: agent, readOnly: true } }, command: `agent/bin/duckdb -c " SELECT service, level, count(*) AS lines, max(ts) AS latest FROM read_json('logs/*/*/*/*.ndjson', union_by_name=true) GROUP BY ALL ORDER BY ALL"`, }); console.log(stdout, `${timing.totalMs} ms`); ``` ``` ┌──────────┬─────────┬───────┬─────────────────────────┐ │ service │ level │ lines │ latest │ │ varchar │ varchar │ int64 │ timestamp │ ├──────────┼─────────┼───────┼─────────────────────────┤ │ api │ INFO │ 648 │ 2026-11-06 15:37:31.867 │ │ checkout │ INFO │ 61 │ 2026-11-06 15:37:31.774 │ │ worker │ INFO │ 29 │ 2026-11-06 15:37:31.619 │ └──────────┴─────────┴───────┴─────────────────────────┘ 237 ms ``` ## Give the agent the logs `createDiskTools` turns a [workspace](/integrations/ai-sdk) of Archil disks into AI SDK tools: `read_file`, `write_file`, `delete_file`, `list_files`, `glob`, `grep`, and `run_bash`. This workspace mounts `app-logs` read-only at `/logs` and `obs-agent` at `/agent`. `run_bash` runs each command in a fresh serverless microVM with both disks mounted, and `grep` searches files in parallel next to the data with [`disk.grep`](/compute/search-files). The instructions describe the log layout and how to query it. Both agents in this guide share this module: ```typescript agent.ts theme={null} import * as archil from "disk"; import { createDiskTools } from "@archildata/ai-sdk"; export const [logs] = await archil.listDisks({ name: "app-logs" }); export const [agentDisk] = await archil.listDisks({ name: "obs-agent" }); // Logs are mounted read-only: the agent can read every line but can't change one. export const tools = createDiskTools( archil.workspace({ logs: { disk: logs, readOnly: true }, agent: { disk: agentDisk, conditional: true }, }), ); export const model = "anthropic/claude-sonnet-5.5"; // Each agent step resends the conversation so far. Caching bills the repeated part at a fraction of the input price. export const providerOptions = { gateway: { caching: "auto" as const } }; export const instructions = ` You are the on-call engineer for a web shop with three services: api, checkout, and worker. Logs are under /logs////.ndjson, partitioned by UTC hour. Each line is one JSON object with ts (UTC), level, service, msg, host, and service-specific fields. request_id links an api request to the checkout call it made. /logs is read-only. Your own files live under /agent. Query logs with SQL in run_bash, from the workspace root: agent/bin/duckdb -c "SELECT ... FROM read_json('logs/*/2026-11-06/*/*.ndjson', union_by_name=true)" Glob only the days and hours you need. Use the grep tool to find specific lines. Base every claim on a query or a log line you have read, and quote request_ids or timestamps as evidence. `.trim(); ``` ## Triage errors on a schedule Most triage runs find nothing, and a frontier model is an expensive way to find nothing. So triage runs in three steps, and each step runs only if the one before it found something: 1. **Detect** with deterministic SQL. A DuckDB query compares the last 15 minutes with the hour before. It flags error types that are new or at least twice as common as before, and paths whose p95 latency at least doubled. It runs in one serverless call, next to the data, with no model. 2. **Classify** with a decision model. [`experimental_decide`](https://ai-sdk.dev/docs/reference/ai-sdk-core/decide) asks a small decision model whether each anomaly needs an investigation, and gets back a typed answer with probabilities instead of prose. This guide uses `typesafe-ai/jev` through AI Gateway. Any decision model in the gateway works. 3. **Investigate** with a frontier model, only for what the classifier flagged. The agent from the previous section confirms the problem, measures its impact, traces requests across services, writes a report, and records the issue in `/agent/triage/known-issues.md`. The classifier reads those notes on later runs, so an issue that's already been reported isn't investigated again unless it gets worse. The detection query groups errors by fingerprint: the message and error text with numbers masked, so `smtp 550` and `smtp 551` count as one error type: ```sql detect.sql theme={null} -- Compare the last 15 minutes with the hour before it. Returns one row per anomaly, or nothing. -- triage.ts fills in the list of log files for the current and previous two hours. SET VARIABLE t1 = (now() AT TIME ZONE 'UTC')::TIMESTAMP; SET VARIABLE t0 = getvariable('t1') - INTERVAL 15 MINUTE; SET VARIABLE b0 = getvariable('t0') - INTERVAL 60 MINUTE; CREATE TEMP TABLE events AS SELECT ts, service, path, duration_ms, ts >= getvariable('t0') AS recent, level = 'ERROR' OR coalesce(status, 0) >= 500 AS is_error, -- An error's fingerprint is its message with numbers masked, so repeats group together. msg || coalesce(': ' || regexp_replace(error, '[0-9]+', 'N', 'g'), '') AS fingerprint, coalesce(error, msg) AS example FROM read_json([FILES], union_by_name=true) WHERE ts >= getvariable('b0') AND ts < getvariable('t1'); WITH volume AS ( SELECT service, count(*) FILTER (recent) AS recent_n, count(*) FILTER (NOT recent) AS baseline_n FROM events GROUP BY service ), errors AS ( SELECT service, fingerprint, any_value(example) AS example, min(ts) FILTER (recent) AS first_seen, count(*) FILTER (recent) AS recent_errors, count(*) FILTER (NOT recent) AS baseline_errors FROM events WHERE is_error GROUP BY ALL ), latency AS ( SELECT service, path, quantile_cont(duration_ms, 0.95) FILTER (recent) AS recent_p95, quantile_cont(duration_ms, 0.95) FILTER (NOT recent) AS baseline_p95, count(*) FILTER (recent) AS recent_n, count(*) FILTER (NOT recent) AS baseline_n FROM events WHERE duration_ms IS NOT NULL GROUP BY ALL ) -- Errors that are new, or whose share of the service's lines at least doubled SELECT 'errors' AS kind, service, fingerprint AS signal, example, first_seen, round(100 * recent_errors / recent_n, 2) AS recent_pct, round(100 * baseline_errors / nullif(baseline_n, 0), 2) AS baseline_pct, recent_errors AS recent_count, baseline_errors AS baseline_count FROM errors JOIN volume USING (service) WHERE recent_errors >= 5 AND (baseline_errors = 0 OR recent_errors / recent_n >= 2 * baseline_errors / baseline_n) UNION ALL -- p95 latency that at least doubled on a path with enough traffic to compare SELECT 'latency', service, coalesce(path, '(all)'), NULL, NULL, round(recent_p95), round(baseline_p95), recent_n, baseline_n FROM latency WHERE recent_n >= 20 AND baseline_n >= 20 AND recent_p95 >= 2 * baseline_p95 ORDER BY kind, recent_count DESC; ``` `triage.ts` runs the three steps and prints the cost of each model call, which AI Gateway reports with every response. The investigation returns structured output, validated against a [Zod](https://zod.dev) schema, so you can route issues by severity or open tickets from them without parsing prose. If `SLACK_WEBHOOK_URL` is set, the script also posts each issue to that [incoming webhook](https://api.slack.com/messaging/webhooks). ```typescript triage.ts theme={null} import { experimental_decide as decide, Output, ToolLoopAgent, isStepCount } from "ai"; import * as archil from "disk"; import { readFile } from "node:fs/promises"; import { z } from "zod"; import { agentDisk, instructions, logs, model, providerOptions, tools } from "./agent"; const cost = (metadata: any) => Number(metadata?.gateway?.cost ?? 0); // 1. Detect: deterministic SQL next to the data. No model. const sql = await readFile("detect.sql", "utf8"); const gate = await archil.exec({ disks: { logs: { disk: logs, readOnly: true }, agent: { disk: agentDisk, readOnly: true } }, command: `files=$(for h in 0 1 2; do ls logs/*/$(date -u -d "-$h hour" +%F/%H)/*.ndjson 2>/dev/null; done | sed "s/.*/'&'/" | paste -sd, -) [ -n "$files" ] || exit 0 sed "s|\\[FILES\\]|[$files]|" <<'SQL' | agent/bin/duckdb -json ${sql} SQL`, }); if (gate.exitCode !== 0) throw new Error(gate.stderr); const anomalies: Record[] = gate.stdout.trim() ? JSON.parse(gate.stdout) : []; console.log(`detect: ${anomalies.length} anomalies in ${gate.timing.totalMs} ms`); if (anomalies.length === 0) process.exit(0); // 2. Classify: a decision model decides which anomalies deserve an investigation. const known = await agentDisk.getObject("triage/known-issues.md").then((b) => new TextDecoder().decode(b), () => ""); const decision = await decide({ model: "typesafe-ai/jev", state: [ { type: "text", text: `Issues already reported, with their rates:\n${known || "(none)"}` }, { type: "json", value: anomalies }, ], questions: Object.fromEntries( anomalies.map((anomaly, i) => [ `anomaly_${i}`, { type: "choice", instructions: `Does this anomaly from a web shop's logs need an engineer to investigate? ${JSON.stringify(anomaly)}`, criteria: { investigate: "Customers are likely affected (failed requests, payments, or orders, or much slower responses), and it is new or clearly worse than an issue already reported.", ignore: "Little or no customer impact, such as a skipped background task or a rejected email, or an issue already reported at a similar rate.", }, } as const, ]), ), }); const flagged = anomalies.filter((anomaly, i) => { const answer = decision.answers[`anomaly_${i}`]; const p = answer.probabilities?.[answer.choice]; console.log(`classify: ${answer.choice} (${p}) ${anomaly.kind} ${anomaly.service} ${anomaly.signal}`); return answer.choice === "investigate"; }); console.log(`classify: $${cost(decision.providerMetadata).toFixed(6)}`); if (flagged.length === 0) process.exit(0); // 3. Investigate: a frontier model digs into what the first two steps flagged. const Issue = z.object({ severity: z.enum(["critical", "warning"]), service: z.string(), title: z.string().describe("One line a human can act on, e.g. 'Checkout payments timing out'"), started: z.string().describe("UTC timestamp of the first occurrence"), impact: z.string().describe("Quantified impact, e.g. '31% of checkouts failing (412 of 1,320)'"), evidence: z.array(z.string()).describe("Two or three log lines or query results that prove it"), nextStep: z.string(), }); const agent = new ToolLoopAgent({ model, instructions, tools, providerOptions, stopWhen: isStepCount(40), output: Output.object({ schema: z.object({ issues: z.array(Issue), summary: z.string() }) }), }); const now = new Date().toISOString(); const result = await agent.generate({ prompt: ` It is ${now}. An anomaly detector compared the last 15 minutes of logs with the hour before and flagged: ${JSON.stringify(flagged, null, 2)} Investigate each one: confirm it, find when it started, quantify how many requests, jobs, or users it affects, and check whether other services are involved. Trace a few request_ids across services. Anomalies with the same cause are one issue. Report only these anomalies. A classifier has already judged the detector's other findings to be low-impact. Write a short Markdown report to /agent/triage/reports/${now.slice(0, 16).replace(":", "")}.md, and add each issue to /agent/triage/known-issues.md with its current rate.`, }); const { issues, summary } = result.output; const usage = result.totalUsage; const spent = result.steps.reduce((sum, step) => sum + cost(step.providerMetadata), 0); console.log( `investigate: ${result.steps.length} steps, ${usage.inputTokens} input tokens ` + `(${usage.inputTokenDetails.cacheReadTokens ?? 0} cached), $${spent.toFixed(4)}\n`, ); console.log(summary); for (const issue of issues) { console.log(`\n[${issue.severity}] ${issue.service}: ${issue.title}`); console.log(` started ${issue.started}; ${issue.impact}`); for (const line of issue.evidence) console.log(` - ${line}`); console.log(` next: ${issue.nextStep}`); } if (issues.length > 0 && process.env.SLACK_WEBHOOK_URL) { const text = issues.map((i) => `*[${i.severity}] ${i.service}: ${i.title}*\n${i.impact}\n${i.nextStep}`); await fetch(process.env.SLACK_WEBHOOK_URL, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ text: text.join("\n\n") }), }); } ``` Let the shop run for an hour so the detector has a baseline, then run triage: ``` detect: 0 anomalies in 553 ms ``` The worker's SMTP failures are steady, so the detector doesn't flag them, and no model is called. Now cause two problems at once, a real incident and a harmless new error: ```typescript break.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "shop")!; // Break checkout, and at the same time start a new, harmless error: users uploading HEIC images. await sandbox.exec("touch /tmp/break-checkout /tmp/heic-uploads"); console.log(`checkout broken at ${new Date().toISOString()}`); ``` Run `triage.ts` again a couple of minutes later: ``` detect: 5 anomalies in 545 ms classify: investigate (0.81) errors api request classify: investigate (0.92) errors checkout payment authorization failed: upstream timeout after Nms (pool=payments-gateway, in_use=N/N) classify: ignore (0.9) errors worker job failed: unsupported image format: heic (thumbnail skipped) classify: investigate (0.79) latency api /checkout classify: investigate (0.85) latency checkout (all) classify: $0.000068 investigate: 9 steps, 948513 input tokens (824558 cached), $0.4653 All four detector anomalies (api errors, checkout errors, api /checkout latency, checkout latency) share one cause. Checkout calls to payments-gateway time out after 5000ms because the pool is full (in_use=64/64). The first failure was at 16:41:32.783, and about a third of checkouts have failed since, with no sign of recovery. The report is at /agent/triage/reports/2026-11-06T1648.md and the issue is added to /agent/triage/known-issues.md. [critical] checkout: Checkout payments timing out: payments-gateway pool exhausted (api 502s and latency are the same issue) started 2026-11-06 16:41:32.783; 410 of 1,241 POST /checkout failed (33.0%) from 16:41:30 to 16:48:00. 395 of 1,096 checkout users were affected, with about $17,847 in failed orders. The last 3 minutes were 201 of 585 failed (34.4%), so it is not recovering. Failed requests take about 5.06s, while successful ones are unchanged (p50 162ms). Other api endpoints are unaffected. - checkout 2026-11-06 16:41:32.783 ERROR payment authorization failed req_ece8f904342e ord_fad33359b9 amount=1299 error='upstream timeout after 5000ms (pool=payments-gateway, in_use=64/64)'. - 410 of 410 api 502s have a checkout ERROR with the same request_id, and no api 200 has one. Before 16:41:30 there were 0 api 5xx in 128,046 requests and 0 checkout errors since 15:37. Checkout errors per minute from 16:41 to 16:47: 30, 47, 64, 68, 72, 74, 55. next: Check payments-gateway status and latency with the provider. Inspect checkout's gateway connection pool for stuck or leaked connections, and look at what changed around 16:41. If nothing clear turns up, restart checkout to reset the pool. As a stopgap, raise the pool size or lower the 5s timeout so requests fail fast. ``` The detector flagged five anomalies in about half a second. The classifier sent the four related to checkout to investigation and dropped the HEIC errors, for less than a hundredth of a cent. The investigation found that the four share one cause, found the first failing request to the millisecond, and matched every api error to its checkout error by `request_id`. It took 9 steps and \$0.47, with most of its input served from the gateway's cache. The model writes its own summary, so your wording and numbers will differ. Run triage again while checkout is still broken. The classifier finds the issue in the notes and stops there: ``` detect: 5 anomalies in 499 ms classify: ignore (0.61) errors api request classify: ignore (0.7) errors checkout payment authorization failed: upstream timeout after Nms (pool=payments-gateway, in_use=N/N) classify: ignore (0.93) errors worker job failed: unsupported image format: heic (thumbnail skipped) classify: ignore (0.67) latency api /checkout classify: ignore (0.68) latency checkout (all) classify: $0.000078 ``` To run triage continuously, schedule `triage.ts` every few minutes with any scheduler, such as cron or a scheduled GitHub Actions workflow. Quiet runs cost one serverless call: ```bash theme={null} */5 * * * * cd /path/to/project && npx tsx triage.ts >> triage.log 2>&1 ``` ## Build dashboards on request The dashboard agent explores the logs, and then writes `build.sh`, a script that queries the logs with DuckDB, renders each chart to SVG, and writes a static `index.html`. Then it runs the script and checks the page. The script uses time windows relative to when it runs, so the page can be rebuilt later without the model. The script shares the page with a [share URL](/sdks/typescript#share-urls). Anyone with the link can open it, with no Archil account, until it expires (24 hours by default, up to 7 days). The link serves the page with scripts disabled, so the charts are rendered to SVG ahead of time rather than in the browser. ```typescript dashboard.ts theme={null} import { Output, ToolLoopAgent, isStepCount } from "ai"; import { z } from "zod"; import { agentDisk, instructions, model, providerOptions, tools } from "./agent"; const question = process.argv.slice(2).join(" "); const agent = new ToolLoopAgent({ model, instructions, tools, providerOptions, stopWhen: isStepCount(40), output: Output.object({ schema: z.object({ slug: z.string().describe("Short kebab-case name of the dashboard directory"), findings: z.string().describe("Two or three sentences answering the question"), }), }), }); const { output, steps } = await agent.generate({ prompt: ` Build a dashboard that answers: "${question}" Explore the logs first, then write /agent/dashboards//build.sh. When it runs from the workspace root, it must query the logs with agent/bin/duckdb -json, render 2 to 4 Vega-Lite charts to SVG with node agent/bin/vl2svg.mjs < spec.json > chart.svg, and write /agent/dashboards//index.html: one static page with a title, a one-line takeaway above each chart, and the SVGs inlined. The page is served with scripts disabled, so don't use JavaScript. Give charts a fixed width of about 800 pixels and keep each one under 500 data points. Use time windows relative to now, so the script still works when it runs again later. Run build.sh, then read index.html back to check it.`, }); const share = await agentDisk.share(`dashboards/${output.slug}/index.html`, { expiresIn: 7 * 24 * 3600 }); const spent = steps.reduce((sum, step) => sum + Number((step.providerMetadata?.gateway as any)?.cost ?? 0), 0); console.log(`${steps.length} steps, $${spent.toFixed(4)}\n`); console.log(output.findings); console.log(`\n${share.url}`); ``` ```bash theme={null} npx tsx dashboard.ts "How is checkout doing? Show the error rate and latency over the last 30 minutes, and which users are affected." ``` ``` 13 steps, $0.6318 Checkout is failing right now. In the 30 minutes to 16:53Z (16:23–16:53), 743 of 5,935 checkouts failed (12.5%). There were no errors before 16:41:32Z. Since then about a third fail, and the last 5 minutes are at 34.1%. [...] Successful checkouts are unchanged (p50 149ms, p95 244ms), but failures wait about 5,048ms, which pushes overall p95 to 5,059ms. 681 of 3,465 users who tried to check out (19.7%) hit at least one failure, about $31.5k in orders, and 225 of them failed on every attempt. [...] https://control.green.us-west-2.aws.prod.archil.com/api/shared/eyJkIjoiZnMt... ``` ![Checkout health dashboard. The share of failed checkouts per minute jumps from 0 to about 35% at 16:41 UTC and drops back to 0 after 16:54. Payment authorization p95 jumps from about 250 ms to 5 seconds over the same minutes while p50 stays near 150 ms. A third chart shows 30 to 70 users a minute hitting a failed checkout during the incident.](https://mintcdn.com/regattastorage/POnfsiHp-CqvMnhg/images/guides/agentic-observability-dashboard.png?fit=max\&auto=format\&n=POnfsiHp-CqvMnhg\&q=85\&s=38e8834a87171edd069b43ad92749444) The top of the dashboard the agent built, rebuilt without the model after checkout recovered. The agent decided what to chart on its own. The run took about two minutes. A dashboard is a question a person asked, so it goes straight to the frontier model, unlike triage. ### Rebuild a dashboard without the model `build.sh` is an ordinary script on the disk, so rebuilding a dashboard is one serverless call, with no model: ```typescript refresh.ts theme={null} import * as archil from "disk"; const slug = process.argv[2]; const [logs] = await archil.listDisks({ name: "app-logs" }); const [agent] = await archil.listDisks({ name: "obs-agent" }); const res = await archil.exec({ disks: { logs: { disk: logs, readOnly: true }, agent: { disk: agent, conditional: true }, }, command: `bash agent/dashboards/${slug}/build.sh`, }); if (res.exitCode !== 0) throw new Error(res.stderr); console.log(`rebuilt in ${res.timing.totalMs} ms`); console.log((await agent.share(`dashboards/${slug}/index.html`)).url); ``` ```bash theme={null} npx tsx refresh.ts checkout-health ``` ``` rebuilt in 3885 ms https://control.green.us-west-2.aws.prod.archil.com/api/shared/eyJkIjoiZnMt... ``` The script recomputes the takeaways as well as the charts. After checkout recovered, the rebuilt page's first takeaway read "the last 5 minutes are at 1.6%, so it has mostly recovered." Rebuild a dashboard on a schedule to keep it current, or ask the agent for a new one when the question changes. ## Run it in production Mount the `app-logs` disk in conditional mode wherever your services run, and log through `archil_logger`. The agent scripts stay the same. > [tab] Kubernetes > > Mount the disk with the [CSI driver](/reference/csi-driver) as a `ReadWriteMany` volume, which mounts it in > conditional mode. Create the PersistentVolume and a PersistentVolumeClaim named `app-logs` as shown in > [Static provisioning](/reference/csi-driver#static-provisioning), then mount the claim into each service's pods: > > ```yaml theme={null} > spec: > template: > spec: > containers: > - name: api > volumeMounts: > - name: app-logs > mountPath: /mnt/logs > volumes: > - name: app-logs > persistentVolumeClaim: > claimName: app-logs > ``` > > A pod's hostname is its name, so each replica writes its own files without setting `INSTANCE`. > [tab] VMs > > [Mount the disk](/mounting/linux) in conditional mode on each host. Each host writes its own files, named after its > hostname. > > ```bash theme={null} > sudo archil mount /mnt/logs --region aws-us-west-2 --conditional > ``` A few things to plan for: * **Several processes per host.** If a host runs several processes that log, such as Gunicorn workers, give each one its own file: pass `host=f"{socket.gethostname()}-{os.getpid()}"` to `archil_logger`. * **Other languages.** Any logger can write the same layout: one JSON object per line with `ts`, `level`, `service`, `host`, and `msg`, appended to `///.ndjson`, one file per process, written off the request path. * **Retention.** Each day is its own directory, so retention is a daily delete. This serverless command removes the logs from 30 days ago. It mounts the disk in conditional mode, so it can delete while your services keep writing: ```typescript theme={null} await archil.exec({ disks: { logs: { disk: logs, conditional: true } }, command: "rm -rf logs/*/$(date -u -d '30 days ago' +%F)", }); ``` * **Big log volumes.** Tell the agent to glob only the hours it needs (the instructions in `agent.ts` already do) so its queries read only those files. For long-range questions, have a scheduled job compact older hours into Parquet with DuckDB's `COPY ... TO`. * **Sensitive data.** The agent and anyone with a share link can see what's in the logs. Remove secrets and personal data before you log them. ## Clean up Delete the sandbox and both disks: ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "shop")!; await sandbox.stop(); await sandbox.delete(); for (const name of ["app-logs", "obs-agent"]) { const [disk] = await archil.listDisks({ name }); await disk.delete(); } ``` ## Next steps * [AI SDK integration](/integrations/ai-sdk): mount several disks as one workspace and choose what the agent can reach * [Search files](/compute/search-files): limits and concurrency for the parallel search behind the `grep` tool * [Search logs across sandboxes](/guides/analytics/log-search): search the same kind of logs with `disk.grep` and `jq`, without an agent * [Analytics with DuckDB](/guides/analytics/duckdb): query Parquet in an S3 bucket and serve results over HTTPS * [Sharing disks](/concepts/sharing-disks): how exclusive, shared, and conditional mounts behave # Claude Code Source: https://docs.archil.com/guides/agents/claude-code Run Claude Code headlessly in an Archil sandbox, with its workspace and session history on an Archil disk. [Claude Code](https://code.claude.com/docs) is Anthropic's coding agent. In an Archil [sandbox](/compute/sandboxes/introduction) it gets a Linux VM of its own, so you can let it work without permission prompts. Its workspace and session history live on an Archil disk, so the work outlives the sandbox, and a new sandbox can pick up the same conversation. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [Anthropic API key](https://console.anthropic.com/settings/keys), then install the SDK: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export ANTHROPIC_API_KEY=sk-ant-... ``` ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` ## Run Claude Code in a sandbox This script creates a disk, starts a sandbox with the disk mounted at `/workspace`, installs Claude Code, and gives it a task in print mode (`claude -p`), which works until the task is done, prints the answer, and exits. Then it deletes the sandbox; everything the agent made is on the disk. ```typescript start.ts theme={null} import * as archil from "disk"; const env = { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, CLAUDE_CONFIG_DIR: "/workspace/.claude", // keep sessions on the disk IS_SANDBOX: "1", // sandboxes run as root; allows --dangerously-skip-permissions }; const { disk } = await archil.createDisk({ name: "claude-code-workspace" }); const sandbox = await archil.createSandbox({ name: "claude-code", mounts: [{ disk, path: "/workspace" }] }); await sandbox.exec("npm install -g @anthropic-ai/claude-code"); const result = await sandbox.exec( 'claude -p "Create a hello-world Express app with a test, and run the test. Reply in two sentences." ' + "--model claude-sonnet-5 --dangerously-skip-permissions", { cwd: "/workspace", env }, ); console.log(result.stdout); await sandbox.stop(); await sandbox.delete(); ``` ```python start.py theme={null} import os import archil env = { "ANTHROPIC_API_KEY": os.environ["ANTHROPIC_API_KEY"], "CLAUDE_CONFIG_DIR": "/workspace/.claude", # keep sessions on the disk "IS_SANDBOX": "1", # sandboxes run as root; allows --dangerously-skip-permissions } disk = archil.create_disk(name="claude-code-workspace").disk sandbox = archil.create_sandbox( name="claude-code", mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")] ) sandbox.exec("npm install -g @anthropic-ai/claude-code") result = sandbox.exec( 'claude -p "Create a hello-world Express app with a test, and run the test. Reply in two sentences." ' "--model claude-sonnet-5 --dangerously-skip-permissions", cwd="/workspace", env=env, ) print(result.stdout) sandbox.stop() sandbox.delete() ``` Run it with `npx tsx start.ts` or `python start.py`. The agent's wording varies from run to run. One run printed: ```text theme={null} Created a hello-world Express app at `/workspace/hello-express` (`app.js` serving "Hello, World!" at `/`, plus `server.js` to run it) with a supertest/Jest test in `app.test.js`. Ran `npm test` and the test passed (1/1). ``` ## Pick up where you left off Claude Code's project and session history are on the disk, so any sandbox that mounts it can continue the conversation. This script starts a new sandbox on the same disk and runs Claude Code with `--continue`, which resumes the most recent conversation in `/workspace`. ```typescript continue.ts theme={null} import * as archil from "disk"; const env = { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, CLAUDE_CONFIG_DIR: "/workspace/.claude", IS_SANDBOX: "1", }; const [disk] = await archil.listDisks({ name: "claude-code-workspace" }); const sandbox = await archil.createSandbox({ name: "claude-code", mounts: [{ disk, path: "/workspace" }] }); await sandbox.exec("npm install -g @anthropic-ai/claude-code"); const result = await sandbox.exec( 'claude --continue -p "Add a /health route with a test and run all the tests. Reply in two sentences." ' + "--model claude-sonnet-5 --dangerously-skip-permissions", { cwd: "/workspace", env }, ); console.log(result.stdout); await sandbox.stop(); await sandbox.delete(); ``` ```python continue.py theme={null} import os import archil env = { "ANTHROPIC_API_KEY": os.environ["ANTHROPIC_API_KEY"], "CLAUDE_CONFIG_DIR": "/workspace/.claude", "IS_SANDBOX": "1", } [disk] = archil.list_disks(name="claude-code-workspace") sandbox = archil.create_sandbox( name="claude-code", mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")] ) sandbox.exec("npm install -g @anthropic-ai/claude-code") result = sandbox.exec( 'claude --continue -p "Add a /health route with a test and run all the tests. Reply in two sentences." ' "--model claude-sonnet-5 --dangerously-skip-permissions", cwd="/workspace", env=env, ) print(result.stdout) sandbox.stop() sandbox.delete() ``` ```text theme={null} Added a `/health` route returning `{ status: 'ok' }` with a corresponding test in `app.test.js`. All tests pass (2/2). ``` ## Going further * **Stream output.** Pass `onOutput: ({ data }) => process.stdout.write(data)` to `exec` (in Python, `on_output=lambda out: sys.stdout.buffer.write(out.data)`) to watch the agent as it works. Add `--output-format stream-json --verbose` to get one JSON event per line. * **Work interactively.** Start `claude` without `-p` in a pseudo-terminal with `sandbox.run`; see [Opening interactive shells](/compute/sandboxes/introduction#opening-interactive-shells). > [accordion] Use Vercel AI Gateway instead of an Anthropic key > > Claude Code can send its requests through [Vercel AI Gateway](https://vercel.com/docs/ai-gateway/coding-agents/claude-code). Export `AI_GATEWAY_API_KEY` instead of `ANTHROPIC_API_KEY`, and replace the `ANTHROPIC_API_KEY` line in `env` with these lines. `ANTHROPIC_API_KEY` must be an empty string, because Claude Code prefers it over `ANTHROPIC_AUTH_TOKEN`. On the Free plan, sandboxes can't reach the gateway, because it isn't on the [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist); use your provider key instead. > > ```typescript TypeScript theme={null} > ANTHROPIC_BASE_URL: "https://ai-gateway.vercel.sh/claude-code", > ANTHROPIC_AUTH_TOKEN: process.env.AI_GATEWAY_API_KEY!, > ANTHROPIC_API_KEY: "", > ``` > > ```python Python theme={null} > "ANTHROPIC_BASE_URL": "https://ai-gateway.vercel.sh/claude-code", > "ANTHROPIC_AUTH_TOKEN": os.environ["AI_GATEWAY_API_KEY"], > "ANTHROPIC_API_KEY": "", > ``` ## Clean up Each script deletes its sandbox when it finishes, so only the disk is left. Delete it: ```typescript cleanup.ts theme={null} import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "claude-code-workspace" }); await disk.delete(); ``` ```python cleanup.py theme={null} import archil [disk] = archil.list_disks(name="claude-code-workspace") disk.delete() ``` ## Next steps > [card] [OpenCode](/guides/agents/opencode) > > Run OpenCode headlessly in a sandbox. > [card] [OpenAI Codex](/guides/agents/codex) > > Run the Codex CLI headlessly in a sandbox. > [card] [Coding agent in a sandbox](/guides/agents/coding-agent) > > Build your own coding agent with the AI SDK instead. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. # Code interpreter Source: https://docs.archil.com/guides/agents/code-interpreter Give an AI agent a stateful Python kernel in a persistent sandbox to analyze a public dataset, with results on an Archil disk. An agent that answers questions about data needs to write Python, look at the result, and keep going. The usual way to give it that is a code interpreter: a `python` tool backed by a long-running Jupyter kernel, so a dataframe loaded in one call is still in memory for the next, just like cells in a notebook. In this guide you build one on Archil and use it to analyze a year of [Ookla's open speed test data](https://github.com/teamookla/ookla-open-data). Archil handles the three parts that are usually awkward: * **The data stays where it is.** The public S3 bucket is attached as an Archil disk and mounted read-only, so the kernel reads the Parquet files directly. There's no download step, and pandas fetches only the columns it asks for. * **The kernel's memory survives between sessions.** The kernel runs in a [persistent sandbox](/compute/sandboxes/introduction). When you pause the sandbox, the kernel is frozen with it, and the next session finds the dataframes still loaded. * **Results outlive the sandbox.** The agent writes charts and reports to a second Archil disk, which your app can read without going through the sandbox. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [AI Gateway](https://vercel.com/ai-gateway) key for the model. The Ookla bucket is in `us-west-2`, so that region is the closest to the data. ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 export AI_GATEWAY_API_KEY=... ``` Then install the dependencies. The TypeScript agent uses the [AI SDK](https://ai-sdk.dev); the Python agent calls the same model through the gateway's OpenAI-compatible API. ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk ai zod tsx ``` ```bash Python theme={null} pip install archil openai ``` ## Run a kernel in the sandbox Two files run inside the sandbox. The first is a setup script that installs pandas and Jupyter's `ipykernel` into a virtual environment and starts a kernel as a supervised service named `kernel`. The kernel's connection file, which holds its ports and signing key, is written in advance to `/root/kernel.json`, so clients can find the kernel and it keeps the same address if the service restarts. `MPLBACKEND=Agg` makes matplotlib render to files instead of trying to display plots. ```bash setup.sh theme={null} set -euo pipefail python3 -m venv /opt/kernel /opt/kernel/bin/pip install --quiet ipykernel pandas pyarrow matplotlib /opt/kernel/bin/python -c 'from jupyter_client import write_connection_file; write_connection_file("/root/kernel.json")' archil-sandbox services create kernel --dir /workspace --env MPLBACKEND=Agg -- \ /opt/kernel/bin/python -m ipykernel_launcher -f /root/kernel.json ``` The second is a small client that runs one piece of code in that kernel, the same way a notebook runs a cell. It reads the code from the `CODE` environment variable, prints whatever the code prints, the value of its last expression, and any traceback, and exits. The kernel keeps running, so the next call sees the same variables. If code runs for more than 100 seconds, the client interrupts it, like pressing stop in a notebook, and the kernel's state is kept. ```python run.py theme={null} import os import re from jupyter_client import BlockingKernelClient client = BlockingKernelClient() client.load_connection_file("/root/kernel.json") client.start_channels() client.wait_for_ready(timeout=60) output = [] def collect(msg): content = msg["content"] if msg["msg_type"] == "stream": output.append(content["text"]) elif msg["msg_type"] in ("execute_result", "display_data"): output.append(content["data"].get("text/plain", "") + "\n") elif msg["msg_type"] == "error": # Tracebacks are colored with ANSI escape codes; strip them for the model output.append(re.sub(r"\x1b\[[0-9;]*m", "", "\n".join(content["traceback"])) + "\n") try: client.execute_interactive(os.environ["CODE"], timeout=100, output_hook=collect) except TimeoutError: # Interrupt the running code so the kernel is ready for the next call client.control_channel.send(client.session.msg("interrupt_request", {})) output.append("Timed out after 100 seconds and interrupted.\n") print("".join(output)[-10_000:], end="") ``` The kernel has no network port, so nothing outside the sandbox can reach it. Your code talks to it only through `sandbox.exec`. ## Define the agent The agent has one tool, `python`, that runs `run.py` with `sandbox.exec`. The instructions describe the dataset, so the agent doesn't spend steps discovering it, and tell it to load data once and reuse it. ```typescript agent.ts theme={null} import { ToolLoopAgent, isStepCount, tool } from "ai"; import type { Sandbox } from "disk"; import { z } from "zod"; const instructions = `You are a data analyst with a Python tool backed by a persistent Jupyter kernel. Variables and dataframes stay in memory between calls, so load data once and reuse it. pandas, pyarrow, and matplotlib are installed. Keep printed output short. /data (read-only) holds Ookla's open fixed-broadband speed tests for 2024, one Parquet file per quarter: /data/quarter=1/2024-01-01_performance_fixed_tiles.parquet, and likewise quarter=2 (2024-04-01), quarter=3 (2024-07-01), and quarter=4 (2024-10-01). About 6.6 million rows each. Each row is a map tile about 600 m across. Columns: tile_x (longitude), tile_y (latitude), avg_d_kbps, avg_u_kbps, avg_lat_ms, tests, devices. Weight averages by tests. Read only the columns you need. Save charts with matplotlib's savefig and write reports to /workspace/output. Keep your final answer to three sentences.`; export async function runPython(sandbox: Sandbox, code: string) { const result = await sandbox.exec("/opt/kernel/bin/python /root/run.py", { env: { CODE: code }, timeoutSeconds: 120, }); return `${result.stdout}${result.stderr}` || "(no output)"; } export function createAnalyst(sandbox: Sandbox) { const python = tool({ description: "Run Python code in a persistent Jupyter kernel. Returns printed output, the value of the last expression, and errors.", inputSchema: z.object({ code: z.string() }), execute: async ({ code }) => { const output = await runPython(sandbox, code); console.log(`\n>>> ${code.trim().replaceAll("\n", "\n... ")}\n${output}`); return output; }, }); return new ToolLoopAgent({ model: "anthropic/claude-sonnet-5.5", instructions, tools: { python }, stopWhen: isStepCount(20), }); } ``` ```python agent.py theme={null} import json import os from openai import OpenAI client = OpenAI( base_url="https://ai-gateway.vercel.sh/v1", api_key=os.environ["AI_GATEWAY_API_KEY"], ) INSTRUCTIONS = """You are a data analyst with a Python tool backed by a persistent Jupyter kernel. Variables and dataframes stay in memory between calls, so load data once and reuse it. pandas, pyarrow, and matplotlib are installed. Keep printed output short. /data (read-only) holds Ookla's open fixed-broadband speed tests for 2024, one Parquet file per quarter: /data/quarter=1/2024-01-01_performance_fixed_tiles.parquet, and likewise quarter=2 (2024-04-01), quarter=3 (2024-07-01), and quarter=4 (2024-10-01). About 6.6 million rows each. Each row is a map tile about 600 m across. Columns: tile_x (longitude), tile_y (latitude), avg_d_kbps, avg_u_kbps, avg_lat_ms, tests, devices. Weight averages by tests. Read only the columns you need. Save charts with matplotlib's savefig and write reports to /workspace/output. Keep your final answer to three sentences.""" PYTHON_TOOL = { "type": "function", "function": { "name": "python", "description": "Run Python code in a persistent Jupyter kernel. " "Returns printed output, the value of the last expression, and errors.", "parameters": { "type": "object", "properties": {"code": {"type": "string"}}, "required": ["code"], }, }, } def run_python(sandbox, code): result = sandbox.exec( "/opt/kernel/bin/python /root/run.py", env={"CODE": code}, timeout_seconds=120, ) return f"{result.stdout}{result.stderr}" or "(no output)" def run_agent(sandbox, prompt, max_steps=20): messages = [ {"role": "system", "content": INSTRUCTIONS}, {"role": "user", "content": prompt}, ] for _ in range(max_steps): reply = client.chat.completions.create( model="anthropic/claude-sonnet-5.5", messages=messages, tools=[PYTHON_TOOL], ).choices[0].message messages.append(reply) if not reply.tool_calls: return reply.content for call in reply.tool_calls: code = json.loads(call.function.arguments)["code"] output = run_python(sandbox, code) print("\n>>> " + code.strip().replace("\n", "\n... ") + "\n" + output) messages.append({"role": "tool", "tool_call_id": call.id, "content": output}) return f"Stopped after {max_steps} steps." ``` Passing the code in an environment variable hands it to the kernel exactly as the model wrote it, with no shell quoting. `runPython` is exported so your own code can use the kernel too, which the second session does. ## Run the first analysis The first session creates two disks. `ookla-speedtests` is backed by the public `ookla-open-data` bucket, so it needs no credentials, and it's mounted at `/data`. `analysis-workspace` is an empty disk for results, mounted at `/workspace`. The script then creates the sandbox, runs `setup.sh`, and asks the agent a question that takes several steps to answer. > [!NOTE] > > A disk backed by a public bucket is read-only: Archil has no credentials to write to someone else's bucket. Always mount it > with `readOnly: true` (`read_only=True` in Python), and send everything the agent writes to a separate disk. When the agent finishes, `archil sync` flushes the kernel's writes to the disk, and the script reads the chart and the report straight from `analysis-workspace` with `getObject`. Finally, it pauses the sandbox, which stops compute and snapshots memory, including the kernel. ```typescript session-1.ts theme={null} import * as archil from "disk"; import { readFile, writeFile } from "node:fs/promises"; import { createAnalyst } from "./agent.ts"; const { disk: speedtests } = await archil.createDisk({ name: "ookla-speedtests", mounts: [ { type: "s3", bucketName: "ookla-open-data", bucketPrefix: "parquet/performance/type=fixed/year=2024/", }, ], }); const { disk: workspace } = await archil.createDisk({ name: "analysis-workspace" }); const sandbox = await archil.createSandbox({ name: "code-interpreter", baseImage: "python:3.13", vcpuCount: 4, memSizeMiB: 8192, mounts: [ { disk: speedtests, path: "/data", readOnly: true }, { disk: workspace, path: "/workspace" }, ], }); for (const file of ["setup.sh", "run.py"]) { await sandbox.files.uploadFile(await readFile(file), `/root/${file}`); } const setup = await sandbox.exec("bash /root/setup.sh", { timeoutSeconds: 600 }); if (setup.exitCode !== 0) throw new Error(setup.stderr); const agent = createAnalyst(sandbox); const { text } = await agent.generate({ prompt: "How did fixed broadband download speeds change over 2024? Load all four quarters into one " + "dataframe named tiles with tile_x, tile_y, avg_d_kbps, avg_lat_ms, tests, and a quarter column. " + "Compare the test-weighted mean download speed per quarter worldwide and in the contiguous US " + "(longitude -125 to -66, latitude 24 to 50). Save a chart " + "to /workspace/output/download-by-quarter.png and a short Markdown report to /workspace/output/report.md.", }); console.log(`\n${text}`); // Flush the kernel's writes, then read the results from the disk instead of the sandbox await sandbox.exec("archil sync /workspace"); await writeFile("download-by-quarter.png", await workspace.getObject("output/download-by-quarter.png")); console.log(new TextDecoder().decode(await workspace.getObject("output/report.md"))); await sandbox.pause(); ``` ```python session-1.py theme={null} import archil from agent import run_agent speedtests = archil.create_disk( name="ookla-speedtests", mounts=[ archil.S3Mount( bucket_name="ookla-open-data", bucket_prefix="parquet/performance/type=fixed/year=2024/", ) ], ).disk workspace = archil.create_disk(name="analysis-workspace").disk sandbox = archil.create_sandbox( name="code-interpreter", base_image="python:3.13", vcpu_count=4, mem_size_mib=8192, mounts=[ archil.SandboxMountSpec(disk=speedtests, path="/data", read_only=True), archil.SandboxMountSpec(disk=workspace, path="/workspace"), ], ) for file in ["setup.sh", "run.py"]: sandbox.files.upload_file(file, f"/root/{file}") setup = sandbox.exec("bash /root/setup.sh", timeout_seconds=600) if setup.exit_code != 0: raise RuntimeError(setup.stderr) text = run_agent( sandbox, "How did fixed broadband download speeds change over 2024? Load all four quarters into one " "dataframe named tiles with tile_x, tile_y, avg_d_kbps, avg_lat_ms, tests, and a quarter column. " "Compare the test-weighted mean download speed per quarter worldwide and in the contiguous US " "(longitude -125 to -66, latitude 24 to 50). Save a chart " "to /workspace/output/download-by-quarter.png and a short Markdown report to /workspace/output/report.md.", ) print(f"\n{text}") # Flush the kernel's writes, then read the results from the disk instead of the sandbox sandbox.exec("archil sync /workspace") with open("download-by-quarter.png", "wb") as f: f.write(workspace.get_object("output/download-by-quarter.png")) print(workspace.get_object("output/report.md").decode()) sandbox.pause() ``` ```bash TypeScript theme={null} npx tsx session-1.ts ``` ```bash Python theme={null} python session-1.py ``` The agent prints each piece of code it runs, followed by the output. Its first call loads the four quarters, about 26 million rows, into `tiles`, and every call after that works on the dataframe already in memory. Output from one run, trimmed: ```text theme={null} >>> import pandas as pd, pyarrow.parquet as pq, os ... fs=[] ... for q in range(1,5): ... p=f"/data/quarter={q}/2024-{(q-1)*3+1:02d}-01_performance_fixed_tiles.parquet" ... d=pd.read_parquet(p,columns=['tile_x','tile_y','avg_d_kbps','avg_lat_ms','tests']);d['quarter']=q;fs.append(d) ... tiles=pd.concat(fs,ignore_index=True) ... print(tiles.shape);print(tiles.isna().sum().to_dict());print(tiles.describe().T[['min','max','mean']]) (26364521, 6) (output trimmed) >>> def wm(g): return (g.avg_d_kbps*g.tests).sum()/g.tests.sum() ... us=tiles[tiles.tile_x.between(-125,-66)&tiles.tile_y.between(24,50)] (code trimmed) world_mbps us_mbps world_tests us_tests world_tiles us_tiles quarter 1 168.8 269.4 118589626 19831638 6655986 1707697 2 175.7 273.6 104427060 18476595 6492072 1726030 3 180.6 280.7 107417601 19345051 6655377 1744184 4 195.9 303.1 104326033 19550227 6561086 1733619 (two more calls trimmed: a sanity check, then the chart and report) Fixed broadband download speeds rose every quarter of 2024. The test-weighted mean went from 168.8 Mbps in Q1 to 195.9 Mbps in Q4 worldwide (+16%). In the contiguous US it went from 269.4 to 303.1 Mbps (+12.5%), and the biggest jump in both came in Q4. (answer and report trimmed) ``` The chart, downloaded from the disk to `download-by-quarter.png`: ![Line chart of test-weighted mean fixed broadband download speed by quarter in 2024, rising from 169 to 196 Mbps worldwide and from 269 to 303 Mbps in the contiguous US](https://mintcdn.com/regattastorage/nttzS7o84Z9f2v63/images/guides/code-interpreter-download-by-quarter.png?fit=max\&auto=format\&n=nttzS7o84Z9f2v63\&q=85\&s=beceff6088d2a28a780316474890a840) ## Continue in a later session A paused sandbox keeps its memory, so the kernel and everything in it come back on `resume()`. Before handing the agent its next question, the script runs a line of its own in the kernel to show that `tiles` is still loaded. The agent then builds on it without reading the files again. ```typescript session-2.ts theme={null} import * as archil from "disk"; import { writeFile } from "node:fs/promises"; import { createAnalyst, runPython } from "./agent.ts"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "code-interpreter")!; await sandbox.resume(); console.log(await runPython(sandbox, 'print(f"tiles: {len(tiles):,} rows in memory")')); const agent = createAnalyst(sandbox); const { text } = await agent.generate({ prompt: "Using the tiles dataframe that's already loaded, look at latency in Q4: what share of tests had " + "an average latency under 20 ms, worldwide and in the contiguous US? Save a chart to " + "/workspace/output/latency-q4.png and append a section to /workspace/output/report.md.", }); console.log(`\n${text}`); await sandbox.exec("archil sync /workspace"); const mount = sandbox.mounts.find((m) => m.path === "/workspace")!; const workspace = await archil.getDisk(mount.diskId); await writeFile("latency-q4.png", await workspace.getObject("output/latency-q4.png")); console.log(new TextDecoder().decode(await workspace.getObject("output/report.md"))); await sandbox.pause(); ``` ```python session-2.py theme={null} import archil from agent import run_agent, run_python sandbox = next(s for s in archil.list_sandboxes() if s.name == "code-interpreter") sandbox.resume() print(run_python(sandbox, 'print(f"tiles: {len(tiles):,} rows in memory")')) text = run_agent( sandbox, "Using the tiles dataframe that's already loaded, look at latency in Q4: what share of tests had " "an average latency under 20 ms, worldwide and in the contiguous US? Save a chart to " "/workspace/output/latency-q4.png and append a section to /workspace/output/report.md.", ) print(f"\n{text}") sandbox.exec("archil sync /workspace") mount = next(m for m in sandbox.mounts if m.path == "/workspace") workspace = archil.get_disk(mount.disk_id) with open("latency-q4.png", "wb") as f: f.write(workspace.get_object("output/latency-q4.png")) print(workspace.get_object("output/report.md").decode()) sandbox.pause() ``` The first line comes from the script's own `runPython` call, before the agent does anything. After that, the agent checks what's in the kernel and answers from `tiles` without reading any Parquet files. Output from one run, trimmed: ```text theme={null} tiles: 26,364,521 rows in memory >>> try: ... print(tiles.shape); print(tiles.dtypes); print(tiles.head()) (output and two more inspection calls trimmed) >>> print(us.columns.tolist(), us.quarter.value_counts().sort_index().to_dict(), us.tile_x.agg(['min','max']).tolist(), us.tile_y.agg(['min','max']).tolist()) ... q4=tiles[tiles.quarter==4] ... m=(q4.tile_x.between(-125,-66))&(q4.tile_y.between(24,50)) ... def share(d): ... return d.tests[d.avg_lat_ms<20].sum()/d.tests.sum(), d.tests.sum(), len(d) ... w=share(q4); u=share(q4[m]); print(w,u) ... print((q4[m].avg_d_kbps*q4[m].tests).sum()/q4[m].tests.sum()/1000) ['tile_x', 'tile_y', 'avg_d_kbps', 'avg_lat_ms', 'tests', 'quarter'] {1: 1707697, 2: 1726030, 3: 1744184, 4: 1733619} [-124.99969999999999, -66.0031] [24.003800000000002, 49.998900000000006] (np.float64(0.8092839397046756), np.int64(104326033), 6561086) (np.float64(0.7952972106155085), np.int64(19550227), 1733619) 303.06130137808634 (chart and report call trimmed) In Q4 2024, 80.9% of worldwide tests (104.3M) were in tiles with average latency under 20 ms. In the contiguous US the figure was 79.5% (19.6M tests). (answer and report trimmed) ``` Pausing is the right default between sessions that are minutes or hours apart. If you `stop()` the sandbox instead, memory is discarded: the `kernel` service starts again on the next boot with an empty kernel, and the agent reloads the data. The Python environment on the root filesystem and everything in `/workspace` are kept either way. ## Clean up A sandbox must be stopped before it can be deleted, and a paused sandbox can be stopped directly. Stop and delete the sandbox, then delete both disks. Deleting `ookla-speedtests` removes only the Archil disk, not the public bucket behind it. ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "code-interpreter")!; await sandbox.stop(); await sandbox.delete(); for (const mount of sandbox.mounts) { await (await archil.getDisk(mount.diskId)).delete(); } ``` ```python cleanup.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "code-interpreter") sandbox.stop() sandbox.delete() for mount in sandbox.mounts: archil.get_disk(mount.disk_id).delete() ``` ## Next steps > [card] [Coding agent](/guides/agents/coding-agent) > > Give an agent a bash tool and a persistent sandbox to build and run a web app. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, pausing, and network access in detail. > [card] [Sharing disks](/concepts/sharing-disks) > > Mount one disk in several sandboxes at once with read-only and conditional mounts. > [card] [Python SDK](/sdks/python) > > Full reference for sandboxes, disks, and `exec` options. # OpenAI Codex Source: https://docs.archil.com/guides/agents/codex Run the OpenAI Codex CLI headlessly in an Archil sandbox, with its workspace and session history on an Archil disk. [Codex](https://github.com/openai/codex) is OpenAI's coding agent CLI. In an Archil [sandbox](/compute/sandboxes/introduction) the whole Linux VM is the sandbox, so you can turn off Codex's own sandbox and approval prompts and let it work. Its workspace and session history live on an Archil disk, so the work outlives the sandbox, and a new sandbox can resume the same session. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [OpenAI API key](https://platform.openai.com/api-keys), then install the SDK: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export OPENAI_API_KEY=sk-... ``` ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` ## Run Codex in a sandbox This script creates a disk, starts a sandbox with the disk mounted at `/workspace`, installs Codex, and gives it a task with `codex exec`, which works until the task is done, prints the final answer, and exits. Then it deletes the sandbox; everything the agent made is on the disk. ```typescript start.ts theme={null} import * as archil from "disk"; const env = { CODEX_API_KEY: process.env.OPENAI_API_KEY!, // codex exec reads its key from CODEX_API_KEY CODEX_HOME: "/workspace/.codex", // keep sessions on the disk }; const { disk } = await archil.createDisk({ name: "codex-workspace" }); const sandbox = await archil.createSandbox({ name: "codex", mounts: [{ disk, path: "/workspace" }] }); await sandbox.exec("npm install -g @openai/codex && mkdir -p /workspace/.codex"); // < /dev/null: no more prompt text is coming on stdin. const result = await sandbox.exec( "codex exec --dangerously-bypass-approvals-and-sandbox " + '"Create a hello-world Express app with a test, and run the test. Reply in two sentences." < /dev/null', { cwd: "/workspace", env }, ); console.log(result.stdout); await sandbox.stop(); await sandbox.delete(); ``` ```python start.py theme={null} import os import archil env = { "CODEX_API_KEY": os.environ["OPENAI_API_KEY"], # codex exec reads its key from CODEX_API_KEY "CODEX_HOME": "/workspace/.codex", # keep sessions on the disk } disk = archil.create_disk(name="codex-workspace").disk sandbox = archil.create_sandbox( name="codex", mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")] ) sandbox.exec("npm install -g @openai/codex && mkdir -p /workspace/.codex") # < /dev/null: no more prompt text is coming on stdin. result = sandbox.exec( "codex exec --dangerously-bypass-approvals-and-sandbox " '"Create a hello-world Express app with a test, and run the test. Reply in two sentences." < /dev/null', cwd="/workspace", env=env, ) print(result.stdout) sandbox.stop() sandbox.delete() ``` Run it with `npx tsx start.ts` or `python start.py`. Codex prints its progress to standard error and only the final answer to standard output. The wording varies from run to run. One run printed: ```text theme={null} Created an Express app that returns “Hello, world!” at `/`; start it with `npm start`. Ran `npm test`: 1 test passed, 0 failed. ``` ## Pick up where you left off Codex's project and session history are on the disk, so any sandbox that mounts it can resume the session. This script starts a new sandbox on the same disk and runs `codex exec resume --last`, which resumes the most recent session in `/workspace`. ```typescript continue.ts theme={null} import * as archil from "disk"; const env = { CODEX_API_KEY: process.env.OPENAI_API_KEY!, CODEX_HOME: "/workspace/.codex", }; const [disk] = await archil.listDisks({ name: "codex-workspace" }); const sandbox = await archil.createSandbox({ name: "codex", mounts: [{ disk, path: "/workspace" }] }); await sandbox.exec("npm install -g @openai/codex"); const result = await sandbox.exec( "codex exec resume --last --dangerously-bypass-approvals-and-sandbox " + '"Add a /health route with a test and run all the tests. Reply in two sentences." < /dev/null', { cwd: "/workspace", env }, ); console.log(result.stdout); await sandbox.stop(); await sandbox.delete(); ``` ```python continue.py theme={null} import os import archil env = { "CODEX_API_KEY": os.environ["OPENAI_API_KEY"], "CODEX_HOME": "/workspace/.codex", } [disk] = archil.list_disks(name="codex-workspace") sandbox = archil.create_sandbox( name="codex", mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")] ) sandbox.exec("npm install -g @openai/codex") result = sandbox.exec( "codex exec resume --last --dangerously-bypass-approvals-and-sandbox " '"Add a /health route with a test and run all the tests. Reply in two sentences." < /dev/null', cwd="/workspace", env=env, ) print(result.stdout) sandbox.stop() sandbox.delete() ``` ```text theme={null} Added `/health`, returning HTTP 200 with `{"status":"ok"}`, and a test covering its response. Ran `npm test`: all 2 tests passed. ``` ## Going further * **Stream output.** Pass `onOutput: ({ data }) => process.stdout.write(data)` to `exec` (in Python, `on_output=lambda out: sys.stdout.buffer.write(out.data)`) to watch the agent as it works. Add `--json` to `codex exec` to get one JSON event per line. * **Work interactively.** Start `codex` in a pseudo-terminal with `sandbox.run` and sign in when it asks; see [Opening interactive shells](/compute/sandboxes/introduction#opening-interactive-shells). > [accordion] Use Vercel AI Gateway instead of an OpenAI key > > Codex can send its requests through [Vercel AI Gateway](https://vercel.com/docs/ai-gateway/coding-agents/openai-codex). Export `AI_GATEWAY_API_KEY` instead of `OPENAI_API_KEY`, replace the `CODEX_API_KEY` line in `env` with this one, and change `codex exec` to `codex exec -c openai_base_url=https://ai-gateway.vercel.sh/codex/v1` in both scripts. On the Free plan, sandboxes can't reach the gateway, because it isn't on the [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist); use your provider key instead. > > ```typescript TypeScript theme={null} > CODEX_API_KEY: process.env.AI_GATEWAY_API_KEY!, > ``` > > ```python Python theme={null} > "CODEX_API_KEY": os.environ["AI_GATEWAY_API_KEY"], > ``` ## Clean up Each script deletes its sandbox when it finishes, so only the disk is left. Delete it: ```typescript cleanup.ts theme={null} import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "codex-workspace" }); await disk.delete(); ``` ```python cleanup.py theme={null} import archil [disk] = archil.list_disks(name="codex-workspace") disk.delete() ``` ## Next steps > [card] [Claude Code](/guides/agents/claude-code) > > Run Claude Code headlessly in a sandbox. > [card] [OpenCode](/guides/agents/opencode) > > Run OpenCode headlessly in a sandbox. > [card] [Coding agent in a sandbox](/guides/agents/coding-agent) > > Build your own coding agent with the AI SDK instead. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. # Coding agent in a sandbox Source: https://docs.archil.com/guides/agents/coding-agent Give an AI agent a persistent Linux sandbox where it can install packages, write code, and run a web app you can open. In the [quickstart](/getting-started/quickstart), every `bash` call runs in a fresh microVM. That's ideal for working with files, but a coding agent wants a real machine: packages it installs should still be there on the next command, and the dev server it starts should keep running after the command returns. This guide gives an agent a [persistent sandbox](/compute/sandboxes/introduction) as its computer. The agent builds a small web app, runs it as a supervised service, and you open it at a public URL. Between sessions you pause the sandbox, and the next session picks up exactly where the last one left off, with the server still running. The project lives on an Archil disk mounted at `/workspace`, so the code outlives the sandbox. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [AI Gateway](https://vercel.com/ai-gateway) key for the model: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export AI_GATEWAY_API_KEY=... ``` Then install the dependencies. The TypeScript agent uses the [AI SDK](https://ai-sdk.dev); the Python agent calls the same model through the gateway's OpenAI-compatible API. ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk ai zod tsx ``` ```bash Python theme={null} pip install archil openai httpx ``` ## Define the agent The agent has one tool, `bash`, that runs a command in the sandbox with `sandbox.exec`. Because it's the same machine every time, `npm install` in one call is visible to the next, and the instructions tell the agent to run servers with `archil-sandbox services create` so they keep running in the background, restart if they crash, and come back if the sandbox reboots. ```typescript agent.ts theme={null} import { ToolLoopAgent, isStepCount, tool } from "ai"; import type { Sandbox } from "disk"; import { z } from "zod"; const instructions = `You are a coding agent with a persistent Linux sandbox (Debian, Node.js 24, root). Your project is in /workspace. Anything you install stays installed between commands. Run servers as services so they keep running after your command returns: archil-sandbox services create --tcp-port --dir /workspace -- After changing code, run: archil-sandbox services restart To debug a service, run: archil-sandbox services logs Servers must listen on 0.0.0.0. Keep your final answer to two sentences.`; export function createCodingAgent(sandbox: Sandbox) { const bash = tool({ description: "Run a bash command in /workspace. Returns the exit status and output.", inputSchema: z.object({ command: z.string() }), execute: async ({ command }) => { console.log(`\n$ ${command}`); // exec runs /bin/sh; passing the command through an env var hands it to bash unmodified const result = await sandbox.exec('bash -c "$COMMAND"', { cwd: "/workspace", env: { COMMAND: command }, timeoutSeconds: 120, onOutput: ({ data }) => process.stdout.write(data), }); const status = result.status === "timed_out" ? "timed out after 120s" : `exit ${result.exitCode}`; return `${status}\n${result.stdout}${result.stderr}`.slice(-10_000); }, }); return new ToolLoopAgent({ model: "anthropic/claude-sonnet-5.5", instructions, tools: { bash }, stopWhen: isStepCount(25), }); } ``` ```python agent.py theme={null} import json import os import sys from openai import OpenAI client = OpenAI( base_url="https://ai-gateway.vercel.sh/v1", api_key=os.environ["AI_GATEWAY_API_KEY"], ) INSTRUCTIONS = """You are a coding agent with a persistent Linux sandbox (Debian, Node.js 24, root). Your project is in /workspace. Anything you install stays installed between commands. Run servers as services so they keep running after your command returns: archil-sandbox services create --tcp-port --dir /workspace -- After changing code, run: archil-sandbox services restart To debug a service, run: archil-sandbox services logs Servers must listen on 0.0.0.0. Keep your final answer to two sentences.""" BASH_TOOL = { "type": "function", "function": { "name": "bash", "description": "Run a bash command in /workspace. Returns the exit status and output.", "parameters": { "type": "object", "properties": {"command": {"type": "string"}}, "required": ["command"], }, }, } def bash(sandbox, command): print(f"\n$ {command}") # exec runs /bin/sh; passing the command through an env var hands it to bash unmodified result = sandbox.exec( 'bash -c "$COMMAND"', cwd="/workspace", env={"COMMAND": command}, timeout_seconds=120, on_output=lambda out: sys.stdout.write(out.data.decode(errors="replace")), ) status = "timed out after 120s" if result.status == "timed_out" else f"exit {result.exit_code}" return f"{status}\n{result.stdout}{result.stderr}"[-10_000:] def run_agent(sandbox, prompt, max_steps=25): messages = [ {"role": "system", "content": INSTRUCTIONS}, {"role": "user", "content": prompt}, ] for _ in range(max_steps): reply = client.chat.completions.create( model="anthropic/claude-sonnet-5.5", messages=messages, tools=[BASH_TOOL], ).choices[0].message messages.append(reply) if not reply.tool_calls: return reply.content for call in reply.tool_calls: command = json.loads(call.function.arguments)["command"] messages.append( {"role": "tool", "tool_call_id": call.id, "content": bash(sandbox, command)} ) return f"Stopped after {max_steps} steps." ``` Two details in the tool keep long commands manageable: * **`onOutput` streams output as it's produced**, so you can watch a slow `npm install` or test run instead of waiting for it to finish. The full output is still returned when the command exits. * **`timeoutSeconds` bounds every command.** If the agent runs something that never exits, such as starting a server in the foreground, the command ends with status `timed_out` and the agent is told so, rather than hanging the loop. That's also why the instructions point it at services for anything long-running. ## Build the app The first session creates the workspace disk and the sandbox, then hands the agent its task. When the agent finishes, the script exposes port 3000 publicly and requests the app from your machine. Finally, it pauses the sandbox, which stops compute and snapshots memory so running processes, including the web server, resume intact. ```typescript session-1.ts theme={null} import * as archil from "disk"; import { createCodingAgent } from "./agent.ts"; const { disk } = await archil.createDisk({ name: "coding-agent-workspace" }); const sandbox = await archil.createSandbox({ name: "coding-agent", baseImage: "node:24", vcpuCount: 2, memSizeMiB: 4096, mounts: [{ disk, path: "/workspace" }], }); const agent = createCodingAgent(sandbox); const { text } = await agent.generate({ prompt: "Build a tiny Express app: GET / serves an HTML page with a Celsius-to-Fahrenheit form, " + 'and GET /api/convert?c=100 returns JSON like {"c":100,"f":212}. ' + "Run it as a service named web on port 3000, then check it with curl.", }); console.log(`\n${text}`); const hostname = await sandbox.exposePort(3000); const response = await fetch(`https://${hostname}/api/convert?c=100`); console.log(response.status, await response.text()); console.log(`Open https://${hostname}`); await sandbox.pause(); ``` ```python session-1.py theme={null} import archil import httpx from agent import run_agent disk = archil.create_disk(name="coding-agent-workspace").disk sandbox = archil.create_sandbox( name="coding-agent", base_image="node:24", vcpu_count=2, mem_size_mib=4096, mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")], ) text = run_agent( sandbox, "Build a tiny Express app: GET / serves an HTML page with a Celsius-to-Fahrenheit form, " 'and GET /api/convert?c=100 returns JSON like {"c":100,"f":212}. ' "Run it as a service named web on port 3000, then check it with curl.", ) print(f"\n{text}") hostname = sandbox.expose_port(3000) response = httpx.get(f"https://{hostname}/api/convert?c=100") print(response.status_code, response.text) print(f"Open https://{hostname}") sandbox.pause() ``` ```bash TypeScript theme={null} npx tsx session-1.ts ``` ```bash Python theme={null} python session-1.py ``` You'll see each command the agent runs, its streamed output, and then the response from your machine. The agent's steps vary from run to run; here is one run, with the long file and JSON output trimmed: ```text theme={null} $ cd /workspace && ls && npm init -y >/dev/null && npm i express 2>&1 | tail -2 found 0 vulnerabilities $ cd /workspace && cat > server.js <<'EOF' const express = require('express'); (output trimmed) archil-sandbox services create web --tcp-port 3000 --dir /workspace -- node server.js sleep 2 curl -s localhost:3000/ | head -5; curl -s "localhost:3000/api/convert?c=100"; echo; curl -s "localhost:3000/api/convert?c=abc" (output trimmed) {"c":100,"f":212} {"error":"Query parameter c must be a number"} The Express app is running as the `web` service on port 3000. I checked it with curl: `GET /` returns the HTML form, `/api/convert?c=100` returns `{"c":100,"f":212}`, and `c=abc` returns a 400 error. 200 {"c":100,"f":212} Open https://3000-agquucjuor64fnjugxbv2lektq.prod-aws-us-west-2-green.archil.app ``` Open the printed URL in your browser to try the form. The script has already paused the sandbox, but a request to an exposed port resumes a paused sandbox automatically, in about a second. The sandbox then keeps running until you pause or stop it again. > [!WARNING] > > `exposePort` makes the port reachable by anyone with the URL. Don't expose anything that handles real data without adding > its own authentication. For programmatic access, use a port token instead; see > [Hosting network services](/compute/sandboxes/introduction#hosting-network-services). ## Resume in a later session A paused sandbox keeps its memory, its root filesystem, and its mounts. When you come back, find it by name, resume it, and give the agent its next task. `resume()` does nothing if a request already woke the sandbox. Either way, the web server answers right away because the process never stopped; it was frozen with the rest of the VM. ```typescript session-2.ts theme={null} import * as archil from "disk"; import { createCodingAgent } from "./agent.ts"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "coding-agent")!; await sandbox.resume(); const [{ hostname }] = await sandbox.listPorts(); let response = await fetch(`https://${hostname}/api/convert?c=37`); console.log("after resume:", response.status, await response.text()); const agent = createCodingAgent(sandbox); const { text } = await agent.generate({ prompt: "In the app in /workspace, make /api/convert also accept f instead of c: " + 'GET /api/convert?f=212 returns {"f":212,"c":100}. Restart the web service and check it with curl.', }); console.log(`\n${text}`); response = await fetch(`https://${hostname}/api/convert?f=212`); console.log("after change:", response.status, await response.text()); await sandbox.pause(); ``` ```python session-2.py theme={null} import archil import httpx from agent import run_agent sandbox = next(s for s in archil.list_sandboxes() if s.name == "coding-agent") sandbox.resume() hostname = sandbox.list_ports()[0].hostname response = httpx.get(f"https://{hostname}/api/convert?c=37") print("after resume:", response.status_code, response.text) text = run_agent( sandbox, "In the app in /workspace, make /api/convert also accept f instead of c: " 'GET /api/convert?f=212 returns {"f":212,"c":100}. Restart the web service and check it with curl.', ) print(f"\n{text}") response = httpx.get(f"https://{hostname}/api/convert?f=212") print("after change:", response.status_code, response.text) sandbox.pause() ``` ```text theme={null} after resume: 200 {"c":37,"f":98.6} $ cd /workspace && ls && cat package.json && grep -rn "convert" --include=*.js --include=*.ts . --exclude-dir=node_modules | head; archil-sandbox services list (output trimmed) `/api/convert` now accepts `f` as well as `c`. After I restarted the web service, `curl 'localhost:3000/api/convert?f=212'` returned `{"f":212,"c":100}`. after change: 200 {"f":212,"c":100} ``` Pausing is the right default between agent sessions that are minutes or hours apart. If the sandbox will sit idle for longer, `stop()` it instead: memory is discarded, but the root filesystem and `/workspace` are kept, and the `web` service starts again on the next boot because services are supervised. Either way, the project files are on the Archil disk, so you can read them with the [disk APIs](/sdks/typescript) or mount the disk somewhere else even after the sandbox is gone. ## Clean up A sandbox must be stopped before it can be deleted, and a paused sandbox can be stopped directly. Stop and delete the sandbox, then delete the workspace disk. ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "coding-agent")!; const diskId = sandbox.mounts[0].diskId; await sandbox.stop(); await sandbox.delete(); await (await archil.getDisk(diskId)).delete(); ``` ```python cleanup.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "coding-agent") disk_id = sandbox.mounts[0].disk_id sandbox.stop() sandbox.delete() archil.get_disk(disk_id).delete() ``` ## Next steps > [card] [Parallel agents](/guides/agents/parallel-agents) > > Fork a prepared sandbox per agent, run them side by side, and keep the best result. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. > [card] [Sharing disks](/concepts/sharing-disks) > > Mount one disk in several sandboxes at once with read-only and conditional mounts. > [card] [TypeScript SDK](/sdks/typescript) > > Full reference for sandboxes, disks, and `exec` options. # OpenCode Source: https://docs.archil.com/guides/agents/opencode Run OpenCode headlessly in an Archil sandbox, with its workspace and session history on an Archil disk. [OpenCode](https://opencode.ai) is an open-source coding agent that works with many model providers. In an Archil [sandbox](/compute/sandboxes/introduction) it gets a Linux VM of its own, so you can let it work without approving each step. Its workspace and session history live on an Archil disk, so the work outlives the sandbox, and a new sandbox can continue the same session. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [Anthropic API key](https://console.anthropic.com/settings/keys), then install the SDK: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export ANTHROPIC_API_KEY=sk-ant-... ``` ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` ## Run OpenCode in a sandbox This script creates a disk, starts a sandbox with the disk mounted at `/workspace`, installs OpenCode, and gives it a task with `opencode run`, which works until the task is done, prints the result, and exits. Then it deletes the sandbox; everything the agent made is on the disk. ```typescript start.ts theme={null} import * as archil from "disk"; const env = { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, XDG_DATA_HOME: "/workspace/.local/share", // keep sessions on the disk }; const { disk } = await archil.createDisk({ name: "opencode-workspace" }); const sandbox = await archil.createSandbox({ name: "opencode", mounts: [{ disk, path: "/workspace" }] }); await sandbox.exec("npm install -g opencode-ai"); // --auto approves every permission request; < /dev/null: no more prompt text is coming on stdin. const result = await sandbox.exec( "opencode run --model anthropic/claude-sonnet-5 --auto " + '"Create a hello-world Express app with a test, and run the test. Reply in two sentences." < /dev/null', { cwd: "/workspace", env }, ); console.log(result.stdout); await sandbox.stop(); await sandbox.delete(); ``` ```python start.py theme={null} import os import archil env = { "ANTHROPIC_API_KEY": os.environ["ANTHROPIC_API_KEY"], "XDG_DATA_HOME": "/workspace/.local/share", # keep sessions on the disk } disk = archil.create_disk(name="opencode-workspace").disk sandbox = archil.create_sandbox( name="opencode", mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")] ) sandbox.exec("npm install -g opencode-ai") # --auto approves every permission request; < /dev/null: no more prompt text is coming on stdin. result = sandbox.exec( "opencode run --model anthropic/claude-sonnet-5 --auto " '"Create a hello-world Express app with a test, and run the test. Reply in two sentences." < /dev/null', cwd="/workspace", env=env, ) print(result.stdout) sandbox.stop() sandbox.delete() ``` Run it with `npx tsx start.ts` or `python start.py`. The agent's wording varies from run to run. One run printed: ```text theme={null} I created a hello-world Express app at `/workspace/hello-world-express` (app.js, server.js, app.test.js) with a Jest/Supertest test verifying `GET /` returns "Hello, World!". The test ran successfully: 1 passed, 1 total. ``` ## Pick up where you left off OpenCode's project and session history are on the disk, so any sandbox that mounts it can continue the session. This script starts a new sandbox on the same disk and runs `opencode run --continue`, which continues the most recent session. ```typescript continue.ts theme={null} import * as archil from "disk"; const env = { ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!, XDG_DATA_HOME: "/workspace/.local/share", }; const [disk] = await archil.listDisks({ name: "opencode-workspace" }); const sandbox = await archil.createSandbox({ name: "opencode", mounts: [{ disk, path: "/workspace" }] }); await sandbox.exec("npm install -g opencode-ai"); const result = await sandbox.exec( "opencode run --continue --model anthropic/claude-sonnet-5 --auto " + '"Add a /health route with a test and run all the tests. Reply in two sentences." < /dev/null', { cwd: "/workspace", env }, ); console.log(result.stdout); await sandbox.stop(); await sandbox.delete(); ``` ```python continue.py theme={null} import os import archil env = { "ANTHROPIC_API_KEY": os.environ["ANTHROPIC_API_KEY"], "XDG_DATA_HOME": "/workspace/.local/share", } [disk] = archil.list_disks(name="opencode-workspace") sandbox = archil.create_sandbox( name="opencode", mounts=[archil.SandboxMountSpec(disk=disk, path="/workspace")] ) sandbox.exec("npm install -g opencode-ai") result = sandbox.exec( "opencode run --continue --model anthropic/claude-sonnet-5 --auto " '"Add a /health route with a test and run all the tests. Reply in two sentences." < /dev/null', cwd="/workspace", env=env, ) print(result.stdout) sandbox.stop() sandbox.delete() ``` ```text theme={null} I added a `/health` route to app.js:9-11 that returns `{ status: 'ok' }` with a 200 status, along with a corresponding Jest test in app.test.js. All tests pass: 2 passed, 2 total. ``` ## Going further * **Stream output.** Pass `onOutput: ({ data }) => process.stdout.write(data)` to `exec` (in Python, `on_output=lambda out: sys.stdout.buffer.write(out.data)`) to watch the agent as it works. Add `--format json` to `opencode run` to get one JSON event per line. * **Work interactively.** Start `opencode` in a pseudo-terminal with `sandbox.run`; see [Opening interactive shells](/compute/sandboxes/introduction#opening-interactive-shells). * **Run the OpenCode server.** `opencode serve` runs OpenCode as an HTTP server that the OpenCode terminal UI (`opencode attach`) and the [OpenCode SDK](https://opencode.ai/docs/sdk/) connect to. Run it as a sandbox [service](/compute/sandboxes/introduction#hosting-network-services) so it keeps running and restarts with the sandbox. Set `OPENCODE_SERVER_PASSWORD` on the server, and reach it with a port token, or with a public port if you use `opencode attach`, which can't send the token header. > [accordion] Use Vercel AI Gateway instead of an Anthropic key > > OpenCode has a built-in [Vercel AI Gateway](https://vercel.com/docs/ai-gateway/coding-agents/opencode) provider, `vercel`, that reads `AI_GATEWAY_API_KEY`. Export `AI_GATEWAY_API_KEY` instead of `ANTHROPIC_API_KEY`, replace the `ANTHROPIC_API_KEY` line in `env` with this one, and change `--model anthropic/claude-sonnet-5` to `--model vercel/anthropic/claude-sonnet-5` in both scripts. On the Free plan, sandboxes can't reach the gateway, because it isn't on the [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist); use your provider key instead. > > ```typescript TypeScript theme={null} > AI_GATEWAY_API_KEY: process.env.AI_GATEWAY_API_KEY!, > ``` > > ```python Python theme={null} > "AI_GATEWAY_API_KEY": os.environ["AI_GATEWAY_API_KEY"], > ``` ## Clean up Each script deletes its sandbox when it finishes, so only the disk is left. Delete it: ```typescript cleanup.ts theme={null} import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "opencode-workspace" }); await disk.delete(); ``` ```python cleanup.py theme={null} import archil [disk] = archil.list_disks(name="opencode-workspace") disk.delete() ``` ## Next steps > [card] [Claude Code](/guides/agents/claude-code) > > Run Claude Code headlessly in a sandbox. > [card] [OpenAI Codex](/guides/agents/codex) > > Run the Codex CLI headlessly in a sandbox. > [card] [Coding agent in a sandbox](/guides/agents/coding-agent) > > Build your own coding agent with the AI SDK instead. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. # Parallel agents Source: https://docs.archil.com/guides/agents/parallel-agents Fork a prepared sandbox once per agent, let each agent try a different approach, and keep the fork with the best score. Agents are nondeterministic: give the same task to three agents and you get three different solutions, some better than others. Instead of hoping one run gets it right, you can run several attempts side by side and keep the best one. This guide does that with [sandbox forks](/compute/sandboxes/introduction#forking-a-sandbox). You prepare a base sandbox once, with the project checked out and its tools installed. Then you fork it three times and run an agent in each fork, each with a different hint or model. Each fork is a full, independent copy of the machine, so the agents can install packages and rewrite files without seeing each other's work. Finally, you score every fork with the same benchmark, keep the winner, and delete the rest. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [AI Gateway](https://vercel.com/ai-gateway) key for the model: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export AI_GATEWAY_API_KEY=... ``` Install the dependencies. The TypeScript agents use the [AI SDK](https://ai-sdk.dev); the Python agents call the same model through the gateway's OpenAI-compatible API. ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk ai zod tsx ``` ```bash Python theme={null} pip install archil openai ``` ## Create the project The agents' task is to speed up a slow function without breaking its tests. Create these three files in a local `project` directory: the function, its tests, and a benchmark that prints the runtime in milliseconds. ```python project/primes.py theme={null} def count_primes(n): """Return the number of primes less than n.""" count = 0 for k in range(2, n): if all(k % d for d in range(2, int(k**0.5) + 1)): count += 1 return count ``` ```python project/test_primes.py theme={null} from primes import count_primes def test_small(): assert [count_primes(n) for n in range(12)] == [0, 0, 0, 1, 2, 2, 3, 3, 4, 4, 4, 4] def test_large(): assert count_primes(10_000) == 1229 assert count_primes(100_000) == 9592 ``` ```python project/bench.py theme={null} import timeit from primes import count_primes assert count_primes(1_000_000) == 78498 print(round(min(timeit.repeat(lambda: count_primes(1_000_000), number=1, repeat=3)) * 1000, 2)) ``` ## Prepare the base sandbox The base sandbox gets the project in `/app`, installs `pytest`, and commits everything to git, so you can diff each agent's changes later. It also records a baseline benchmark. The project lives on the sandbox's root filesystem rather than on an Archil disk, because the root filesystem is copied into every fork. Once it's ready, the base is stopped: it waits as a template without using compute, and forks of it boot as running sandboxes. ```typescript prepare.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const base = await archil.createSandbox({ name: "parallel-agents-base", baseImage: "python:3.13", vcpuCount: 2, memSizeMiB: 4096, }); for (const file of ["primes.py", "test_primes.py", "bench.py"]) { await base.files.uploadFile(await readFile(`project/${file}`), `/app/${file}`); } const setup = await base.exec( "pip install -q --root-user-action=ignore pytest && git init -q && git add -A && " + "git -c user.name=base -c user.email=base@example.com commit -qm baseline && python bench.py", { cwd: "/app" }, ); if (setup.exitCode !== 0) throw new Error(setup.stderr); console.log(`Baseline: ${setup.stdout.trim()} ms`); // The base is only a template from here on; forks can be made while it's stopped await base.stop(); ``` ```python prepare.py theme={null} import archil base = archil.create_sandbox( name="parallel-agents-base", base_image="python:3.13", vcpu_count=2, mem_size_mib=4096, ) for file in ["primes.py", "test_primes.py", "bench.py"]: base.files.upload_file(f"project/{file}", f"/app/{file}") setup = base.exec( "pip install -q --root-user-action=ignore pytest && git init -q && git add -A && " "git -c user.name=base -c user.email=base@example.com commit -qm baseline && python bench.py", cwd="/app", ) if setup.exit_code != 0: raise RuntimeError(setup.stderr) print(f"Baseline: {setup.stdout.strip()} ms") # The base is only a template from here on; forks can be made while it's stopped base.stop() ``` ```bash TypeScript theme={null} npx tsx prepare.ts ``` ```bash Python theme={null} python prepare.py ``` ```text theme={null} Baseline: 2295.34 ms ``` ## Fork, race, and keep the winner The race script forks the base once per approach and runs an agent in each fork at the same time. Every agent gets the same task plus a different hint, two of them run on Claude Sonnet and one on Claude Haiku, and each has a `bash` tool bound to its own fork. When an agent finishes, the script scores its fork: the tests and benchmark must be untouched (`git diff --quiet`), the tests must pass, and the score is the benchmark time. The fastest fork wins; the others are stopped and deleted. ```typescript race.ts theme={null} import { ToolLoopAgent, isStepCount, tool } from "ai"; import * as archil from "disk"; import type { Sandbox } from "disk"; import { z } from "zod"; const approaches = [ { name: "parallel-agents-1", model: "anthropic/claude-sonnet-5.5", hint: "Use only the standard library." }, { name: "parallel-agents-2", model: "anthropic/claude-sonnet-5.5", hint: "You may pip install numpy." }, { name: "parallel-agents-3", model: "anthropic/claude-haiku-5.5", hint: "Keep the change small." }, ]; const task = "Make count_primes in /app/primes.py as fast as you can. `python bench.py` prints its runtime in ms, " + "and `python -m pytest -q` must keep passing. Don't edit test_primes.py or bench.py. " + "Keep your final answer to one sentence."; function bashTool(sandbox: Sandbox) { return tool({ description: "Run a bash command in /app. Returns the exit status and output.", inputSchema: z.object({ command: z.string() }), execute: async ({ command }) => { console.log(`[${sandbox.name}] $ ${command.split("\n")[0].slice(0, 80)}`); const result = await sandbox.exec('bash -c "$COMMAND"', { cwd: "/app", env: { COMMAND: command }, timeoutSeconds: 120, }); return `exit ${result.exitCode ?? result.status}\n${result.stdout}${result.stderr}`.slice(-10_000); }, }); } async function score(sandbox: Sandbox) { const result = await sandbox.exec( "git diff --quiet HEAD -- test_primes.py bench.py && python -m pytest -q >/dev/null && python bench.py", { cwd: "/app" }, ); return result.exitCode === 0 ? Number(result.stdout.trim()) : Infinity; } const base = (await archil.listSandboxes()).find((s) => s.name === "parallel-agents-base")!; const forks = await Promise.all(approaches.map(({ name }) => base.fork({ name }))); const results = await Promise.all( forks.map(async (sandbox, i) => { const agent = new ToolLoopAgent({ model: approaches[i].model, tools: { bash: bashTool(sandbox) }, stopWhen: isStepCount(15), }); const { text } = await agent.generate({ prompt: `${task} ${approaches[i].hint}` }); const ms = await score(sandbox); console.log(`${sandbox.name}: ${ms} ms. ${text}`); return { sandbox, ms }; }), ); results.sort((a, b) => a.ms - b.ms); const [winner, ...losers] = results; for (const { sandbox } of losers) { await sandbox.stop(); await sandbox.delete(); } console.log(`\nWinner: ${winner.sandbox.name} (${winner.ms} ms)`); console.log((await winner.sandbox.exec("git diff --stat", { cwd: "/app" })).stdout); ``` ```python race.py theme={null} import json import os from concurrent.futures import ThreadPoolExecutor import archil from openai import OpenAI client = OpenAI( base_url="https://ai-gateway.vercel.sh/v1", api_key=os.environ["AI_GATEWAY_API_KEY"], ) APPROACHES = [ {"name": "parallel-agents-1", "model": "anthropic/claude-sonnet-5.5", "hint": "Use only the standard library."}, {"name": "parallel-agents-2", "model": "anthropic/claude-sonnet-5.5", "hint": "You may pip install numpy."}, {"name": "parallel-agents-3", "model": "anthropic/claude-haiku-5.5", "hint": "Keep the change small."}, ] TASK = ( "Make count_primes in /app/primes.py as fast as you can. `python bench.py` prints its runtime in ms, " "and `python -m pytest -q` must keep passing. Don't edit test_primes.py or bench.py. " "Keep your final answer to one sentence." ) BASH_TOOL = { "type": "function", "function": { "name": "bash", "description": "Run a bash command in /app. Returns the exit status and output.", "parameters": { "type": "object", "properties": {"command": {"type": "string"}}, "required": ["command"], }, }, } def bash(sandbox, command): print(f"[{sandbox.name}] $ {command.splitlines()[0][:80]}") result = sandbox.exec( 'bash -c "$COMMAND"', cwd="/app", env={"COMMAND": command}, timeout_seconds=120 ) status = result.status if result.exit_code is None else result.exit_code return f"exit {status}\n{result.stdout}{result.stderr}"[-10_000:] def run_agent(sandbox, prompt, model, max_steps=15): messages = [{"role": "user", "content": prompt}] for _ in range(max_steps): reply = client.chat.completions.create( model=model, messages=messages, tools=[BASH_TOOL], ).choices[0].message messages.append(reply) if not reply.tool_calls: return reply.content for call in reply.tool_calls: command = json.loads(call.function.arguments)["command"] messages.append( {"role": "tool", "tool_call_id": call.id, "content": bash(sandbox, command)} ) return f"Stopped after {max_steps} steps." def score(sandbox): result = sandbox.exec( "git diff --quiet HEAD -- test_primes.py bench.py && python -m pytest -q >/dev/null && python bench.py", cwd="/app", ) return float(result.stdout) if result.exit_code == 0 else float("inf") def attempt(sandbox, approach): text = run_agent(sandbox, f"{TASK} {approach['hint']}", approach["model"]) ms = score(sandbox) print(f"{sandbox.name}: {ms} ms. {text}") return sandbox, ms base = next(s for s in archil.list_sandboxes() if s.name == "parallel-agents-base") with ThreadPoolExecutor() as pool: forks = list(pool.map(lambda approach: base.fork(name=approach["name"]), APPROACHES)) results = list(pool.map(attempt, forks, APPROACHES)) results.sort(key=lambda result: result[1]) (winner, best_ms), *losers = results for sandbox, _ in losers: sandbox.stop() sandbox.delete() print(f"\nWinner: {winner.name} ({best_ms} ms)") print(winner.exec("git diff --stat", cwd="/app").stdout) ``` ```bash TypeScript theme={null} npx tsx race.ts ``` ```bash Python theme={null} python race.py ``` The agents' commands interleave as they work in parallel, and the whole race usually takes well under a minute. Results vary from run to run; in this one, the fork that was allowed to install numpy won: ```text theme={null} [parallel-agents-3] $ cd /app && cat primes.py && echo --- && cat bench.py && echo --- && cat test_pri [parallel-agents-1] $ cd /app; ls; cat primes.py bench.py test_primes.py; python --version [parallel-agents-2] $ cd /app; ls; cat primes.py bench.py test_primes.py; python -V; python -c "import [parallel-agents-1] $ cd /app; cat > primes.py <<'E' [parallel-agents-1] $ cd /app; python -c " [parallel-agents-2] $ cd /app; pip install numpy 2>&1 | tail -1 [parallel-agents-1] $ cd /app; cat > primes.py <<'E' [parallel-agents-3] $ cd /app && cat > primes.py <<'EOF' parallel-agents-1: 1.42 ms. I rewrote `count_primes` in `/app/primes.py` as an odd-only bytearray sieve, which cuts the `bench.py` time to about 1.4 ms, and `python -m pytest -q` passes. I also checked it against the original trial-division method for every n below 3000, and the counts match. parallel-agents-3: 3.66 ms. I replaced the trial-division loop in `/app/primes.py` with a bytearray Sieve of Eratosthenes, cutting the benchmark from about 2274 ms to 3.69 ms while `python -m pytest -q` still passes. parallel-agents-2: 0.41 ms. I replaced `count_primes` in `/app/primes.py` with a NumPy sieve over odd numbers only. `bench.py` now reports about 0.41 ms, and `python -m pytest -q` passes (2 tests). I also spot-checked values just below and above a few primes, and they were correct. Winner: parallel-agents-2 (0.41 ms) primes.py | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) ``` The winning fork is still running, with the agent's change applied and anything it installed still in place, so you can keep working in it: run more tests, push the change from it, or hand it to the next agent. The base is untouched, so you can fork it again for the next task without repeating the setup. ## Clean up A sandbox can't be deleted while forks of it still exist; the API rejects the request with `sandbox has dependent forks`. Delete the winning fork first, then the base. ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandboxes = (await archil.listSandboxes()).filter((s) => s.name.startsWith("parallel-agents-")); const base = sandboxes.find((s) => s.name === "parallel-agents-base")!; const forks = sandboxes.filter((s) => s !== base); for (const sandbox of [...forks, base]) { await sandbox.stop(); await sandbox.delete(); } ``` ```python cleanup.py theme={null} import archil sandboxes = [s for s in archil.list_sandboxes() if s.name.startswith("parallel-agents-")] base = next(s for s in sandboxes if s.name == "parallel-agents-base") forks = [s for s in sandboxes if s is not base] for sandbox in [*forks, base]: sandbox.stop() sandbox.delete() ``` ## Next steps > [card] [Coding agent in a sandbox](/guides/agents/coding-agent) > > Give one agent a persistent machine, run its app as a service, and pause it between sessions. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. > [card] [Branches and checkpoints](/concepts/branches-and-checkpoints) > > Fork an Archil disk instead of a whole machine. > [card] [Python SDK](/sdks/python) > > Full reference for sandboxes, disks, and `exec` options. # Vector search with LanceDB Source: https://docs.archil.com/guides/agents/vector-search Store a LanceDB table on an Archil disk, serve vector, filtered, and hybrid search from read-only replicas, and update it in place. This guide builds a search API over public-domain books with [LanceDB](https://lancedb.com), an embedded vector database that stores tables as files, so there's no database server to run. An indexer in a [persistent sandbox](/compute/sandboxes/introduction) writes the table to an Archil disk. Search replicas mount the same disk read-only and answer vector, filtered, and hybrid (vector plus full-text) queries over HTTPS. When you add or delete documents, the indexer updates the table in place and every replica serves the new version without a restart. Embeddings come from `all-MiniLM-L6-v2`, a small open model that runs on CPU, so no model-provider API key is needed. ## Prerequisites Export your Archil API key and a region that offers sandboxes (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and install the SDK for the language you orchestrate from. You'll also need `curl` and `jq`. ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 npm init -y && npm pkg set type=module && npm install disk tsx # or: pip install archil ``` > [!NOTE] > > The indexer sandbox downloads books from Project Gutenberg and the embedding model from Hugging Face. Sandboxes on the > Free plan can reach only a [managed allowlist](/compute/sandboxes/firewall#free-plan-allowlist) that doesn't > include those hosts, so this guide needs the Developer plan or above. ## Write the ingest job This script runs inside the indexer sandbox. `add` downloads books from Project Gutenberg, splits them into paragraphs, embeds each one, and writes them to the `books` table. The first `add` creates the table with a vector index and a full-text index; later ones append rows and call `optimize()`, which folds the new rows into both indexes without rebuilding them. `delete` removes a book's rows. ```python ingest.py theme={null} import json, os, re, urllib.request import lancedb from fastembed import TextEmbedding from lancedb.index import FTS, IvfHnswSq db = lancedb.connect("/mnt/archil/lancedb") command, *args = json.loads(os.environ["ARGS"]) if command == "add": model = TextEmbedding("sentence-transformers/all-MiniLM-L6-v2") rows = [] for book in args: text = urllib.request.urlopen(f"https://www.gutenberg.org/cache/epub/{book}/pg{book}.txt").read().decode("utf-8-sig") title = re.search(r"Title: (.+)", text).group(1).strip() for para in re.split(r"\n\s*\n", text.split("*** START OF")[1].split("*** END OF")[0]): para = " ".join(para.split()) if len(para) >= 200: rows.append({"book": title, "text": para}) for row, vector in zip(rows, model.embed([r["text"] for r in rows], batch_size=64)): row["vector"] = vector if "books" in db.list_tables().tables: table = db.open_table("books") table.add(rows) table.optimize() else: table = db.create_table("books", rows) table.create_index("vector", config=IvfHnswSq(distance_type="cosine")) table.create_index("text", config=FTS()) elif command == "delete": table = db.open_table("books") table.delete("book = '{}'".format(args[0].replace("'", "''"))) print(f"{table.count_rows()} paragraphs, version {table.version}") ``` ## Run the indexer This script creates the disk and a long-lived indexer sandbox the first time, reuses them after that, and runs `ingest.py` with whatever arguments you pass. The indexer mounts the disk in [conditional mode](/concepts/sharing-disks), which sends writes to the server and takes no exclusive lock on the disk, so search replicas can stay attached read-only while it writes. ```typescript indexer.ts theme={null} import * as archil from "disk"; import { readFile } from "node:fs/promises"; const [existing] = await archil.listDisks({ name: "vector-index" }); const disk = existing ?? (await archil.createDisk({ name: "vector-index" })).disk; const indexer = (await archil.listSandboxes()).find((s) => s.name === "vector-indexer") ?? (await archil.createSandbox({ name: "vector-indexer", baseImage: "python:3.13", vcpuCount: 4, memSizeMiB: 8192, mounts: [{ disk, path: "/mnt/archil", conditional: true }], })); await indexer.files.uploadFile(await readFile("ingest.py"), "/opt/search/ingest.py"); const job = await indexer.exec( "pip install -q --root-user-action=ignore lancedb==0.40.0 fastembed && python3 /opt/search/ingest.py", { env: { ARGS: JSON.stringify(process.argv.slice(2)) }, timeoutSeconds: 900 }, ); if (job.exitCode !== 0) throw new Error(job.stderr); console.log(job.stdout.trim()); ``` ```python indexer.py theme={null} import json, sys import archil existing = archil.list_disks(name="vector-index") disk = existing[0] if existing else archil.create_disk(name="vector-index").disk indexer = next((s for s in archil.list_sandboxes() if s.name == "vector-indexer"), None) or archil.create_sandbox( name="vector-indexer", base_image="python:3.13", vcpu_count=4, mem_size_mib=8192, mounts=[archil.SandboxMountSpec(disk=disk, path="/mnt/archil", conditional=True)], ) indexer.files.upload_file("ingest.py", "/opt/search/ingest.py") job = indexer.exec( "pip install -q --root-user-action=ignore lancedb==0.40.0 fastembed && python3 /opt/search/ingest.py", env={"ARGS": json.dumps(sys.argv[1:])}, timeout_seconds=900, ) if job.exit_code != 0: raise RuntimeError(job.stderr) print(job.stdout.strip()) ``` Load five books: Sherlock Holmes, Pride and Prejudice, Frankenstein, Moby Dick, and Alice in Wonderland. The first run takes about a minute, most of it embedding. ```bash theme={null} npx tsx indexer.ts add 1661 1342 84 2701 11 # or: python indexer.py add 1661 1342 84 2701 11 ``` ``` 4396 paragraphs, version 3 ``` ## Write the search server Each replica runs this server. `GET /search?q=...` runs a vector search; add `book=...` to filter by title, or `mode=hybrid` to combine vector and full-text ranking. A background thread follows the table as the indexer commits. LanceDB records the latest version in `latest_version_hint.json`; every second the server re-reads just that file from the server with [`archil invalidate-cache`](/reference/archil-cli#invalidate-cache), and when it moves, refreshes the mount's cached lookups and checks out the new version. Requests keep running against the old version until the switch. ```python search_server.py theme={null} import json, subprocess, threading, time from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from urllib.parse import parse_qs, urlparse import lancedb from fastembed import TextEmbedding ROOT = "/mnt/archil/lancedb" HINT = f"{ROOT}/books.lance/_versions/latest_version_hint.json" model = TextEmbedding("sentence-transformers/all-MiniLM-L6-v2") table = lancedb.connect(ROOT).open_table("books") def follow(): while True: time.sleep(1) try: subprocess.run(["archil", "invalidate-cache", "-q", HINT], check=True, capture_output=True) with open(HINT) as f: latest = json.load(f)["version"] if latest > table.version: subprocess.run(["archil", "invalidate-cache", "-q", "/mnt/archil"], check=True, capture_output=True) table.checkout_latest() print(f"serving version {table.version}", flush=True) except Exception as e: print(f"still serving version {table.version}: {e}", flush=True) class Handler(BaseHTTPRequestHandler): def do_GET(self): # GET /search?q=...[&book=...][&mode=hybrid] params = {k: v[0] for k, v in parse_qs(urlparse(self.path).query).items()} start, version = time.perf_counter(), table.version vector = list(model.embed([params.get("q", "")]))[0] if params.get("mode") == "hybrid": # results include _relevance_score query, columns = table.search(query_type="hybrid").vector(vector).text(params.get("q", "")), ["book", "text"] else: query, columns = table.search(vector), ["book", "text", "_distance"] if "book" in params: query = query.where("book = '{}'".format(params["book"].replace("'", "''"))) results = query.limit(3).select(columns).to_list() took_ms = round((time.perf_counter() - start) * 1000, 1) body = json.dumps({"version": version, "took_ms": took_ms, "results": results}).encode() self.send_response(200) self.send_header("Content-Type", "application/json") self.end_headers() self.wfile.write(body) threading.Thread(target=follow, daemon=True).start() ThreadingHTTPServer(("0.0.0.0", 8080), Handler).serve_forever() ``` ## Deploy a search replica This script creates a sandbox that mounts the disk with `readOnly`, runs the server as a supervised service (restarted on failure and after reboots), and waits until it answers. The port stays private: the script creates a port token, which callers send in the `X-Archil-Token` header, and appends the replica's URL and token to `replicas.txt`. ```typescript deploy.ts theme={null} import * as archil from "disk"; import { appendFile, readFile } from "node:fs/promises"; const [disk] = await archil.listDisks({ name: "vector-index" }); const replica = await archil.createSandbox({ name: process.argv[2], baseImage: "python:3.13", vcpuCount: 2, memSizeMiB: 4096, mounts: [{ disk, path: "/mnt/archil", readOnly: true }], }); await replica.files.uploadFile(await readFile("search_server.py"), "/opt/search/search_server.py"); const setup = await replica.exec( "pip install -q --root-user-action=ignore lancedb==0.40.0 fastembed && " + "archil-sandbox services create search --tcp-port 8080 -- python3 /opt/search/search_server.py && " + "timeout 120 sh -c 'until curl -s localhost:8080; do sleep 1; done'", { timeoutSeconds: 600 }, ); if (setup.exitCode !== 0) throw new Error(setup.stderr); const { hostname, token } = await replica.createPortToken(8080, { ttl: "24h" }); await appendFile("replicas.txt", `https://${hostname} ${token}\n`); ``` ```python deploy.py theme={null} import sys import archil disk = archil.list_disks(name="vector-index")[0] replica = archil.create_sandbox( name=sys.argv[1], base_image="python:3.13", vcpu_count=2, mem_size_mib=4096, mounts=[archil.SandboxMountSpec(disk=disk, path="/mnt/archil", read_only=True)], ) replica.files.upload_file("search_server.py", "/opt/search/search_server.py") setup = replica.exec( "pip install -q --root-user-action=ignore lancedb==0.40.0 fastembed && " "archil-sandbox services create search --tcp-port 8080 -- python3 /opt/search/search_server.py && " "timeout 120 sh -c 'until curl -s localhost:8080; do sleep 1; done'", timeout_seconds=600, ) if setup.exit_code != 0: raise RuntimeError(setup.stderr) access = replica.create_port_token(8080, ttl="24h") with open("replicas.txt", "a") as f: f.write(f"https://{access.hostname} {access.token}\n") ``` Deploy the first replica and query it from your machine. The second query narrows the search to one book, and the third ranks by both meaning and the literal words: ```bash theme={null} npx tsx deploy.ts vector-search-1 # or: python deploy.py vector-search-1 read -r URL TOKEN < replicas.txt search() { curl -s -G "$URL/search" -H "X-Archil-Token: $TOKEN" --data-urlencode "q=$1" --data-urlencode "$2" | jq -c '.results[] | [.book[:24], .text[:60]]'; } search "a detective deduces from a stranger's appearance" "mode=vector" search "a detective deduces from a stranger's appearance" "book=Pride and Prejudice" search "the white whale" "mode=hybrid" ``` ``` ["The Adventures of Sherlo","“As a rule,” said Holmes, “the more bizarre a thing is the l"] ["The Adventures of Sherlo","“The case has been an interesting one,” remarked Holmes when"] ["The Adventures of Sherlo","“Had there been women in the house, I should have suspected "] ["Pride and Prejudice","But the attention of every lady was soon caught by a young m"] ["Pride and Prejudice","Mrs. Philips was always glad to see her nieces; and the two "] ["Pride and Prejudice","Elizabeth had settled it that Mr. Darcy would bring his sist"] ["Moby Dick; Or, The Whale","In the distance, a great white mass lazily rose, and rising "] ["Moby Dick; Or, The Whale","“It was the first time in my life that I ever cruised on the"] ["Moby Dick; Or, The Whale","Receiving the top-maul from Starbuck, he advanced towards th"] ``` The vector index is approximate, so your exact results can differ slightly. Without the `X-Archil-Token` header, the same request gets a `401`. ## Scale out Read-only mounts take no lock, so any number of replicas can attach to the disk alongside the indexer. Add a second replica, then send a query to both: ```bash theme={null} npx tsx deploy.ts vector-search-2 # or: python deploy.py vector-search-2 while read -r URL TOKEN; do curl -s -G "$URL/search" -H "X-Archil-Token: $TOKEN" --data-urlencode "q=wolves howling in the mountains at night" \ | jq -c '[.version, .results[0].book]' done < replicas.txt ``` ``` [3,"Frankenstein; or, the modern prometheus"] [3,"Frankenstein; or, the modern prometheus"] ``` ## Add and delete documents Dracula isn't in the table yet, so that query has no good answer. Add it while both replicas keep serving. Only the new paragraphs are embedded and indexed: ```bash theme={null} npx tsx indexer.ts add 345 # or: python indexer.py add 345 ``` ``` 5457 paragraphs, version 5 ``` Run the same loop again, and both replicas answer from the new version: ``` [5,"Dracula"] [5,"Dracula"] ``` Deleting works the same way. Remove Alice in Wonderland, give the replicas a moment to pick up the new version, then search each one with a filter on that book: ```bash theme={null} npx tsx indexer.ts delete "Alice's Adventures in Wonderland" # or: python indexer.py delete "Alice's Adventures in Wonderland" sleep 2 while read -r URL TOKEN; do curl -s -G "$URL/search" -H "X-Archil-Token: $TOKEN" --data-urlencode "q=a rabbit with a pocket watch" \ --data-urlencode "book=Alice's Adventures in Wonderland" | jq -c '[.version, (.results | length)]' done < replicas.txt ``` ``` 5218 paragraphs, version 6 [6,0] [6,0] ``` To change rows in place, use `table.update()`, or `table.merge_insert()` for upserts keyed on a column. Each commit is a new table version, which the replicas pick up the same way. ## Clean up Stop and delete every sandbox that mounts the disk, then delete the disk. ```typescript cleanup.ts theme={null} import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "vector-index" }); for (const sandbox of await archil.listSandboxes({ disk })) { if (sandbox.status !== "stopped") await sandbox.stop(); await sandbox.delete(); } await disk.delete(); ``` ```python cleanup.py theme={null} import archil disk = archil.list_disks(name="vector-index")[0] for sandbox in archil.list_sandboxes(disk=disk): if sandbox.status != "stopped": sandbox.stop() sandbox.delete() disk.delete() ``` ## Next steps * [LanceDB documentation](https://lancedb.com/docs): index types and tuning for larger tables, reranking, and schema evolution. * [Sharing disks](/concepts/sharing-disks): how exclusive, conditional, and read-only mounts interact. * [Persistent sandboxes](/compute/sandboxes/introduction): services, port tokens, and the sandbox lifecycle. # ClickHouse Source: https://docs.archil.com/guides/analytics/clickhouse Run open-source ClickHouse with its data directory on an Archil disk, move it to a new server by remounting, and give heavy queries their own copy of the database with a branch. This guide runs open-source [ClickHouse](https://clickhouse.com/) on Linux servers with its whole data directory on an Archil disk. ClickHouse keeps using ordinary `MergeTree` tables on what looks like a local file system, and Archil keeps the files durable in object storage. That gives a self-hosted ClickHouse three properties it doesn't have on local disks: * **One copy of the data, in object storage.** Archil makes a write durable across Availability Zones as soon as `fsync` returns, so you don't need ClickHouse replicas to keep the data safe. * **Replaceable servers.** No table data or metadata lives on the server. When a server dies, another one mounts the disk and starts ClickHouse with every table intact. * **Isolated copies for heavy queries.** A [branch](/concepts/branches-and-checkpoints) gives you a writable copy of the whole database in about 50 ms (a 15 ms checkpoint plus a 35 ms branch), whatever its size. Run a long modeling query or a backfill on a second server against the branch, and it never competes with ingest on the primary. ClickHouse can also store parts in S3 with an `s3` disk, but the server then keeps the metadata for those parts on its local disk, and sharing one copy of the data between replicas relies on zero-copy replication, which ClickHouse doesn't recommend for production. With the data directory on Archil, nothing in ClickHouse needs to know about object storage. You'll set up a primary server, load 100 million rows, move the database to a second server, and then run a heavy query on a branch while the primary keeps taking inserts. ## Prerequisites * **Two Linux servers** running Ubuntu 22.04 or later, x86-64 or arm64, in the same cloud region as your disk. This guide calls them `ch-primary` and `ch-analytics`. Any instance with 4 vCPUs and 16 GiB of memory is enough for the examples; we used EC2 `m7i.xlarge` instances in `us-east-1`. Both need [outbound access to Archil](/mounting/linux#requirements). * **An Archil disk** for the database, created in the [Archil console](https://console.archil.com). Branches need Archil-managed storage or a flat-block bring-your-own-bucket disk; see [Branches & Checkpoints](/concepts/branches-and-checkpoints). * **A disk token** for that disk, which both servers use to mount it. In the console, open the disk, select **Connections**, click **Add User**, and choose **Disk Token**. * **An API key** in the disk's region, for creating branches. On both servers, export the disk's name, its ID, its region, and the disk token: ```bash theme={null} export ARCHIL_DISK=myorg/clickhouse # owner-qualified name, or the disk ID export ARCHIL_DISK_ID=dsk-0123456789abcdef # the disk ID, for branch commands export ARCHIL_REGION=aws-us-east-1 export ARCHIL_MOUNT_TOKEN=... ``` ## Install ClickHouse and the Archil client Run this on **both** servers. It installs the Archil client and ClickHouse from ClickHouse's official package repository, then stops ClickHouse, which still points at its default data directory on the local disk: ```bash install.sh theme={null} set -euo pipefail curl -s https://archil.com/install | sh sudo apt-get update sudo apt-get install -y apt-transport-https ca-certificates curl gnupg curl -fsSL https://packages.clickhouse.com/rpm/lts/repodata/repomd.xml.key \ | sudo gpg --dearmor -o /usr/share/keyrings/clickhouse-keyring.gpg echo "deb [signed-by=/usr/share/keyrings/clickhouse-keyring.gpg arch=$(dpkg --print-architecture)] https://packages.clickhouse.com/deb stable main" \ | sudo tee /etc/apt/sources.list.d/clickhouse.list sudo apt-get update sudo DEBIAN_FRONTEND=noninteractive apt-get install -y clickhouse-server clickhouse-client sudo systemctl stop clickhouse-server ``` ## Point ClickHouse at the Archil disk Still on **both** servers, add a configuration file that moves everything ClickHouse needs to restart onto the disk. The `path` setting covers table data, table and database definitions, and ClickHouse's own state. Users and grants created with SQL are stored separately, so `user_directories` moves them too. `tmp_path`, which ClickHouse uses for large sorts and joins that spill out of memory, stays on the local disk: it's scratch space, and it's faster there. ```xml /etc/clickhouse-server/config.d/archil.xml theme={null} /mnt/archil/clickhouse/ /mnt/archil/clickhouse/user_files/ /mnt/archil/clickhouse/format_schemas/ /mnt/archil/clickhouse/access/ /var/lib/clickhouse/tmp/ 1 1 ``` By default, ClickHouse doesn't `fsync` the parts it writes, so an `INSERT` can return before its data is durable. On an Archil disk, writes that haven't been flushed are held by the client on that server, and they're lost if the server dies. The two `merge_tree` settings make every `INSERT` wait until its part is durable, so a server failure never loses an acknowledged insert. They also make sure a [checkpoint](#run-heavy-queries-on-a-branch) contains every insert that finished before it. Each insert now waits for a round trip to Archil, so send data in large batches, as ClickHouse recommends anyway, or turn on [asynchronous inserts](https://clickhouse.com/docs/optimize/asynchronous-inserts). Finally, stop ClickHouse from ever starting while the disk isn't mounted. Without a guard, it would create an empty database in `/mnt/archil` on the local disk: ```bash theme={null} sudo mkdir -p /etc/systemd/system/clickhouse-server.service.d sudo tee /etc/systemd/system/clickhouse-server.service.d/archil.conf <<'EOF' [Service] ExecStartPre=/usr/bin/mountpoint -q /mnt/archil EOF sudo systemctl daemon-reload ``` ## Start the primary On `ch-primary`, mount the disk, hand a directory on it to the `clickhouse` user, and start the server: ```bash theme={null} sudo mkdir -p /mnt/archil sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount "$ARCHIL_DISK" /mnt/archil --region "$ARCHIL_REGION" sudo install -d -o clickhouse -g clickhouse /mnt/archil/clickhouse sudo systemctl start clickhouse-server ``` `archil mount` takes the disk in exclusive mode, so `ch-primary` is the only server that can write to it. That's what ClickHouse expects: one server owns a data directory. Create a table and load 100 million rows of synthetic events: ```sql schema.sql theme={null} CREATE TABLE events ( ts DateTime, user_id UInt64, event LowCardinality(String), duration_ms UInt32 ) ENGINE = MergeTree ORDER BY (event, ts); INSERT INTO events SELECT toDateTime('2026-01-01 00:00:00', 'UTC') + intDiv(number, 100) AS ts, cityHash64(number) % 1000000 AS user_id, ['view', 'view', 'view', 'click', 'click', 'purchase'][number % 6 + 1] AS event, cityHash64(number, 'duration') % 5000 AS duration_ms FROM numbers(100000000); ``` ```bash theme={null} clickhouse-client --multiquery < schema.sql clickhouse-client -q "SELECT event, count() AS events, round(avg(duration_ms)) AS avg_ms FROM events GROUP BY event ORDER BY events DESC" --format PrettyCompactMonoBlock ``` ```text theme={null} ┌─event────┬───events─┬─avg_ms─┐ 1. │ view │ 50000001 │ 2499 │ 2. │ click │ 33333333 │ 2500 │ 3. │ purchase │ 16666666 │ 2500 │ └──────────┴──────────┴────────┘ ``` In our runs, the load took about 12 seconds, with every part fsynced. The query took 2.5 seconds right after the load, while ClickHouse was still merging the new parts in the background, and about 0.5 seconds on repeat runs. The table takes about 740 MiB after compression. Every file ClickHouse wrote for it is under `/mnt/archil/clickhouse`. > [!NOTE] > > By default, ClickHouse listens only on `localhost`, and its `default` user has no password, so this guide runs > `clickhouse-client` on the servers themselves. Before you open ClickHouse to the network, set up users and passwords as > described in ClickHouse's [access control guide](https://clickhouse.com/docs/operations/access-rights), then set > `listen_host`. ## Move the database to another server Because the whole database is on the disk, moving it is a remount. On `ch-primary`, stop ClickHouse and unmount the disk. `archil unmount` flushes every outstanding write before it returns: ```bash theme={null} sudo systemctl stop clickhouse-server sudo archil unmount /mnt/archil ``` Then, on `ch-analytics`, mount the disk and start ClickHouse: ```bash theme={null} sudo mkdir -p /mnt/archil sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount "$ARCHIL_DISK" /mnt/archil --region "$ARCHIL_REGION" sudo systemctl start clickhouse-server clickhouse-client -q "SELECT count() FROM events" ``` ```text theme={null} 100000000 ``` `ch-analytics` didn't copy anything. It read the table's files from Archil as ClickHouse opened them, and anything already in Archil's server-side cache was served from there. In our runs, unmounting took about 0.15 seconds, mounting on `ch-analytics` 0.1 seconds, and starting ClickHouse about 1 second. The category query from the previous section took 0.9 seconds the first time on `ch-analytics`, against 0.5 seconds on a server that had already read the table. ### When a server dies If a server crashes instead of shutting down, it still holds the disk, and a normal `archil mount` on a replacement fails with `another client has an outstanding delegation to the root of the disk`. Once you know the old server is down, take the disk over with `--force`: ```bash theme={null} sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount "$ARCHIL_DISK" /mnt/archil --region "$ARCHIL_REGION" --force sudo systemctl start clickhouse-server ``` `--force` revokes the old server's ownership of the disk, and any writes it hadn't flushed are lost. With `fsync_after_insert` on, those are only inserts that hadn't returned yet, and background merges that ClickHouse redoes on its own. At startup, ClickHouse logs `Removing temporary directory` for each part that was still being written, and deletes it, as it would after a crash on a local disk. ClickHouse commits a large `INSERT` one block at a time, so an insert that was still running can be partly applied. We tested this by force-stopping `ch-analytics` in the middle of an `INSERT ... SELECT`. `--force` mounted the disk on `ch-primary` in 0.1 seconds, ClickHouse started in about 1 second, every row from inserts that had returned was there, and no parts were detached as broken. Move the database back to `ch-primary` the same way before you continue: stop ClickHouse and unmount on `ch-analytics`, then mount and start on `ch-primary`. ## Run heavy queries on a branch A long query on the primary competes with inserts and with every other query for CPU, memory, and I/O. Instead, take a checkpoint of the running database, branch the disk from it, and run the query on `ch-analytics` against the branch. The primary keeps running the whole time. On `ch-primary`, take a checkpoint with a unique name: ```bash theme={null} export CHECKPOINT=analytics-$(date -u +%Y%m%dT%H%M%S) sudo archil checkpoints create /mnt/archil "$CHECKPOINT" ``` A checkpoint captures the disk exactly as it is at that moment, like a crash at that instant without losing any writes. `MergeTree` commits a part by renaming its finished directory into place, so the checkpoint contains only whole parts, and every insert that returned before it. Create a branch from the checkpoint. This needs your API key and the disk ID, and it can run anywhere the `archil` client is installed: ```bash theme={null} export ARCHIL_API_KEY=key-... archil branches --region "$ARCHIL_REGION" create "$ARCHIL_DISK_ID" "$CHECKPOINT" --from-checkpoint "$CHECKPOINT" ``` On `ch-analytics`, mount the branch and start ClickHouse on it. Pass the same `CHECKPOINT` value you used on `ch-primary`: ```bash theme={null} export CHECKPOINT=analytics-... sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount "$ARCHIL_DISK" /mnt/archil --region "$ARCHIL_REGION" --branch "$CHECKPOINT" sudo systemctl start clickhouse-server ``` In our runs, the checkpoint took 40 ms and the branch 80 ms, both measured from the command line. ClickHouse logs `Status file ... already exists - unclean restart` when it starts on the branch, because the checkpoint was taken while the primary was running; that's expected. `ch-analytics` now has its own writable copy of the database as of the checkpoint. Run the heavy query there: ```bash theme={null} clickhouse-client -q "SELECT user_id, count() AS events, sum(duration_ms) AS total_ms FROM events GROUP BY user_id ORDER BY total_ms DESC LIMIT 3" --format PrettyCompactMonoBlock ``` ```text theme={null} ┌─user_id─┬─events─┬─total_ms─┐ 1. │ 596401 │ 154 │ 400707 │ 2. │ 761945 │ 139 │ 395268 │ 3. │ 667180 │ 145 │ 394841 │ └─────────┴────────┴──────────┘ ``` Meanwhile, the primary keeps taking inserts. On `ch-primary`, add 10 million rows: ```bash theme={null} clickhouse-client -q "INSERT INTO events SELECT toDateTime('2026-01-01 00:00:00', 'UTC') + intDiv(number, 100), cityHash64(number) % 1000000, ['view', 'view', 'view', 'click', 'click', 'purchase'][number % 6 + 1], cityHash64(number, 'duration') % 5000 FROM numbers(100000000, 10000000)" clickhouse-client -q "SELECT count() FROM events" ``` ```text theme={null} 110000000 ``` The branch on `ch-analytics` still has 100 million rows. Nothing either server does reaches the other: the branch can create tables, run backfills, or drop data without touching the primary. When you're done with the branch, stop ClickHouse and unmount it: ```bash theme={null} sudo systemctl stop clickhouse-server sudo archil unmount /mnt/archil ``` > [!TIP] > > The server on a branch runs background merges like any other ClickHouse server. If you only read from it, run > `SYSTEM STOP MERGES` after it starts, so all of its CPU goes to your queries. > [!WARNING] > > You can't delete branches and checkpoints yet. Use a new name for each branch, as the timestamped `CHECKPOINT` above does. > See [Branches & Checkpoints](/concepts/branches-and-checkpoints). Checkpoints also work as instant backups. If someone drops a table on the primary, branch the disk from a checkpoint taken before the drop, start ClickHouse on the branch, and copy the table back with the [`remote` table function](https://clickhouse.com/docs/sql-reference/table-functions/remote). ## Running it in production * **One server per disk or branch.** Two ClickHouse servers must never share one data directory, and exclusive mounts enforce that. To scale writes, shard: give each shard its own disk and query across them with a [`Distributed`](https://clickhouse.com/docs/engines/table-engines/special/distributed) table. * **No replication needed for durability.** Use plain `MergeTree` tables. ClickHouse Keeper and `ReplicatedMergeTree` are only worth running if you need a standby that's already started when the primary fails. * **Failover time grows with the number of parts.** At startup, ClickHouse reads a few small files from every active part. Keep the part count in check with a sensible partition key and batched inserts, and measure a remount on your own data. * **Size the client cache for your working set.** The Archil client caches file data in memory, and by default it uses at most 2 GiB. On a dedicated ClickHouse server, raise it with `--max-cache-mb` when you mount. See [Client caching](/details/caching). * **Mount at boot.** The guide's `ExecStartPre` check keeps ClickHouse from starting until the disk is mounted, and the service retries every 30 seconds. Mount the disk from your provisioning or a boot script, and ClickHouse starts once the mount appears. ## Run it in Archil sandboxes Everything above also works in [persistent sandboxes](/compute/sandboxes/introduction), and that's the most cost-effective way to run it: reads between a sandbox and its disks are [transfer within Archil](/administration/billing#data-transfer), which is free. The official `clickhouse/clickhouse-server` image already includes the `archil` client in a sandbox. Sandboxes don't run systemd, so ClickHouse runs as a [sandbox service](/compute/sandboxes/introduction) instead, which restarts if it crashes. This script sets up the primary. It's the configuration from earlier in this guide, with ClickHouse registered as a service: ```bash primary-setup.sh theme={null} set -euo pipefail install -m 644 /root/archil.xml /etc/clickhouse-server/config.d/archil.xml mkdir -p /var/lib/clickhouse/tmp && chown -R clickhouse:clickhouse /var/lib/clickhouse install -d -o clickhouse -g clickhouse /mnt/archil/clickhouse archil-sandbox services create clickhouse -- \ setpriv --reuid=clickhouse --regid=clickhouse --init-groups /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml timeout 60 bash -c 'until clickhouse-client -q "SELECT 1" >/dev/null 2>&1; do sleep 0.5; done' ``` Create the primary with the disk mounted at `/mnt/archil`, upload `archil.xml`, `schema.sql`, and the script, and run it: ```typescript primary.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "clickhouse" }); const sandbox = await archil.createSandbox({ name: "ch-primary", baseImage: "clickhouse/clickhouse-server:26.10", vcpuCount: 4, memSizeMiB: 16384, mounts: [{ disk, path: "/mnt/archil" }], }); for (const file of ["archil.xml", "schema.sql", "primary-setup.sh"]) { await sandbox.files.uploadFile(await readFile(file), `/root/${file}`); } const setup = await sandbox.exec("bash /root/primary-setup.sh && clickhouse-client --multiquery < /root/schema.sql"); if (setup.exitCode !== 0) throw new Error(setup.stderr); ``` ```python primary.py theme={null} import archil disk = archil.list_disks(name="clickhouse")[0] sandbox = archil.create_sandbox( name="ch-primary", base_image="clickhouse/clickhouse-server:26.10", vcpu_count=4, mem_size_mib=16384, mounts=[archil.SandboxMountSpec(disk=disk, path="/mnt/archil")], ) for file in ["archil.xml", "schema.sql", "primary-setup.sh"]: sandbox.files.upload_file(file, f"/root/{file}") setup = sandbox.exec("bash /root/primary-setup.sh && clickhouse-client --multiquery < /root/schema.sql") if setup.exit_code != 0: raise RuntimeError(setup.stderr) ``` Take checkpoints in the primary with `archil checkpoints create /mnt/archil "$CHECKPOINT"`, exactly as on a server, and create branches with `archil branches create` as before. A sandbox's `mounts` attach the disk itself, not a branch, so the analytics sandbox is created without any and mounts the branch itself with `archil mount --branch`. Pass the disk's ID, region, and token, and the checkpoint name, as environment variables: ```bash analytics-setup.sh theme={null} set -euo pipefail mkdir -p /mnt/archil archil mount "$ARCHIL_DISK" /mnt/archil --region "$ARCHIL_REGION" --branch "$CHECKPOINT" install -m 644 /root/archil.xml /etc/clickhouse-server/config.d/archil.xml mkdir -p /var/lib/clickhouse/tmp && chown -R clickhouse:clickhouse /var/lib/clickhouse archil-sandbox services create clickhouse -- \ setpriv --reuid=clickhouse --regid=clickhouse --init-groups /usr/bin/clickhouse-server --config-file=/etc/clickhouse-server/config.xml timeout 120 bash -c 'until clickhouse-client -q "SELECT 1" >/dev/null 2>&1; do sleep 0.2; done' ``` ```typescript analytics.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const sandbox = await archil.createSandbox({ name: "ch-analytics", baseImage: "clickhouse/clickhouse-server:26.10", vcpuCount: 4, memSizeMiB: 16384, }); for (const file of ["archil.xml", "analytics-setup.sh"]) { await sandbox.files.uploadFile(await readFile(file), `/root/${file}`); } const setup = await sandbox.exec("bash /root/analytics-setup.sh", { env: { ARCHIL_DISK: process.env.ARCHIL_DISK_ID!, ARCHIL_REGION: process.env.ARCHIL_REGION!, ARCHIL_MOUNT_TOKEN: process.env.ARCHIL_MOUNT_TOKEN!, CHECKPOINT: process.env.CHECKPOINT!, }, }); if (setup.exitCode !== 0) throw new Error(setup.stderr); ``` ```python analytics.py theme={null} import os import archil sandbox = archil.create_sandbox( name="ch-analytics", base_image="clickhouse/clickhouse-server:26.10", vcpu_count=4, mem_size_mib=16384, ) for file in ["archil.xml", "analytics-setup.sh"]: sandbox.files.upload_file(file, f"/root/{file}") setup = sandbox.exec( "bash /root/analytics-setup.sh", env={ "ARCHIL_DISK": os.environ["ARCHIL_DISK_ID"], "ARCHIL_REGION": os.environ["ARCHIL_REGION"], "ARCHIL_MOUNT_TOKEN": os.environ["ARCHIL_MOUNT_TOKEN"], "CHECKPOINT": os.environ["CHECKPOINT"], }, ) if setup.exit_code != 0: raise RuntimeError(setup.stderr) ``` In our runs in `aws-us-west-2`, the 100-million-row load took about 7 seconds in a 4 vCPU, 16 GiB sandbox. In the analytics sandbox, `archil mount --branch` took under half a second, ClickHouse was ready about 2 seconds later, and the heavy query from the previous section returned the same result in about 2 seconds. Delete the analytics sandbox when you're done with the branch: it mounts the branch from its setup script, so it isn't meant to be stopped and started again. ## Cost compared with ClickHouse Cloud Running this setup in Archil sandboxes costs **17% to 61% less** than ClickHouse Cloud's published example services. This section compares list prices in AWS `us-east-1` as of November 2026: Archil's Developer and Team rates from [Pricing & Billing](/administration/billing), and ClickHouse Cloud's from [its pricing page](https://clickhouse.com/pricing). **One server instead of two.** ClickHouse Cloud's Scale and Enterprise services keep at least two replicas once they're larger than 12 GiB. On Archil, the disk already keeps every write durable across Availability Zones, so one ClickHouse server per shard is enough. Per unit of compute, a sandbox also costs less: | Per 2 vCPU and 8 GiB, per hour | | | - | - | | ClickHouse Cloud Scale | \$0.298 | | ClickHouse Cloud Enterprise | \$0.390 | | Archil sandbox | **\$0.27** | **Storage without a second copy.** Archil stores the data for $0.025 per GB-month, or $25 per TB. ClickHouse Cloud charges $25.30 per TB, and bills its default daily backup as a second copy at the same rate; on Archil, checkpoints take its place. Archil also bills $0.20 per GiB-month for active data, which is data read or written in the last hour, so the total depends on how much of the table your queries touch each hour. Here is the monthly cost of ClickHouse Cloud's published examples, running 24 hours a day, next to one sandbox the size of one of their replicas, with 10% and 100% of the data active: | ClickHouse Cloud example | ClickHouse Cloud | Archil, 10% active | Archil, 100% active | | - | - | - | - | | Scale, 2 × 8 GiB, 1 TB | \$499 | **\$241** (52% less) | **\$409** (18% less) | | Scale, 2 × 16 GiB, 2 TB | \$987 | **\$483** (51% less) | **\$818** (17% less) | | Enterprise, 2 × 64 GiB, 10 TB | \$5,208 | **\$2,017** (61% less) | **\$3,694** (29% less) | The Archil columns are one sandbox (2 vCPU and 8 GiB, 4 vCPU and 16 GiB, or 16 vCPU and 64 GiB) running all month, plus storage. A branch sandbox for heavy queries, like a ClickHouse Cloud warehouse, only costs compute while it runs. To see where your workload falls between the two Archil columns, run it for a day and check the disk's active data on its **Usage** tab in the [Archil console](https://console.archil.com). ## Clean up On each server that's running ClickHouse, stop it and unmount the disk: ```bash theme={null} sudo systemctl stop clickhouse-server sudo archil unmount /mnt/archil ``` Then delete the disk in the Archil console, and terminate both servers. If you ran ClickHouse in sandboxes, stop and delete them instead. ## Next steps > [card] [Branches & Checkpoints](/concepts/branches-and-checkpoints) > > How checkpoints and branches work, and their CLI commands. > [card] [Mount on Linux](/mounting/linux) > > Requirements, authorization, and mount options for Linux servers. > [card] [Client caching](/details/caching) > > Size, warm, and refresh the client's in-memory cache. > [card] [Analytics with DuckDB](/guides/analytics/duckdb) > > Query Parquet in S3 in place, without loading it into a database. # Analytics with DuckDB Source: https://docs.archil.com/guides/analytics/duckdb Query 81 million places from the Overture Maps public bucket with DuckDB, without copying the data, and serve results over HTTPS. [Overture Maps](https://overturemaps.org) publishes open map data as Parquet files in a public S3 bucket. Its places dataset alone is about 81 million points of interest worldwide, in 11 GB of Parquet. This guide attaches that bucket to an Archil disk and queries it in place with [DuckDB](https://duckdb.org), with no download step and no AWS credentials. You'll run ad-hoc SQL against the bucket from a [persistent sandbox](/compute/sandboxes/introduction), save an extract as a DuckDB database on a second disk, put a small HTTP query endpoint in front of it, and run a one-off query with [serverless execution](/compute/serverless-execution). ## Prerequisites The Overture bucket is in AWS `us-west-2`, so this guide uses the `aws-us-west-2` Archil region. Export an API key created in that region, then install the SDK for the language you'll orchestrate from. You'll also use `curl` to call the query endpoint. ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 ``` ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` Save each script in this guide in your project directory. Run TypeScript scripts with `npx tsx .ts` and Python scripts with `python .py`. ## Create the disks and a sandbox You need two disks: * `overture-maps` is backed by the public bucket, using the `release/2026-09-23.0/` prefix as its root. Archil reads the Parquet files from the bucket as DuckDB asks for them, and caches what it reads. * `duckdb-workspace` is a regular Archil disk for everything you write: the DuckDB binary and the database you build. > [!WARNING] > > A disk backed by a public bucket is read-only: Archil has no credentials to write to the bucket. Always mount it with > `readOnly: true` (`read_only=True` in Python), in sandboxes and in serverless commands, and write your output to another disk. This script runs inside the sandbox. It shows what the bucket contains, puts the DuckDB command-line binary on the workspace disk, and installs the DuckDB Python package for the query server you'll run later. ```bash setup.sh theme={null} set -euo pipefail ls /mnt/overture/theme=places/type=place | wc -l du -sh --apparent-size /mnt/overture/theme=places/type=place # Keep the CLI on the workspace disk so serverless commands can use it too mkdir -p /mnt/work/bin curl -sfL -o /tmp/duckdb.zip https://github.com/duckdb/duckdb/releases/download/v1.5.6/duckdb_cli-linux-amd64.zip unzip -oq /tmp/duckdb.zip -d /mnt/work/bin /mnt/work/bin/duckdb --version pip install -q --root-user-action=ignore duckdb==1.5.6 ``` Create both disks and a sandbox that mounts the bucket-backed disk read-only at `/mnt/overture` and the workspace at `/mnt/work`, then run the script: ```typescript setup.ts theme={null} import * as archil from "disk"; import { readFile } from "node:fs/promises"; const { disk: overture } = await archil.createDisk({ name: "overture-maps", mounts: [{ type: "s3", bucketName: "overturemaps-us-west-2", bucketPrefix: "release/2026-09-23.0/" }], }); const { disk: workspace } = await archil.createDisk({ name: "duckdb-workspace" }); const sandbox = await archil.createSandbox({ name: "duckdb", baseImage: "python:3.13", vcpuCount: 4, memSizeMiB: 8192, mounts: [ { disk: overture, path: "/mnt/overture", readOnly: true }, { disk: workspace, path: "/mnt/work" }, ], }); await sandbox.files.uploadFile(await readFile("setup.sh"), "/root/setup.sh"); const setup = await sandbox.exec("bash /root/setup.sh", { timeoutSeconds: 600 }); if (setup.exitCode !== 0) throw new Error(setup.stderr); console.log(setup.stdout); ``` ```python setup.py theme={null} import archil overture = archil.create_disk( name="overture-maps", mounts=[archil.S3Mount(bucket_name="overturemaps-us-west-2", bucket_prefix="release/2026-09-23.0/")], ).disk workspace = archil.create_disk(name="duckdb-workspace").disk sandbox = archil.create_sandbox( name="duckdb", base_image="python:3.13", vcpu_count=4, mem_size_mib=8192, mounts=[ archil.SandboxMountSpec(disk=overture, path="/mnt/overture", read_only=True), archil.SandboxMountSpec(disk=workspace, path="/mnt/work"), ], ) sandbox.files.upload_file("setup.sh", "/root/setup.sh") setup = sandbox.exec("bash /root/setup.sh", timeout_seconds=600) if setup.exit_code != 0: raise RuntimeError(setup.stderr) print(setup.stdout) ``` The places theme is 16 Parquet files: ``` 16 11G /mnt/overture/theme=places/type=place v1.5.6 (Variegata) 069cc9f9b5 ``` ## Query the bucket in place Every Overture row has a `bbox` column with its bounding box, and the files store min/max statistics for it. When a query filters on `bbox`, DuckDB uses those statistics to skip the row groups that can't match, so it only reads a small part of the 11 GB. This script finds the sandbox by name and runs two statements. The first breaks down the places in Seattle by category, straight from the bucket. The second saves every place in Washington state as a table in `places.duckdb` on the workspace disk. ```typescript explore.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "duckdb")!; const duckdb = async (sql: string) => { const res = await sandbox.exec(`bin/duckdb places.duckdb -c "${sql}"`, { cwd: "/mnt/work" }); console.log(res.stdout, res.stderr); }; await duckdb(` SELECT basic_category AS category, count(*) AS places FROM '/mnt/overture/theme=places/type=place/*.parquet' WHERE bbox.xmin BETWEEN -122.44 AND -122.24 AND bbox.ymin BETWEEN 47.49 AND 47.74 GROUP BY 1 ORDER BY 2 DESC LIMIT 5`); await duckdb(` CREATE OR REPLACE TABLE places AS SELECT id, names.primary AS name, basic_category AS category, addresses[1].locality AS city, bbox.xmin AS lon, bbox.ymin AS lat FROM '/mnt/overture/theme=places/type=place/*.parquet' WHERE bbox.xmin BETWEEN -124.8 AND -116.9 AND bbox.ymin BETWEEN 45.5 AND 49.0 AND addresses[1].region = 'WA'; SELECT count(*) AS places FROM places`); ``` ```python explore.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "duckdb") def duckdb(sql): res = sandbox.exec(f"""bin/duckdb places.duckdb -c "{sql}" """, cwd="/mnt/work") print(res.stdout, res.stderr) duckdb(""" SELECT basic_category AS category, count(*) AS places FROM '/mnt/overture/theme=places/type=place/*.parquet' WHERE bbox.xmin BETWEEN -122.44 AND -122.24 AND bbox.ymin BETWEEN 47.49 AND 47.74 GROUP BY 1 ORDER BY 2 DESC LIMIT 5""") duckdb(""" CREATE OR REPLACE TABLE places AS SELECT id, names.primary AS name, basic_category AS category, addresses[1].locality AS city, bbox.xmin AS lon, bbox.ymin AS lat FROM '/mnt/overture/theme=places/type=place/*.parquet' WHERE bbox.xmin BETWEEN -124.8 AND -116.9 AND bbox.ymin BETWEEN 45.5 AND 49.0 AND addresses[1].region = 'WA'; SELECT count(*) AS places FROM places""") ``` ``` ┌──────────────────────┬────────┐ │ category │ places │ │ varchar │ int64 │ ├──────────────────────┼────────┤ │ restaurant │ 3745 │ │ home_service │ 2791 │ │ real_estate_service │ 2635 │ │ professional_service │ 2238 │ │ financial_service │ 2232 │ └──────────────────────┴────────┘ ┌────────┐ │ places │ │ int64 │ ├────────┤ │ 382501 │ └────────┘ ``` In our runs, the Seattle query took under a second the first time, while DuckDB fetched the row groups it needed from the bucket, and about 0.15 seconds on repeat runs, which Archil serves from its cache. Saving the 382,501 Washington places took under a second. `places.duckdb` is an ordinary file on the workspace disk, so it survives the sandbox stopping, and anything that mounts the workspace can query it. ## Serve queries over HTTPS This server opens `places.duckdb` read-only and runs the SQL in each POST body. External file access is turned off and the setting is locked, so a query can read the `places` table but not other files in the sandbox or on the disks. ```python server.py theme={null} import json from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer import duckdb db = duckdb.connect( "/mnt/work/places.duckdb", read_only=True, config={"enable_external_access": False, "lock_configuration": True}, ) class QueryHandler(BaseHTTPRequestHandler): def do_POST(self): sql = self.rfile.read(int(self.headers["Content-Length"])).decode() try: result = db.cursor().execute(sql) body = {"columns": [c[0] for c in result.description], "rows": result.fetchall()} status = 200 except duckdb.Error as e: body, status = {"error": str(e)}, 400 payload = json.dumps(body, default=str).encode() self.send_response(status) self.send_header("Content-Type", "application/json") self.send_header("Content-Length", str(len(payload))) self.end_headers() self.wfile.write(payload) ThreadingHTTPServer(("0.0.0.0", 8080), QueryHandler).serve_forever() ``` Upload it, run it as a supervised service so it restarts on failure and comes back when the sandbox reboots, and create a port token. Sandbox ports are private by default: a request without a valid token in the `X-Archil-Token` header gets a `401`, so you don't need to build authentication into the server. ```typescript serve.ts theme={null} import * as archil from "disk"; import { readFile } from "node:fs/promises"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "duckdb")!; await sandbox.files.uploadFile(await readFile("server.py"), "/root/server.py"); await sandbox.exec("archil-sandbox services create duckdb-api --tcp-port 8080 -- python3 /root/server.py"); const { hostname, token } = await sandbox.createPortToken(8080, { ttl: "1h" }); console.log(`export DUCKDB_URL=https://${hostname}`); console.log(`export DUCKDB_TOKEN=${token}`); ``` ```python serve.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "duckdb") sandbox.files.upload_file("server.py", "/root/server.py") sandbox.exec("archil-sandbox services create duckdb-api --tcp-port 8080 -- python3 /root/server.py") port_token = sandbox.create_port_token(8080, ttl="1h") print(f"export DUCKDB_URL=https://{port_token.hostname}") print(f"export DUCKDB_TOKEN={port_token.token}") ``` Run the two `export` lines the script prints, then send a query from your machine: ```bash theme={null} curl -s "$DUCKDB_URL" -H "X-Archil-Token: $DUCKDB_TOKEN" \ --data "SELECT city, count(*) AS places, count(*) FILTER (WHERE category = 'coffee_shop') AS coffee_shops FROM places GROUP BY 1 ORDER BY 2 DESC LIMIT 5" ``` ```json theme={null} {"columns": ["city", "places", "coffee_shops"], "rows": [["Seattle", 57689, 592], ["Spokane", 16961, 190], ["Vancouver", 16671, 147], ["Tacoma", 15690, 133], ["Bellevue", 15158, 76]]} ``` Each request took about 0.2 seconds end to end from our machine, including the TLS handshake. The same request without the header returns `401`. Hand the URL and token to a dashboard, a notebook, or an agent tool, and create a new token when this one expires. ## Run a one-off query with serverless execution For a quick question, you don't have to go through the sandbox. [Serverless execution](/compute/serverless-execution) runs a command in a short-lived VM, and `archil.exec` can mount several disks at once, each under `/mnt/archil/`, which is also the working directory. This command mounts both disks read-only and runs the DuckDB binary from the workspace against the bucket, this time for Paris: ```typescript oneoff.ts theme={null} import * as archil from "disk"; const [overture] = await archil.listDisks({ name: "overture-maps" }); const [workspace] = await archil.listDisks({ name: "duckdb-workspace" }); const { stdout, timing } = await archil.exec({ disks: { overture: { disk: overture, readOnly: true }, work: { disk: workspace, readOnly: true }, }, command: `work/bin/duckdb -c "SELECT basic_category AS category, count(*) AS places FROM 'overture/theme=places/type=place/*.parquet' WHERE bbox.xmin BETWEEN 2.25 AND 2.42 AND bbox.ymin BETWEEN 48.81 AND 48.91 GROUP BY 1 ORDER BY 2 DESC LIMIT 5"`, }); console.log(stdout, `${timing.totalMs} ms`); ``` ```python oneoff.py theme={null} import archil overture = archil.list_disks(name="overture-maps")[0] workspace = archil.list_disks(name="duckdb-workspace")[0] res = archil.exec( disks={ "overture": archil.ExecMountSpec(overture, read_only=True), "work": archil.ExecMountSpec(workspace, read_only=True), }, command="""work/bin/duckdb -c "SELECT basic_category AS category, count(*) AS places FROM 'overture/theme=places/type=place/*.parquet' WHERE bbox.xmin BETWEEN 2.25 AND 2.42 AND bbox.ymin BETWEEN 48.81 AND 48.91 GROUP BY 1 ORDER BY 2 DESC LIMIT 5" """, ) print(res.stdout, f"{res.timing.total_ms} ms") ``` ``` ┌────────────────────────────┬────────┐ │ category │ places │ │ varchar │ int64 │ ├────────────────────────────┼────────┤ │ restaurant │ 17286 │ │ fashion_and_apparel_store │ 9977 │ │ professional_service │ 7466 │ │ personal_or_beauty_service │ 5253 │ │ food_and_beverage_store │ 4384 │ └────────────────────────────┴────────┘ 1245 ms ``` `totalMs` is measured by the server, from receiving the request to returning the result. In our runs the first Paris query took about 1.2 seconds, including starting the VM, mounting both disks, and reading the row groups it needed from the bucket. Repeat runs took under a second. Each serverless command is limited to 5 minutes, so use the sandbox for long-running work. ## Clean up Stop and delete the sandbox, then delete both disks. Deleting `overture-maps` doesn't affect the public bucket. ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "duckdb")!; await sandbox.stop(); await sandbox.delete(); for (const name of ["overture-maps", "duckdb-workspace"]) { const [disk] = await archil.listDisks({ name }); await disk.delete(); } ``` ```python cleanup.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "duckdb") sandbox.stop() sandbox.delete() for name in ["overture-maps", "duckdb-workspace"]: archil.list_disks(name=name)[0].delete() ``` ## Next steps * [S3 Object Storage](/data-sources/s3-object-storage) — attach your own buckets with read-write access * [Persistent sandboxes](/compute/sandboxes/introduction) — services, port tokens, and stopping a sandbox between sessions * [Serverless execution](/compute/serverless-execution) — run commands against several disks at once * [Search logs across sandboxes](/guides/analytics/log-search) — another analytics workflow on Archil disks # Search logs across sandboxes Source: https://docs.archil.com/guides/analytics/log-search Write structured logs from several sandboxes to one Archil disk, then search and aggregate them from your laptop. This guide runs three instances of a small Python service in [persistent sandboxes](/compute/sandboxes/introduction). Each instance writes JSON logs with Python's standard `logging` module to its own directory on a shared Archil disk. From your laptop, you search every instance's logs at once with [`disk.grep`](/compute/search-files) and compute per-instance error counts with [serverless execution](/compute/serverless-execution) and `jq`. There's no log shipper, agent, or index to run. The application writes ordinary files, and the search runs next to the data. ## Prerequisites Export your Archil API key and a region that offers sandboxes (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), then install the SDK for the language you'll orchestrate from. ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 ``` ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` Save each script in this guide in your project directory. Run TypeScript scripts with `npx tsx .ts` and Python scripts with `python .py`. ## Write the service The service below stands in for your application: it handles a stream of fake requests and logs one JSON line per request, with timeouts on `/checkout` logged as errors. Logging is plain `logging` with two additions: * A formatter that turns each record into one line of JSON, so the logs are easy to grep and to parse. * A `RotatingFileHandler`, which renames `app.log` to `app.log.1` when it reaches 1 MB and keeps five old files. Each instance writes to `/mnt/archil/logs//`. Pass the instance name in explicitly rather than using the hostname, because every sandbox reports the same hostname. ```python app.py theme={null} import json import logging import os import random import time from datetime import datetime, timezone from logging.handlers import RotatingFileHandler INSTANCE = os.environ["INSTANCE"] ERROR_RATE = float(os.environ.get("ERROR_RATE", "0.02")) LOG_DIR = f"/mnt/archil/logs/{INSTANCE}" class JsonFormatter(logging.Formatter): def format(self, record): entry = { "ts": datetime.fromtimestamp(record.created, timezone.utc).isoformat(timespec="milliseconds"), "level": record.levelname, "instance": INSTANCE, "msg": record.getMessage(), **getattr(record, "fields", {}), } return json.dumps(entry) os.makedirs(LOG_DIR, exist_ok=True) handler = RotatingFileHandler(f"{LOG_DIR}/app.log", maxBytes=1_000_000, backupCount=5) handler.setFormatter(JsonFormatter()) log = logging.getLogger("app") log.addHandler(handler) log.setLevel(logging.INFO) log.info("started") while True: path = random.choice(["/", "/search", "/cart", "/checkout"]) ms = round(random.lognormvariate(3, 0.6)) if path == "/checkout" and random.random() < ERROR_RATE * 4: log.error("upstream timeout", extra={"fields": {"path": path, "status": 504, "ms": ms + 3000}}) else: log.info("request", extra={"fields": {"path": path, "status": 200, "ms": ms}}) time.sleep(random.uniform(0.005, 0.02)) ``` The 1 MB limit is small so that you see a rotation while following this guide; use a larger limit in production. ## Start three instances Create the disk and three sandboxes that mount it in [conditional mode](/concepts/sharing-disks). Conditional mode sends writes to the server instead of taking an exclusive lock on the disk, so all three instances can write at the same time. Each instance writes to its own directory, so their writes never overlap. Writes in conditional mode are slower than on an exclusive mount, which doesn't matter for appending log lines. Each sandbox runs the service as a supervised [service](/compute/sandboxes/introduction#hosting-network-services), which restarts it if it crashes and starts it again when the sandbox reboots. `app-3` gets a higher error rate, so there's something to find. ```typescript start.ts theme={null} import * as archil from "disk"; import { readFile } from "node:fs/promises"; const { disk } = await archil.createDisk({ name: "app-logs" }); for (const name of ["app-1", "app-2", "app-3"]) { const sandbox = await archil.createSandbox({ name, baseImage: "python:3.13", vcpuCount: 1, memSizeMiB: 1024, mounts: [{ disk, path: "/mnt/archil", conditional: true }], }); await sandbox.files.uploadFile(await readFile("app.py"), "/root/app.py"); const errorRate = name === "app-3" ? "0.1" : "0.01"; await sandbox.exec( `archil-sandbox services create app --env INSTANCE=${name} --env ERROR_RATE=${errorRate} -- python3 /root/app.py`, ); console.log(`${name} is running`); } ``` ```python start.py theme={null} import archil disk = archil.create_disk(name="app-logs").disk for name in ["app-1", "app-2", "app-3"]: sandbox = archil.create_sandbox( name=name, base_image="python:3.13", vcpu_count=1, mem_size_mib=1024, mounts=[archil.SandboxMountSpec(disk=disk, path="/mnt/archil", conditional=True)], ) sandbox.files.upload_file("app.py", "/root/app.py") error_rate = "0.1" if name == "app-3" else "0.01" sandbox.exec( f"archil-sandbox services create app --env INSTANCE={name} --env ERROR_RATE={error_rate} -- python3 /root/app.py" ) print(f"{name} is running") ``` Each instance logs roughly 80 lines per second. Give them a couple of minutes before you search, so the logs rotate at least once. ## Search every instance at once `disk.grep` takes an extended regular expression and a directory on the disk. Archil lists the files under that directory and fans the search out across parallel workers that run next to the data, so you don't download anything. This search finds every error line, in current and rotated files, across all three instances: ```typescript search.ts theme={null} import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "app-logs" }); const res = await disk.grep({ directory: "logs", pattern: '"level": "ERROR"', recursive: true, maxResults: 10000 }); console.log(`${res.matches.length} matches in ${res.filesScanned} files (${res.stoppedReason}, ${res.durationMs} ms)`); for (const m of res.matches.slice(0, 3)) console.log(`${m.file}:${m.line} ${m.text}`); ``` ```python search.py theme={null} import archil disk = archil.list_disks(name="app-logs")[0] res = disk.grep(directory="logs", pattern='"level": "ERROR"', recursive=True, max_results=10000) print(f"{len(res.matches)} matches in {res.files_scanned} files ({res.stopped_reason}, {res.duration_ms} ms)") for m in res.matches[:3]: print(f"{m.file}:{m.line} {m.text}") ``` ``` 1113 matches in 6 files (completed, 98 ms) logs/app-3/app.log:8 {"ts": "2026-10-17T13:35:40.941+00:00", "level": "ERROR", "instance": "app-3", "msg": "upstream timeout", "path": "/checkout", "status": 504, "ms": 3026} logs/app-3/app.log:15 {"ts": "2026-10-17T13:35:41.038+00:00", "level": "ERROR", "instance": "app-3", "msg": "upstream timeout", "path": "/checkout", "status": 504, "ms": 3022} logs/app-3/app.log:20 {"ts": "2026-10-17T13:35:41.114+00:00", "level": "ERROR", "instance": "app-3", "msg": "upstream timeout", "path": "/checkout", "status": 504, "ms": 3037} ``` `stoppedReason` is `completed` when every file was scanned. A search stops early, and returns a sample of the matches, when it reaches `maxResults` (1,000 by default) or `maxDurationSeconds` (30 by default). There are more than 1,000 errors here, so the search raises `maxResults` to 10,000. Lines become searchable without closing or rotating the file. Python's file handler writes every record to the file as it's logged, and the `archil` client buffers recent writes and sends them to the disk in the background, like on any Archil mount. If you use a logger that buffers output in memory, lines become searchable after it flushes. ## Count errors per instance For questions that need more than matching lines, run a command next to the data with `disk.exec`. The serverless runtime includes `jq`, and the disk is mounted at `/mnt/archil`, which is also the working directory. This command lists `app-3`'s log files, then parses every log line from every instance and counts lines and errors per instance: ```typescript stats.ts theme={null} import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "app-logs" }); const { stdout } = await disk.exec( `ls -l logs/app-3 && jq -nr '(["instance", "lines", "errors"] | @tsv), ([inputs] | group_by(.instance)[] | [.[0].instance, length, (map(select(.level == "ERROR")) | length)] | @tsv)' logs/*/app.log*`, ); console.log(stdout); ``` ```python stats.py theme={null} import archil disk = archil.list_disks(name="app-logs")[0] res = disk.exec( """ls -l logs/app-3 && jq -nr '(["instance", "lines", "errors"] | @tsv), ([inputs] | group_by(.instance)[] | [.[0].instance, length, (map(select(.level == "ERROR")) | length)] | @tsv)' logs/*/app.log*""" ) print(res.stdout) ``` ``` total 153 -rw-r--r-- 1 root root 246059 Oct 17 13:36 app.log -rw-r--r-- 1 root root 999989 Oct 17 13:35 app.log.1 instance lines errors app-1 9143 91 app-2 9077 101 app-3 8920 943 ``` The `app.log.1` file is the rotated log. Because the glob `logs/*/app.log*` matches rotated files too, the counts cover everything the instances have logged since they started (up to the five rotated files each one keeps). Serverless commands mount the disk in shared mode, so they read the logs while the instances keep writing. ## Clean up Stop and delete the three sandboxes, then delete the disk and the logs on it. ```typescript cleanup.ts theme={null} import * as archil from "disk"; for (const sandbox of await archil.listSandboxes()) { if (["app-1", "app-2", "app-3"].includes(sandbox.name)) { await sandbox.stop(); await sandbox.delete(); } } const [disk] = await archil.listDisks({ name: "app-logs" }); await disk.delete(); ``` ```python cleanup.py theme={null} import archil for sandbox in archil.list_sandboxes(): if sandbox.name in ["app-1", "app-2", "app-3"]: sandbox.stop() sandbox.delete() archil.list_disks(name="app-logs")[0].delete() ``` ## Next steps * [Search files](/compute/search-files) — concurrency and limits for `disk.grep` * [Serverless execution](/compute/serverless-execution) — run commands across several disks, read-only or pinned to a subdirectory * [Sharing disks](/concepts/sharing-disks) — how exclusive, shared, and conditional mounts behave * [Analytics with DuckDB](/guides/analytics/duckdb) — query Parquet data on an Archil disk with SQL * [Agentic observability](/guides/agents/agentic-observability) — have an agent triage errors and build dashboards from the same kind of logs # Apache Spark Source: https://docs.archil.com/guides/analytics/spark Run an Apache Spark cluster across Archil sandboxes that reads Parquet from S3 through one Archil disk and writes Delta Lake tables to another, only while a job runs: 2 to 5 times faster than Databricks serverless, for under half the cost. This guide runs an open-source [Apache Spark](https://spark.apache.org/) cluster across four [sandboxes](/compute/sandboxes/introduction) with 32 vCPU each. Every node mounts two Archil disks at the same paths: one backed by an S3 bucket of Parquet files, and one for the [Delta Lake](https://delta.io/) tables Spark writes. That gives the cluster three properties: * **A cluster only while a job runs.** A stopped sandbox starts in about a second, so the job script starts the nodes, runs the job, and stops them again. In our runs, the cluster was ready 5 seconds after the script started, and the whole job took about 30 seconds once Archil had cached the data. * **Your bucket, read in place.** Spark reads the bucket's Parquet files as ordinary files, with no S3 connector or credentials to configure, and Archil caches what it reads. Each node sees the same files at `/mnt/overture` and `/mnt/lake`, so a Delta table is a directory of ordinary files that every executor writes to directly. * **Faster and cheaper than Databricks serverless.** In our runs, the cluster finished the same jobs 2 to 5 times sooner than Databricks serverless in performance-optimized mode, for a fifth to a half of the cost, even on a first run with nothing cached. You'll aggregate 2.5 billion buildings from the [Overture Maps](https://overturemaps.org) public bucket into a Delta table, query the table, and compare the cluster with Databricks. ## Prerequisites The Overture bucket is in AWS `us-west-2`, so this guide uses the `aws-us-west-2` Archil region. Export an API key created in that region, then install the SDK for the language you'll orchestrate from. The cluster uses 128 vCPU across four sandboxes, so you need the Developer plan or above; see [Plans and limits](/administration/billing#plans-and-limits). ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 ``` ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` Save each script in this guide in your project directory. Run TypeScript scripts with `npx tsx .ts` and Python scripts with `python .py`. ## How the nodes connect Spark's master, workers, and driver open connections to each other on fixed ports. Sandboxes don't share a private network, so each node runs [wstunnel](https://github.com/erebe/wstunnel), an open-source tunnel over WebSockets: * **A tunnel server on port 9000.** The setup script creates a [port token](/compute/sandboxes/introduction#hosting-network-services) for each node's port 9000, so only the other nodes, which hold that token, can reach it. Nothing else in the sandbox is exposed. * **A tunnel to every other node.** Each node gets an address of its own, `127.0.1.1` for the first node, `127.0.1.2` for the second, and so on. Spark listens only on its node's address, and on every other node, a tunnel forwards that address's Spark ports to the node through its port 9000. Every node reaches every other node's Spark ports at the same address, so Spark runs unchanged. ## Create the disks and the nodes You need two disks: * `overture-maps` is backed by the public bucket, using the `release/2026-09-23.0/` prefix as its root. Its buildings theme is 512 Parquet files, 258 GiB in total. Archil reads them from the bucket as Spark asks for them. * `spark-lake` is a regular Archil disk for the tables Spark writes. > [!WARNING] > > A disk backed by a public bucket is read-only: Archil has no credentials to write to the bucket. The nodes mount it with > `readOnly: true` (`read_only=True` in Python), and write their output to the lake. An Overture release never changes, so the setup script marks `overture-maps` immutable with `archil set-immutable`. An immutable disk's files can't change, so any number of sandboxes can read them at full speed at the same time. Marking a disk takes a writable mount, so the script runs the command once in a small sandbox and deletes it. This script runs once in every node, during setup. It downloads the tunnel and Delta Lake to the node's root filesystem, so no job waits on a download: ```bash install.sh theme={null} # Runs once in every node, during setup. Downloads what the cluster needs to the node's root filesystem, # which survives stopping the node, so no job waits on a download. set -euo pipefail # The tunnel curl -fsSL https://github.com/erebe/wstunnel/releases/download/v11.1.0/wstunnel_11.1.0_linux_amd64.tar.gz \ | tar -xz -C /root wstunnel chmod +x /root/wstunnel # Delta Lake, into Spark's package cache /opt/spark/bin/spark-sql --packages io.delta:delta-spark_2.13:4.4.1 -S -e "SELECT 1" >/dev/null 2>&1 ls ~/.ivy2*/jars | grep -c delta ``` This script runs in every node while the cluster is up. It mounts the lake, starts the tunnels, writes Spark's settings, and runs Spark's master on the first node and a worker on every node: ```bash node.sh theme={null} # Runs in every node while the cluster is up. The setup script put cluster.env and peers in /root. set -euo pipefail source /root/cluster.env # NODE, LAKE, LAKE_TOKEN ip() { echo "127.0.1.$1"; } # each node's address, the same on every node mkdir -p /tmp/spark-logs # The lake, mounted so each node sees files other nodes write within a second mkdir -p /mnt/lake mountpoint -q /mnt/lake || ARCHIL_MOUNT_TOKEN=$LAKE_TOKEN archil mount "$LAKE" /mnt/lake \ --region aws-us-west-2 --conditional --cache-expiry-secs 1 >/dev/null # A tunnel server on the sandbox's private port 9000, and a tunnel to each other node's Spark ports /root/wstunnel server ws://0.0.0.0:9000 >/tmp/spark-logs/wstunnel-server.log 2>&1 & while read -r peer host token; do ports="7078 7079"; [ "$peer" = 1 ] && ports="7077 7078 7079 7080 7081" forwards=$(for p in $ports; do printf -- '-L tcp://%s:%s:%s:%s ' "$(ip "$peer")" "$p" "$(ip "$peer")" "$p"; done) /root/wstunnel client $forwards "wss://$host" --http-headers "X-Archil-Token: $token" \ >"/tmp/spark-logs/wstunnel-$peer.log" 2>&1 & done /opt/spark/conf/spark-env.sh < /opt/spark/conf/spark-defaults.conf </tmp/spark-logs/master.log 2>&1 & fi /opt/spark/bin/spark-class org.apache.spark.deploy.worker.Worker "spark://$(ip 1):7077" >/tmp/spark-logs/worker.log 2>&1 & wait ``` Each node mounts the lake itself, in conditional mode, so every node can write to it, and with [`--cache-expiry-secs 1`](/reference/archil-cli#mount), so each node sees the files other nodes write within a second. Delta Lake needs that: after the driver commits a table, executors on every node read the commit's files. The script runs four tasks per vCPU, because tasks reading uncached data mostly wait on object storage, not compute. The setup script creates both disks and a token for the lake, then creates the four nodes, each with 32 vCPU and 64 GiB of memory and `overture-maps` mounted read-only. It gives every node its settings, the other nodes' tunnel addresses, and the scripts, runs `install.sh`, and stops the node: ```typescript setup.ts theme={null} import * as archil from "disk"; import { readFile } from "node:fs/promises"; const NODES = 4; const IMAGE = "spark:4.2.1-scala2.13-java17-python3-ubuntu"; // The Overture release, read in place from its public bucket, and a disk for the tables Spark writes const { disk: overture } = await archil.createDisk({ name: "overture-maps", mounts: [{ type: "s3", bucketName: "overturemaps-us-west-2", bucketPrefix: "release/2026-09-23.0/" }], }); const { disk: lake } = await archil.createDisk({ name: "spark-lake" }); const { token: lakeToken } = await lake.createToken("spark"); // Mark the release immutable, so every sandbox can read it at full speed at the same time const marker = await archil.createSandbox({ mounts: [{ disk: overture, path: "/mnt/overture" }] }); const marked = await marker.exec("archil set-immutable /mnt/overture"); if (marked.exitCode !== 0) throw new Error(marked.stdout + marked.stderr); await marker.stop(); await marker.delete(); // The nodes, each with a private port that only holders of its token can reach const nodes = await Promise.all( Array.from({ length: NODES }, (_, i) => archil.createSandbox({ name: `spark-${i + 1}`, baseImage: IMAGE, vcpuCount: 32, memSizeMiB: 65536, idleTtlSeconds: 600, mounts: [{ disk: overture, path: "/mnt/overture", readOnly: true }], }), ), ); const tunnels = await Promise.all(nodes.map((node) => node.createPortToken(9000))); // Give every node its settings, the other nodes' tunnel addresses, and the scripts, and download what it needs await Promise.all( nodes.map(async (node, i) => { const peers = tunnels.map((t, j) => `${j + 1} ${t.hostname} ${t.token}`).filter((_, j) => j !== i); const env = `NODE=${i + 1}\nLAKE=${lake.id}\nLAKE_TOKEN=${lakeToken}\n`; await node.files.uploadFile(Buffer.from(env), "/root/cluster.env"); await node.files.uploadFile(Buffer.from(peers.join("\n") + "\n"), "/root/peers"); for (const file of ["install.sh", "node.sh", "buildings.py", "query.sql"]) { await node.files.uploadFile(await readFile(file), `/root/${file}`); } const install = await node.exec("bash /root/install.sh", { timeoutSeconds: 600 }); if (install.exitCode !== 0) throw new Error(install.stderr); await node.stop(); }), ); console.log(`Created ${NODES} nodes`); ``` ```python setup.py theme={null} import tempfile from concurrent.futures import ThreadPoolExecutor import archil NODES = 4 IMAGE = "spark:4.2.1-scala2.13-java17-python3-ubuntu" # The Overture release, read in place from its public bucket, and a disk for the tables Spark writes overture = archil.create_disk( name="overture-maps", mounts=[archil.S3Mount(bucket_name="overturemaps-us-west-2", bucket_prefix="release/2026-09-23.0/")], ).disk lake = archil.create_disk(name="spark-lake").disk lake_token = lake.create_token("spark").token # Mark the release immutable, so every sandbox can read it at full speed at the same time marker = archil.create_sandbox(mounts=[archil.SandboxMountSpec(disk=overture, path="/mnt/overture")]) marked = marker.exec("archil set-immutable /mnt/overture") marker.stop() marker.delete() if marked.exit_code != 0: raise RuntimeError(marked.stdout + marked.stderr) # The nodes, each with a private port that only holders of its token can reach def create(i): return archil.create_sandbox( name=f"spark-{i + 1}", base_image=IMAGE, vcpu_count=32, mem_size_mib=65536, idle_ttl_seconds=600, mounts=[archil.SandboxMountSpec(disk=overture, path="/mnt/overture", read_only=True)], ) with ThreadPoolExecutor(NODES) as pool: nodes = list(pool.map(create, range(NODES))) tunnels = [node.create_port_token(9000) for node in nodes] # Give every node its settings, the other nodes' tunnel addresses, and the scripts, and download what it needs def configure(i): node = nodes[i] peers = [f"{j + 1} {t.hostname} {t.token}" for j, t in enumerate(tunnels) if j != i] for name, text in [ ("cluster.env", f"NODE={i + 1}\nLAKE={lake.id}\nLAKE_TOKEN={lake_token}\n"), ("peers", "\n".join(peers) + "\n"), ]: with tempfile.NamedTemporaryFile("w") as f: f.write(text) f.flush() node.files.upload_file(f.name, f"/root/{name}") for file in ["install.sh", "node.sh", "buildings.py", "query.sql"]: node.files.upload_file(file, f"/root/{file}") install = node.exec("bash /root/install.sh", timeout_seconds=600) if install.exit_code != 0: raise RuntimeError(install.stderr) node.stop() with ThreadPoolExecutor(NODES) as pool: list(pool.map(configure, range(NODES))) print(f"Created {NODES} nodes") ``` In our runs, the setup script took 30 to 55 seconds. A stopped sandbox doesn't accrue compute charges, and it keeps its root filesystem, so the next start still has the scripts, the tunnel, and Delta Lake. ## Run the job This is the job. It reads only the four columns it needs from every building, counts the buildings in each 1-degree grid cell, with their average height where it's known, and saves the result as a Delta table. Spark spreads the files across every executor in the cluster: ```python buildings.py theme={null} import time from pyspark.sql import SparkSession, functions as F spark = SparkSession.builder.appName("buildings-by-cell").getOrCreate() start = time.time() buildings = spark.read.parquet("/mnt/overture/theme=buildings/type=building") # Count buildings in each 1-degree grid cell, with their average height where it's known by_cell = ( buildings .select( F.floor("bbox.xmin").cast("int").alias("lon"), F.floor("bbox.ymin").cast("int").alias("lat"), "subtype", "height", ) .groupBy("lat", "lon") .agg( F.count("*").alias("buildings"), F.count("height").alias("with_height"), F.round(F.avg("height"), 1).alias("avg_height_m"), F.count_if(F.col("subtype") == "residential").alias("residential"), ) ) by_cell.write.format("delta").mode("overwrite").save("/mnt/lake/tables/buildings_by_cell") print(f"Wrote buildings_by_cell in {time.time() - start:.0f} s") ``` The job script starts every node and its part of the cluster, waits for the workers to join the master, submits the job on the first node, and stops every node, so you pay for the nodes only while the job runs: ```typescript job.ts theme={null} import * as archil from "disk"; const nodes = (await archil.listSandboxes()) .filter((s) => /^spark-\d+$/.test(s.name ?? "")) .sort((a, b) => a.name!.localeCompare(b.name!, undefined, { numeric: true })); const main = nodes[0]; async function run(sandbox: archil.Sandbox, command: string): Promise { const result = await sandbox.exec(command, { timeoutSeconds: 3600 }); if (result.exitCode !== 0) throw new Error(`${sandbox.name}: ${result.stderr}`); return result.stdout.trim(); } try { // Start every node, and its part of the cluster, which runs until the node stops await Promise.all( nodes.map(async (node) => { if (node.status === "stopped") await node.start(); if (node.status === "paused") await node.resume(); await run(node, "rm -rf /tmp/spark-logs"); // logs from the last run await node.run("bash /root/node.sh", { collectOutput: false }); }), ); // Wait up to two minutes for every worker to join the master const joined = "grep -c 'Registering worker' /tmp/spark-logs/master.log || true"; for (let i = 0; (await run(main, joined)) !== String(nodes.length); i++) { if (i === 120) throw new Error(await run(main, "tail -20 /tmp/spark-logs/*.log")); await new Promise((r) => setTimeout(r, 1000)); } console.log(await run(main, "cd /root && spark-submit buildings.py 2>/dev/null")); } finally { // Stop every node, so you pay only while the job runs await Promise.all(nodes.map((node) => node.stop())); } ``` ```python job.py theme={null} import time from concurrent.futures import ThreadPoolExecutor import archil nodes = sorted( (s for s in archil.list_sandboxes() if (s.name or "").startswith("spark-") and s.name[6:].isdigit()), key=lambda s: int(s.name[6:]), ) main = nodes[0] def run(sandbox, command): result = sandbox.exec(command, timeout_seconds=3600) if result.exit_code != 0: raise RuntimeError(f"{sandbox.name}: {result.stderr}") return result.stdout.strip() # Start a node, and its part of the cluster, which runs until the node stops def start(node): if node.status == "stopped": node.start() if node.status == "paused": node.resume() run(node, "rm -rf /tmp/spark-logs") # logs from the last run node.run("bash /root/node.sh", collect_output=False) try: with ThreadPoolExecutor(len(nodes)) as pool: list(pool.map(start, nodes)) # Wait up to two minutes for every worker to join the master joined = "grep -c 'Registering worker' /tmp/spark-logs/master.log || true" for i in range(121): if run(main, joined) == str(len(nodes)): break if i == 120: raise RuntimeError(run(main, "tail -20 /tmp/spark-logs/*.log")) time.sleep(1) print(run(main, "cd /root && spark-submit buildings.py 2>/dev/null")) finally: # Stop every node, so you pay only while the job runs with ThreadPoolExecutor(len(nodes)) as pool: list(pool.map(lambda node: node.stop(), nodes)) ``` ``` Wrote buildings_by_cell in 32 s ``` Spark read only the four columns the job uses, about 26 GiB of the 258 GiB in the bucket, and wrote 16,192 grid cells. Run the script again: the second time, the executors read from Archil's cache instead of the bucket. | In our runs | First run | Later runs | | - | - | - | | The Spark job | 32 to 37 s | 16 to 19 s | | The whole script, including starting and stopping every node | 44 to 62 s | 28 to 33 s | Run the job from any scheduler that can run a script, such as cron, GitHub Actions, or Airflow. If the script dies before it stops the nodes, `idleTtlSeconds` pauses each node after 10 minutes without a connection, which also stops compute charges. ## Query the table The table is an ordinary Delta table, so Spark SQL can query it by path. This query finds the five grid cells with the most buildings: ```sql query.sql theme={null} SELECT lat, lon, buildings, avg_height_m, round(100 * residential / buildings) AS pct_residential FROM delta.`/mnt/lake/tables/buildings_by_cell` ORDER BY buildings DESC LIMIT 5; ``` One node is plenty for a query of the finished table. This script starts the first node, runs the query on it, and stops it: ```typescript query.ts theme={null} import * as archil from "disk"; const main = (await archil.listSandboxes()).find((s) => s.name === "spark-1")!; if (main.status === "stopped") await main.start(); if (main.status === "paused") await main.resume(); try { // One sandbox is plenty for a query of the finished table await main.exec("rm -rf /tmp/spark-logs"); await main.run("bash /root/node.sh", { collectOutput: false }); const query = await main.exec( "until grep -q 'Registering worker' /tmp/spark-logs/master.log; do sleep 1; done; " + "spark-sql -S -f /root/query.sql 2>/dev/null", ); if (query.exitCode !== 0) throw new Error(query.stderr); console.log(query.stdout); } finally { await main.stop(); } ``` ```python query.py theme={null} import archil main = next(s for s in archil.list_sandboxes() if s.name == "spark-1") if main.status == "stopped": main.start() if main.status == "paused": main.resume() try: # One sandbox is plenty for a query of the finished table main.exec("rm -rf /tmp/spark-logs") main.run("bash /root/node.sh", collect_output=False) query = main.exec( "until grep -q 'Registering worker' /tmp/spark-logs/master.log; do sleep 1; done; " "spark-sql -S -f /root/query.sql 2>/dev/null" ) if query.exit_code != 0: raise RuntimeError(query.stderr) print(query.stdout) finally: main.stop() ``` ``` -7 106 8265307 12.4 1.0 35 139 7720135 7.7 18.0 -24 -47 6989059 5.3 1.0 -8 110 6815862 7.6 1.0 22 88 6603065 15.6 0.0 ``` The busiest cells are Jakarta, Tokyo, São Paulo, Central Java, and Kolkata. The script took under 20 seconds, most of it starting the JVMs. Any sandbox, server, or cluster that mounts `spark-lake` can read the table too. ## Running it in production * **Size the cluster per job.** Change `NODES` in the setup script for more or fewer nodes. Because the nodes run only while a job runs, twice the nodes for half the time costs about the same, so give big jobs more nodes. A sandbox can have up to 32 vCPU and 64 GiB of memory, and `node.sh` sizes Spark to the sandbox. Spark writes shuffle and spill files to each node's root filesystem. * **Mark data immutable once it's final.** Overture publishes each release under its own prefix and never changes it. When you [back a disk with your own bucket](/data-sources/s3-object-storage), give each finished dataset, such as a dated partition, its own immutable disk, and keep data that's still being written on a regular disk. Objects your other systems add to a bucket can take minutes to appear on a disk; see [From the bucket to the disk](/details/consistency#from-the-bucket-to-the-disk). * **Share the lake.** Any number of clusters, servers, and sandboxes can mount `spark-lake` in conditional mode at the same time, and read or write tables there. * **Keep the tokens to the nodes.** The port tokens and the lake token are stored only on the nodes' root filesystems. Deleting a node revokes its port token, and removing the lake's token user revokes the lake token; see [Managing disks](/sdks/typescript#managing-disks). * **Pin your versions.** The nodes run the official [`spark` image](https://hub.docker.com/_/spark) with Spark 4.2.1, Delta Lake 4.4.1, and wstunnel 11.1.0. To start faster and control when they change, build an image with them installed; see [Images](/compute/sandboxes/images). ## Compared with Databricks serverless Databricks serverless compute includes the instances in its DBU rate, and Databricks chooses how much compute each job gets, so we measured it. We ran `buildings.py` as a Databricks serverless job on the Premium tier in AWS `us-east-2`, reading a copy of the same Overture release from an S3 bucket in that region, in both of its modes. We also ran a second job that reads every column of every building, 252 GB. Each Databricks job's cost is its DBU usage times the Premium list price of $0.35 per DBU. This cluster's cost is the time its four nodes ran, at Archil's list [sandbox rates](/administration/billing#persistent-sandboxes-and-services), $12.16 an hour for the four nodes. Each time is from start to finish, including startup: for this cluster, starting and stopping every node, submitting the job, and Spark's startup. A first run reads from the bucket; later runs on this cluster read what Archil cached, while Databricks serverless starts each run on fresh compute: | In our runs | First run | Later runs | | - | - | - | | **This guide's job, 26 GiB read** | | | | This cluster | 44 to 62 s, $0.15 to $0.21 | 28 to 33 s, \$0.10 | | Databricks serverless, performance optimized | 157 s, \$0.45 | 134 to 137 s, \$0.38 | | Databricks serverless, standard | 380 s, \$0.13 | | | **Every column, 252 GB read** | | | | This cluster | 64 s, \$0.22 | 63 s, \$0.21 | | Databricks serverless, performance optimized | 146 s, \$1.14 | 180 s, \$1.09 | | Databricks serverless, standard | 503 s, \$0.38 | | Archil samples running sandboxes once a minute, so a single run this short costs a minute of the nodes or nothing, and the time they ran on average. Against performance-optimized mode, the cluster finished 2 to 5 times sooner, for a fifth to a half of the cost. Against standard mode, it finished 6 to 14 times sooner, for 40% less on the scan of every column. Most of the difference is size: Databricks' query history shows that serverless ran these jobs 7 to 24 tasks at a time, while this cluster starts 128 vCPU on the job at once. To price your own serverless jobs, add up each job's rows in the `system.billing.usage` table, which Databricks fills in over the few hours after a job runs: ```sql theme={null} SELECT usage_metadata.job_id, sum(usage_quantity) AS dbus FROM system.billing.usage WHERE usage_metadata.job_id = '' GROUP BY 1; ``` Archil bills the disks separately, for [active data](/concepts/metering): the data read in the last hour. For a job like this one that runs every hour, the 26 GiB it reads is about \$5 a month; see [Pricing & Billing](/administration/billing). ## Clean up Delete the nodes and both disks: ```typescript cleanup.ts theme={null} import * as archil from "disk"; for (const sandbox of await archil.listSandboxes()) { if (!/^spark-\d+$/.test(sandbox.name ?? "")) continue; if (sandbox.status !== "stopped") await sandbox.stop(); await sandbox.delete(); } for (const disk of await archil.listDisks()) { if (["overture-maps", "spark-lake"].includes(disk.name)) await disk.delete(); } ``` ```python cleanup.py theme={null} import archil for sandbox in archil.list_sandboxes(): name = sandbox.name or "" if name.startswith("spark-") and name[6:].isdigit(): if sandbox.status != "stopped": sandbox.stop() sandbox.delete() for disk in archil.list_disks(): if disk.name in ("overture-maps", "spark-lake"): disk.delete() ``` Deleting `overture-maps` doesn't touch the public bucket. ## Next steps > [card] [Sandboxes](/compute/sandboxes/introduction) > > Create, start, and stop sandboxes, and reach their ports. > [card] [Sharing disks](/concepts/sharing-disks) > > Mount one disk on many sandboxes and servers, and choose between shared and conditional mode. > [card] [Analytics with DuckDB](/guides/analytics/duckdb) > > Query the same Overture data with DuckDB, and serve results over HTTPS. > [card] [ClickHouse](/guides/analytics/clickhouse) > > Run ClickHouse on an Archil disk, and branch the database for heavy queries. # MySQL Source: https://docs.archil.com/guides/apps/mysql Run MySQL in a sandbox with its data directory on an Archil disk, and connect to it from your machine with the mysql client. This guide runs [MySQL](https://www.mysql.com/) 8.4 in a [persistent sandbox](/compute/sandboxes/introduction) with its data directory on an Archil disk. MySQL runs as a supervised service, listens on a public hostname, and only accepts encrypted connections from a user with a password. You connect from your own machine with the `mysql` client, write some rows, stop the sandbox, and read them back. MySQL doesn't need to know anything about Archil. It reads and writes files in its data directory as usual, and Archil keeps them durable in object storage, so the database outlives any one sandbox. ## Prerequisites Export your Archil API key and a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`): ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 ``` Then install the SDK: ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` On your machine you also need the `mysql` client and [`socat`](http://www.dest-unreach.org/socat/), which carries the connection over TLS to the sandbox: ```bash macOS theme={null} brew install mysql-client socat export PATH="$(brew --prefix mysql-client)/bin:$PATH" ``` ```bash Ubuntu theme={null} sudo apt-get install -y mysql-client socat ``` Finally, generate a password. The setup script gives it to MySQL's `root` user and to an `app` user, and the `mysql` client reads it later: ```bash theme={null} export MYSQL_PASSWORD=$(openssl rand -hex 24) ``` ## Write the setup script This script runs inside the sandbox. The Archil disk is mounted by root, and MySQL runs as the `mysql` user, so the script hands the data directory over first. On an empty disk it then initializes MySQL with an init file that sets the `root` password, creates an `app` database, and creates an `app` user that can connect from anywhere with the password. Finally, it registers MySQL as a service, which restarts if it crashes and starts again whenever the sandbox boots. `--require-secure-transport` makes MySQL reject network connections that don't use TLS; the server generates its own certificate during initialization. ```bash setup.sh theme={null} #!/bin/bash set -euo pipefail DATADIR=/mnt/archil/mysql mkdir -p "$DATADIR" chown mysql:mysql "$DATADIR" if [ ! -d "$DATADIR/mysql" ]; then cat > /tmp/init.sql </dev/null; do sleep 1; done' ``` Because the script only initializes an empty data directory, you can run it on a new sandbox that mounts an existing `mysql-data` disk, and it just starts the server on the data that's already there. ## Start the database The next script creates a disk, creates a sandbox from the official `mysql:8.4` image with the disk mounted at `/mnt/archil`, uploads `setup.sh`, and runs it with your password. Then it exposes port 3306 publicly and prints the hostname. ```typescript create.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const { disk } = await archil.createDisk({ name: "mysql-data" }); const sandbox = await archil.createSandbox({ name: "mysql", baseImage: "mysql:8.4", vcpuCount: 2, memSizeMiB: 4096, mounts: [{ disk, path: "/mnt/archil" }], }); await sandbox.files.uploadFile(await readFile("setup.sh"), "/root/setup.sh", { mode: 0o755 }); const setup = await sandbox.exec("/root/setup.sh", { env: { MYSQL_PASSWORD: process.env.MYSQL_PASSWORD! }, }); if (setup.exitCode !== 0) throw new Error(setup.stderr); console.log(await sandbox.exposePort(3306)); ``` ```python create.py theme={null} import os import archil disk = archil.create_disk(name="mysql-data").disk sandbox = archil.create_sandbox( name="mysql", base_image="mysql:8.4", vcpu_count=2, mem_size_mib=4096, mounts=[archil.SandboxMountSpec(disk=disk, path="/mnt/archil")], ) sandbox.files.upload_file("setup.sh", "/root/setup.sh", mode=0o755) setup = sandbox.exec("/root/setup.sh", env={"MYSQL_PASSWORD": os.environ["MYSQL_PASSWORD"]}) if setup.exit_code != 0: raise RuntimeError(setup.stderr) print(sandbox.expose_port(3306)) ``` ```bash TypeScript theme={null} npx tsx create.ts ``` ```bash Python theme={null} python create.py ``` ```text theme={null} 3306-agquue2c3jzwhbjukocuwzr7d4.prod-aws-us-west-2-green.archil.app ``` > [!WARNING] > > An exposed port is reachable by anyone who knows the hostname. The `app` user accepts connections from any address, > so keep its password long and random, as generated above, and never create a remote MySQL user with a weak password > or none. ## Connect from your machine Archil serves public ports over TLS on port 443 with a publicly trusted certificate, and forwards the decrypted TCP stream to the sandbox. The `mysql` client can't open that kind of TLS connection on its own, so start a local `socat` tunnel that listens on port 13306, wraps each connection in TLS, and verifies Archil's certificate. Use the hostname that `create` printed: ```bash theme={null} export MYSQL_HOST=3306-agquue2c3jzwhbjukocuwzr7d4.prod-aws-us-west-2-green.archil.app socat TCP-LISTEN:13306,fork,reuseaddr OPENSSL:$MYSQL_HOST:443,verify=1 & ``` `socat` may print a warning that it doesn't check certificate revocation lists; the tunnel still verifies the certificate. Now connect through the tunnel as the `app` user. Use `127.0.0.1` rather than `localhost`, which the `mysql` client treats as a local socket. `--ssl-mode=REQUIRED` makes MySQL encrypt the stream inside the tunnel, too, so it stays encrypted all the way to the server: ```bash theme={null} export MYSQL_PWD=$MYSQL_PASSWORD mysql --host=127.0.0.1 --port=13306 --user=app --ssl-mode=REQUIRED --table app <<'SQL' CREATE TABLE notes (id INT AUTO_INCREMENT PRIMARY KEY, body TEXT NOT NULL); INSERT INTO notes (body) VALUES ('hello from my laptop'), ('stored on an Archil disk'); SELECT id, body FROM notes; SELECT @@datadir; SQL ``` ```text theme={null} +----+--------------------------+ | id | body | +----+--------------------------+ | 1 | hello from my laptop | | 2 | stored on an Archil disk | +----+--------------------------+ +--------------------+ | @@datadir | +--------------------+ | /mnt/archil/mysql/ | +--------------------+ ``` ## Stop it and let a connection wake it A stopped sandbox doesn't use compute, and a connection to its public port starts it again. Stop the sandbox: ```typescript stop.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "mysql")!; await sandbox.stop(); console.log("sandbox is", sandbox.status); ``` ```python stop.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "mysql") sandbox.stop() print("sandbox is", archil.get_sandbox(sandbox.id).status) ``` ```bash TypeScript theme={null} npx tsx stop.ts ``` ```bash Python theme={null} python stop.py ``` ```text theme={null} sandbox is stopped ``` Stopping sends `SIGTERM` to MySQL, which shuts down cleanly. Query the table again through the same tunnel: ```bash theme={null} mysql --host=127.0.0.1 --port=13306 --user=app --ssl-mode=REQUIRED --table app -e "SELECT id, body FROM notes;" ``` ```text theme={null} +----+--------------------------+ | id | body | +----+--------------------------+ | 1 | hello from my laptop | | 2 | stored on an Archil disk | +----+--------------------------+ ``` The connection booted the sandbox, which mounted the disk and started the `mysql` service before answering, in a few seconds. The rows were on the Archil disk the whole time. With MySQL's default settings, InnoDB flushes its redo log to the Archil disk on every commit, so a committed transaction survives a crash, too. ## Clean up Stop the tunnel, then stop and delete the sandbox and delete the disk: ```bash theme={null} kill %1 ``` ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "mysql"); await sandbox?.stop(); await sandbox?.delete(); const [disk] = await archil.listDisks({ name: "mysql-data" }); await disk?.delete(); ``` ```python cleanup.py theme={null} import archil for sandbox in archil.list_sandboxes(): if sandbox.name == "mysql": sandbox.stop() sandbox.delete() for disk in archil.list_disks(name="mysql-data"): disk.delete() ``` ## Next steps > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. > [card] [PostgreSQL](/guides/apps/postgres) > > Run PostgreSQL the same way, with its data directory on an Archil disk. > [card] [Sharing disks](/concepts/sharing-disks) > > Mount modes, and reading a disk from several machines. > [card] [TypeScript SDK](/sdks/typescript) > > Full reference for sandboxes, disks, and serverless execution. # PostgreSQL Source: https://docs.archil.com/guides/apps/postgres Run PostgreSQL in a sandbox with its data directory on an Archil disk, and connect to it from your machine with psql. This guide runs [PostgreSQL](https://www.postgresql.org/) in a [persistent sandbox](/compute/sandboxes/introduction) with its data directory on an Archil disk. Postgres runs as a supervised service, listens on a public hostname, and requires a password for every network connection. You connect from your own machine with `psql`, write some rows, stop the sandbox, and read them back. Postgres doesn't need to know anything about Archil. It reads and writes files in its data directory as usual, and Archil keeps them durable in object storage, so the database outlives any one sandbox. ## Prerequisites Export your Archil API key and a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`): ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 ``` Then install the SDK: ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` On your machine you also need `psql` and [`socat`](http://www.dest-unreach.org/socat/), which carries the connection over TLS to the sandbox. Any recent `psql` works. ```bash macOS theme={null} brew install libpq socat export PATH="$(brew --prefix libpq)/bin:$PATH" ``` ```bash Debian/Ubuntu theme={null} sudo apt-get install -y postgresql-client socat ``` Finally, generate a password for the `postgres` user. The setup script reads it from this variable, and so does `psql` later: ```bash theme={null} export POSTGRES_PASSWORD=$(openssl rand -hex 24) ``` ## Write the setup script This script runs inside the sandbox. The Archil disk is mounted by root, but Postgres refuses to run as root and requires its data directory to belong to the `postgres` user with mode `0700`, so the script hands the directory over first. On an empty disk it then initializes a cluster with a password for the `postgres` user, creates a self-signed certificate so connections are encrypted all the way to the server, and only accepts network connections that use SSL and a password. Finally, it registers Postgres as a service, which restarts if it crashes and starts again whenever the sandbox boots. ```bash setup.sh theme={null} #!/bin/bash set -euo pipefail PGDATA=/mnt/archil/pgdata mkdir -p "$PGDATA" chown postgres:postgres "$PGDATA" chmod 700 "$PGDATA" if [ ! -s "$PGDATA/PG_VERSION" ]; then echo "$POSTGRES_PASSWORD" > /tmp/pwfile chown postgres /tmp/pwfile gosu postgres initdb --pgdata="$PGDATA" --username=postgres --pwfile=/tmp/pwfile \ --auth-local=peer --auth-host=scram-sha-256 rm /tmp/pwfile openssl req -new -x509 -days 3650 -nodes -subj "/CN=postgres" \ -keyout "$PGDATA/server.key" -out "$PGDATA/server.crt" 2>/dev/null chown postgres:postgres "$PGDATA/server.key" "$PGDATA/server.crt" chmod 600 "$PGDATA/server.key" echo "listen_addresses = '*'" >> "$PGDATA/postgresql.conf" echo "ssl = on" >> "$PGDATA/postgresql.conf" echo "hostssl all all all scram-sha-256" >> "$PGDATA/pg_hba.conf" fi archil-sandbox services create postgres --tcp-port 5432 -- \ gosu postgres /usr/lib/postgresql/17/bin/postgres -D "$PGDATA" timeout 60 bash -c 'until gosu postgres pg_isready --quiet; do sleep 1; done' ``` Because the script only initializes an empty data directory, you can run it on a new sandbox that mounts an existing `postgres-data` disk, and it just starts the server on the data that's already there. ## Start the database The next script creates a disk, creates a sandbox from the official `postgres:17` image with the disk mounted at `/mnt/archil`, uploads `setup.sh`, and runs it with your password. Then it exposes port 5432 publicly and prints the hostname. ```typescript create.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const { disk } = await archil.createDisk({ name: "postgres-data" }); const sandbox = await archil.createSandbox({ name: "postgres", baseImage: "postgres:17", vcpuCount: 2, memSizeMiB: 4096, mounts: [{ disk, path: "/mnt/archil" }], }); await sandbox.files.uploadFile(await readFile("setup.sh"), "/root/setup.sh", { mode: 0o755 }); const setup = await sandbox.exec("/root/setup.sh", { env: { POSTGRES_PASSWORD: process.env.POSTGRES_PASSWORD! }, }); if (setup.exitCode !== 0) throw new Error(setup.stderr); console.log(await sandbox.exposePort(5432)); ``` ```python create.py theme={null} import os import archil disk = archil.create_disk(name="postgres-data").disk sandbox = archil.create_sandbox( name="postgres", base_image="postgres:17", vcpu_count=2, mem_size_mib=4096, mounts=[archil.SandboxMountSpec(disk=disk, path="/mnt/archil")], ) sandbox.files.upload_file("setup.sh", "/root/setup.sh", mode=0o755) setup = sandbox.exec("/root/setup.sh", env={"POSTGRES_PASSWORD": os.environ["POSTGRES_PASSWORD"]}) if setup.exit_code != 0: raise RuntimeError(setup.stderr) print(sandbox.expose_port(5432)) ``` ```bash TypeScript theme={null} npx tsx create.ts ``` ```bash Python theme={null} python create.py ``` ```text theme={null} 5432-agquud7y7fzstcgaybnhkhoov4.prod-aws-us-west-2-green.archil.app ``` > [!WARNING] > > An exposed port is reachable by anyone who knows the hostname. The setup script requires SSL and a password for every > network connection; keep the password long and random, as generated above, and never expose a Postgres user that has > a weak password or none. ## Connect from your machine Archil serves public ports over TLS on port 443 with a publicly trusted certificate, and forwards the decrypted TCP stream to the sandbox. `psql` can't open that kind of TLS connection on its own, so start a local `socat` tunnel that listens on port 15432, wraps each connection in TLS, and verifies Archil's certificate. Use the hostname that `create` printed: ```bash theme={null} export PG_HOST=5432-agquud7y7fzstcgaybnhkhoov4.prod-aws-us-west-2-green.archil.app socat TCP-LISTEN:15432,fork,reuseaddr OPENSSL:$PG_HOST:443,verify=1 & ``` `socat` may print a warning that it doesn't check certificate revocation lists; the tunnel still verifies the certificate. Now connect through the tunnel with your password. `sslmode=require` makes Postgres encrypt the stream inside the tunnel, too, so it stays encrypted all the way to the server: ```bash theme={null} export PGPASSWORD=$POSTGRES_PASSWORD psql "host=localhost port=15432 user=postgres sslmode=require" <<'SQL' CREATE TABLE notes (id serial PRIMARY KEY, body text NOT NULL); INSERT INTO notes (body) VALUES ('hello from my laptop'), ('stored on an Archil disk'); SELECT id, body FROM notes; SHOW data_directory; SQL ``` ```text theme={null} CREATE TABLE INSERT 0 2 id | body ----+-------------------------- 1 | hello from my laptop 2 | stored on an Archil disk (2 rows) data_directory -------------------- /mnt/archil/pgdata (1 row) ``` ## Restart it Stop the sandbox and start it again. Stopping sends `SIGTERM` to Postgres, which shuts down cleanly. Starting boots a fresh VM, which mounts the disk and starts the `postgres` service; the script waits until Postgres accepts connections. ```typescript restart.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "postgres")!; await sandbox.stop(); console.log("sandbox is", sandbox.status); await sandbox.start(); await sandbox.exec("timeout 60 bash -c 'until gosu postgres pg_isready --quiet; do sleep 1; done'"); console.log("sandbox is", sandbox.status); ``` ```python restart.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "postgres") sandbox.stop() print("sandbox is", archil.get_sandbox(sandbox.id).status) sandbox.start() sandbox.exec("timeout 60 bash -c 'until gosu postgres pg_isready --quiet; do sleep 1; done'") print("sandbox is", archil.get_sandbox(sandbox.id).status) ``` ```bash TypeScript theme={null} npx tsx restart.ts ``` ```bash Python theme={null} python restart.py ``` ```text theme={null} sandbox is stopped sandbox is running ``` Query the table again through the same tunnel: ```bash theme={null} psql "host=localhost port=15432 user=postgres sslmode=require" -c "SELECT id, body FROM notes;" ``` ```text theme={null} id | body ----+-------------------------- 1 | hello from my laptop 2 | stored on an Archil disk (2 rows) ``` The rows were on the Archil disk the whole time. > [!NOTE] > > You don't have to start a stopped sandbox yourself: a connection to its public port boots it, usually within a > couple of seconds. Postgres starts listening a moment before it's ready, though, so that first connection can fail > with `the database system is starting up`. Connect again and it succeeds. ## Benchmark it (optional) The `postgres:17` image includes `pgbench`. This script creates a 10-scale benchmark database (about 160 MB) and runs a 30-second read-write test with 8 clients inside the sandbox: ```typescript bench.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "postgres")!; const bench = await sandbox.exec( "gosu postgres createdb bench && gosu postgres pgbench -i -s 10 -q bench && gosu postgres pgbench -c 8 -j 2 -T 30 bench", ); console.log(bench.stdout); ``` ```python bench.py theme={null} import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "postgres") bench = sandbox.exec( "gosu postgres createdb bench && gosu postgres pgbench -i -s 10 -q bench && gosu postgres pgbench -c 8 -j 2 -T 30 bench", ) print(bench.stdout) ``` When the run finishes, look at the `tps` (transactions per second) and `latency average` lines of pgbench's summary. Every commit waits for `fsync` on the Archil disk, so Postgres keeps its usual durability guarantees. ## Clean up Stop the tunnel, then stop and delete the sandbox and delete the disk: ```bash theme={null} kill %1 ``` ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "postgres"); await sandbox?.stop(); await sandbox?.delete(); const [disk] = await archil.listDisks({ name: "postgres-data" }); await disk?.delete(); ``` ```python cleanup.py theme={null} import archil for sandbox in archil.list_sandboxes(): if sandbox.name == "postgres": sandbox.stop() sandbox.delete() for disk in archil.list_disks(name="postgres-data"): disk.delete() ``` ## Next steps > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. > [card] [MySQL](/guides/apps/mysql) > > Run MySQL the same way, with its data directory on an Archil disk. > [card] [Sharing disks](/concepts/sharing-disks) > > Mount modes, and reading a disk from several machines. > [card] [Python SDK](/sdks/python) > > Full reference for sandboxes, disks, and serverless execution. # Host a web app Source: https://docs.archil.com/guides/apps/web-app Run a Python web app with a SQLite database on an Archil disk, serve it at a public URL, and fork preview environments from a template. This guide deploys a small guestbook app: a Python HTTP server that stores messages in [SQLite](https://www.sqlite.org/). The code and the database live on an Archil disk mounted into a [persistent sandbox](/compute/sandboxes/introduction), the server runs as a supervised service, and you reach it at a public HTTPS URL. Then you fork preview environments that run a new version of the code, without touching production. SQLite doesn't need to know anything about Archil. It reads and writes a file on the mounted disk, and Archil keeps the data durable in object storage, so the database outlives any one sandbox. ## Prerequisites Export your Archil API key and a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`): ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 ``` Then install the SDK: ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk tsx ``` ```bash Python theme={null} pip install archil ``` ## Write the app The app uses only the Python standard library. `POST /` saves the request body as a message, and `GET /` lists every message under the app's version. At startup it switches the database to [write-ahead logging](https://www.sqlite.org/wal.html) (WAL), so readers don't block the writer. ```python app.py theme={null} import sqlite3 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer VERSION = "v1" def query(sql, *params): db = sqlite3.connect("guestbook.db", timeout=5) try: with db: return db.execute(sql, params).fetchall() finally: db.close() query("PRAGMA journal_mode=WAL") query("CREATE TABLE IF NOT EXISTS messages (id INTEGER PRIMARY KEY, body TEXT NOT NULL)") class Handler(BaseHTTPRequestHandler): def do_GET(self): rows = query("SELECT body FROM messages ORDER BY id") self.reply(200, f"guestbook {VERSION}\n" + "".join(f"- {body}\n" for (body,) in rows)) def do_POST(self): body = self.rfile.read(int(self.headers.get("Content-Length", 0))).decode() query("INSERT INTO messages (body) VALUES (?)", body) self.reply(201, "saved\n") def reply(self, status, text): data = text.encode() self.send_response(status) self.send_header("Content-Type", "text/plain; charset=utf-8") self.send_header("Content-Length", str(len(data))) self.end_headers() self.wfile.write(data) ThreadingHTTPServer(("0.0.0.0", 8080), Handler).serve_forever() ``` ## Deploy it The deploy script creates a disk for the app, creates a sandbox from the `python:3.13` image with the disk mounted at `/srv/app`, and uploads `app.py` onto the disk. It then registers the server as a service with `archil-sandbox services create`. Services run in the background, restart if they crash, and start again whenever the sandbox boots. Finally, the script exposes port 8080 publicly and prints the URL. ```typescript deploy.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const { disk } = await archil.createDisk({ name: "guestbook-data" }); const sandbox = await archil.createSandbox({ name: "guestbook", baseImage: "python:3.13", mounts: [{ disk, path: "/srv/app" }], }); await sandbox.files.uploadFile(await readFile("app.py"), "/srv/app/app.py"); await sandbox.exec("archil-sandbox services create web --tcp-port 8080 --dir /srv/app -- python3 app.py"); const hostname = await sandbox.exposePort(8080); console.log(`App URL: https://${hostname}`); ``` ```python deploy.py theme={null} import archil disk = archil.create_disk(name="guestbook-data").disk sandbox = archil.create_sandbox( name="guestbook", base_image="python:3.13", mounts=[archil.SandboxMountSpec(disk=disk, path="/srv/app")], ) sandbox.files.upload_file("app.py", "/srv/app/app.py") sandbox.exec("archil-sandbox services create web --tcp-port 8080 --dir /srv/app -- python3 app.py") hostname = sandbox.expose_port(8080) print(f"App URL: https://{hostname}") ``` ```bash TypeScript theme={null} npx tsx deploy.ts ``` ```bash Python theme={null} python deploy.py ``` ```text theme={null} App URL: https://8080-agquugxt3zyrdjwtv2avpwfa5m.prod-aws-us-west-2-green.archil.app ``` Save the printed URL, then sign the guestbook from your machine: ```bash theme={null} export APP_URL=https://8080-agquugxt3zyrdjwtv2avpwfa5m.prod-aws-us-west-2-green.archil.app curl --data "hello from production" $APP_URL curl $APP_URL ``` ```text theme={null} saved guestbook v1 - hello from production ``` > [!WARNING] > > `exposePort` makes the port reachable by anyone who has the URL. This guestbook accepts writes from anyone; a real > app needs its own authentication before you expose it. For programmatic clients, use a port token instead; see > [Hosting network services](/compute/sandboxes/introduction#hosting-network-services). ## Stop it and let traffic wake it A stopped sandbox doesn't use compute, and a request to its public URL starts it again. The next script lists the files on the disk and checks the database's journal mode, then stops the sandbox and requests its URL. ```typescript wake.ts theme={null} import * as archil from "disk"; const sandbox = (await archil.listSandboxes()).find((s) => s.name === "guestbook")!; const check = await sandbox.exec("ls && python3 -m sqlite3 guestbook.db 'PRAGMA journal_mode'", { cwd: "/srv/app", }); console.log(check.stdout); await sandbox.stop(); const [{ hostname }] = await sandbox.listPorts(); const response = await fetch(`https://${hostname}/`); console.log(response.status, await response.text()); console.log("sandbox is", (await archil.getSandbox(sandbox.id)).status); ``` ```python wake.py theme={null} import urllib.request import archil sandbox = next(s for s in archil.list_sandboxes() if s.name == "guestbook") check = sandbox.exec("ls && python3 -m sqlite3 guestbook.db 'PRAGMA journal_mode'", cwd="/srv/app") print(check.stdout) sandbox.stop() hostname = sandbox.list_ports()[0].hostname with urllib.request.urlopen(f"https://{hostname}/") as response: print(response.status, response.read().decode()) print("sandbox is", archil.get_sandbox(sandbox.id).status) ``` ```text theme={null} app.py guestbook.db ('wal',) 200 guestbook v1 - hello from production sandbox is running ``` The database file sits on the Archil disk next to the code, in WAL mode. SQLite keeps its `-wal` and `-shm` files beside it while connections are open. The request booted the sandbox, which mounted the disk and started the `web` service before answering, typically in a second or two. Nothing in the app had to handle the restart. A plain mount like this one gives the sandbox exclusive access to the disk, so SQLite's locking works exactly as it does on a local disk. Don't open the same database file from several sandboxes at once: SQLite's locks don't extend across machines. To read the data from elsewhere while the app runs, see [Sharing disks](/concepts/sharing-disks). ## Create preview environments To try a change before it reaches production, give it its own sandbox. Rather than setting each one up from scratch, keep a template sandbox with the app already installed and fork it: a fork starts from a copy of the template's root filesystem, with every package, file, and service definition in place. The template keeps the app and its database on its root filesystem rather than on an Archil disk. A fork mounts the same disks as its source, and a plain mount can be held by only one sandbox at a time, so a template with a disk couldn't run more than one preview. Previews are disposable, so their data doesn't need to outlive them. Production isn't involved at all: it keeps running on `guestbook-data` while you create and delete previews. Create the template once. It runs the same `app.py` as a service, then stops; a stopped sandbox doesn't use compute. ```typescript template.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const template = await archil.createSandbox({ name: "guestbook-template", baseImage: "python:3.13" }); await template.exec("mkdir -p /srv/app"); await template.files.uploadFile(await readFile("app.py"), "/srv/app/app.py"); await template.exec("archil-sandbox services create web --tcp-port 8080 --dir /srv/app -- python3 app.py"); await template.stop(); ``` ```python template.py theme={null} import archil template = archil.create_sandbox(name="guestbook-template", base_image="python:3.13") template.exec("mkdir -p /srv/app") template.files.upload_file("app.py", "/srv/app/app.py") template.exec("archil-sandbox services create web --tcp-port 8080 --dir /srv/app -- python3 app.py") template.stop() ``` Each preview is a fork of the template. The fork boots with the `web` service running; the script deploys a change to it, exposes its port, and prints its URL. The `sed` command stands in for deploying a new version of `app.py`. ```typescript preview.ts theme={null} import * as archil from "disk"; const template = (await archil.listSandboxes()).find((s) => s.name === "guestbook-template")!; const preview = await template.fork({ name: "guestbook-preview" }); await preview.exec( `sed -i 's/^VERSION = .*/VERSION = "v2"/' app.py && archil-sandbox services restart web`, { cwd: "/srv/app" }, ); const hostname = await preview.exposePort(8080); console.log(`Preview URL: https://${hostname}`); ``` ```python preview.py theme={null} import archil template = next(s for s in archil.list_sandboxes() if s.name == "guestbook-template") preview = template.fork(name="guestbook-preview") preview.exec( """sed -i 's/^VERSION = .*/VERSION = "v2"/' app.py && archil-sandbox services restart web""", cwd="/srv/app", ) hostname = preview.expose_port(8080) print(f"Preview URL: https://{hostname}") ``` ```bash TypeScript theme={null} npx tsx template.ts npx tsx preview.ts ``` ```bash Python theme={null} python template.py python preview.py ``` Sign the preview's guestbook, then compare the two environments: ```bash theme={null} export PREVIEW_URL=https://8080-agquugzjsr6ilkopelwwgkhslq.prod-aws-us-west-2-green.archil.app curl --data "hello from the preview" $PREVIEW_URL curl $PREVIEW_URL curl $APP_URL ``` ```text theme={null} saved guestbook v2 - hello from the preview guestbook v1 - hello from production ``` The preview runs the new code with its own database, and production is untouched. Fork the template again, with a different name, for each branch you want to preview. ## Clean up A fork must be deleted before the sandbox it was forked from, and a sandbox must be stopped before it's deleted. Delete the preview, the template, and production, then production's disk. ```typescript cleanup.ts theme={null} import * as archil from "disk"; for (const name of ["guestbook-preview", "guestbook-template", "guestbook"]) { const sandbox = (await archil.listSandboxes()).find((s) => s.name === name); await sandbox?.stop(); await sandbox?.delete(); } const [disk] = await archil.listDisks({ name: "guestbook-data" }); await disk?.delete(); ``` ```python cleanup.py theme={null} import archil for name in ["guestbook-preview", "guestbook-template", "guestbook"]: for sandbox in archil.list_sandboxes(): if sandbox.name == name: sandbox.stop() sandbox.delete() for disk in archil.list_disks(name="guestbook-data"): disk.delete() ``` ## Next steps > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. > [card] [Coding agent in a sandbox](/guides/agents/coding-agent) > > Have an agent build and run an app like this one in a sandbox. > [card] [Sharing disks](/concepts/sharing-disks) > > Mount modes, delegations, and reading a disk from several machines. > [card] [TypeScript SDK](/sdks/typescript) > > Full reference for sandboxes, disks, and serverless execution. # Google Cloud Run Source: https://docs.archil.com/guides/platforms/cloud-run Mount an Archil disk in a Cloud Run service so its files outlive every instance and revision. [Cloud Run](https://cloud.google.com/run/docs/overview/what-is-cloud-run) runs your container on instances that come and go as traffic changes, and anything an instance writes to its own filesystem is gone when it shuts down. Mount an Archil disk in the container and those files stay on the disk, where every other instance, the next revision, and the Archil SDK can read them. Cloud Run's second-generation execution environment supports FUSE, so the disk mounts as a regular directory with no extra permissions. Mount it when the container starts and unmount it when Cloud Run sends `SIGTERM`. ## Prerequisites * The [gcloud CLI](https://cloud.google.com/sdk/docs/install), signed in to a project with billing enabled. * An Archil disk, ideally in the Archil region closest to your Cloud Run region, such as `gcp-us-central1` for `us-central1`. See [Region availability](/reference/regions). Set your Archil API key and region: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=gcp-us-central1 ``` Create the disk with the [`disk` CLI](/reference/disk-cli). The output includes the disk ID and a one-time [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only: ```bash theme={null} npx disk create cloud-run-data ``` Export both for the steps below, along with your Google Cloud project and region: ```bash theme={null} export ARCHIL_DISK_ID=dsk-... export ARCHIL_MOUNT_TOKEN=adt_... export PROJECT=$(gcloud config get-value project) export REGION=us-central1 ``` ## Set up the project Enable the APIs the steps below use. Cloud Run builds and runs the service as the Compute Engine default service account unless you choose another, so give that account permission to build the image, and store the mount token in Secret Manager where only that account can read it, rather than in the service's environment variables: ```bash theme={null} gcloud services enable run.googleapis.com cloudbuild.googleapis.com secretmanager.googleapis.com SERVICE_ACCOUNT=$(gcloud projects describe "$PROJECT" --format='value(projectNumber)')-compute@developer.gserviceaccount.com gcloud projects add-iam-policy-binding "$PROJECT" \ --member "serviceAccount:$SERVICE_ACCOUNT" --role roles/run.builder --condition None printf %s "$ARCHIL_MOUNT_TOKEN" | gcloud secrets create archil-mount-token --data-file=- gcloud secrets add-iam-policy-binding archil-mount-token \ --member "serviceAccount:$SERVICE_ACCOUNT" --role roles/secretmanager.secretAccessor ``` ## Build the service The image needs the `archil` client and your app: ```dockerfile Dockerfile theme={null} FROM python:3.12-slim RUN apt-get update && apt-get install -y --no-install-recommends curl ca-certificates \ && curl -fsSL https://archil.com/install | sh \ && rm -rf /var/lib/apt/lists/* WORKDIR /app COPY server.py . CMD ["python3", "-u", "server.py"] ``` `server.py` mounts the disk before it starts serving, appends a line to a file on the disk for every request, and unmounts the disk when Cloud Run sends `SIGTERM`. It mounts with `--conditional` because more than one instance can have the disk mounted at once: Cloud Run scales out under load, and a new revision starts before the old one stops. ```python server.py theme={null} import os import signal import subprocess from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from threading import Thread MOUNTPOINT = "/mnt/archil" class App(BaseHTTPRequestHandler): def do_GET(self): with open(f"{MOUNTPOINT}/visits.log", "a+") as log: log.write(f"Hello from {os.environ['K_REVISION']}\n") log.flush() os.fsync(log.fileno()) log.seek(0) content = log.read().encode() self.send_response(200) self.send_header("Content-Length", str(len(content))) self.end_headers() self.wfile.write(content) os.makedirs(MOUNTPOINT, exist_ok=True) subprocess.run(["archil", "mount", os.environ["ARCHIL_DISK_ID"], MOUNTPOINT, "--region", os.environ["ARCHIL_REGION"], "--conditional"], check=True) server = ThreadingHTTPServer(("0.0.0.0", int(os.environ.get("PORT", "8080"))), App) signal.signal(signal.SIGTERM, lambda *_: Thread(target=server.shutdown).start()) server.serve_forever() subprocess.run(["archil", "unmount", MOUNTPOINT], check=True) ``` Deploy from the directory that holds both files. Cloud Build builds the image, and `--execution-environment gen2` selects the environment that supports FUSE: ```bash theme={null} gcloud run deploy archil-app --source . --region "$REGION" \ --execution-environment gen2 \ --set-env-vars "ARCHIL_DISK_ID=$ARCHIL_DISK_ID,ARCHIL_REGION=$ARCHIL_REGION" \ --set-secrets ARCHIL_MOUNT_TOKEN=archil-mount-token:latest \ --no-allow-unauthenticated ``` ## Use the disk Request the service with your own identity token: ```bash theme={null} URL=$(gcloud run services describe archil-app --region "$REGION" --format 'value(status.url)') curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" "$URL" ``` ```text theme={null} Hello from archil-app-00001-xyz ``` Deploy a new revision, for example after a code change, and request the service again. The new revision mounts the same disk and finds the line the first one wrote: ```bash theme={null} gcloud run services update archil-app --region "$REGION" --update-env-vars RELEASE=2 curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" "$URL" ``` ```text theme={null} Hello from archil-app-00001-xyz Hello from archil-app-00002-abc ``` ## How the disk behaves on Cloud Run **Mount with `--conditional`.** Every instance of every revision can then read and write anywhere on the disk. For faster writes to a directory only one instance uses, check it out with [`archil checkout`](/reference/archil-cli#checkout); see [Sharing disks](/concepts/sharing-disks). **Unmount on `SIGTERM`.** Unmounting sends the client's buffered writes to the disk. Cloud Run sends `SIGTERM` before it shuts an instance down, and waits 10 seconds before stopping it. ## Clean up Delete the service, the secret, the service's image, and the disk once you no longer need them: ```bash theme={null} gcloud run services delete archil-app --region "$REGION" gcloud secrets delete archil-mount-token gcloud artifacts docker images delete "$REGION-docker.pkg.dev/$PROJECT/cloud-run-source-deploy/archil-app" --delete-tags npx disk delete "$ARCHIL_DISK_ID" ``` ## Next steps * [Python SDK](/sdks/python): read, write, and search the disk without mounting it. * [Sharing disks](/concepts/sharing-disks): how conditional mode and checkouts work. * [`archil` CLI reference](/reference/archil-cli): every `archil mount` option. # Cloudflare Containers Source: https://docs.archil.com/guides/platforms/cloudflare-containers Mount an Archil disk in a Cloudflare Container so its files outlive the container. A [Cloudflare Container](https://developers.cloudflare.com/containers/) runs your image in a VM that belongs to a Durable Object, which starts it, runs commands in it, and stops it after a period of inactivity. When the container stops, its disk is thrown away. Mount an Archil disk in the container and anything written there stays on the disk, where the next container can mount it and your app can read it with the Archil SDK. Cloudflare Containers support FUSE, so the disk mounts as a regular directory. The Durable Object mounts the disk when it starts the container, and the container unmounts it when Cloudflare stops it. The same approach works with the [Sandbox SDK](https://developers.cloudflare.com/sandbox/), which runs on the same Container API. ## Prerequisites * A Cloudflare account on the Workers Paid plan, which Containers require. * [Docker](https://docs.docker.com/get-started/get-docker/) running locally. Wrangler uses it to build the image. * An Archil disk in the [Archil region](/reference/regions) of your choice. Set your Archil API key and region: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 ``` Create the disk with the [`disk` CLI](/reference/disk-cli). The output includes the disk ID and a one-time [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only: ```bash theme={null} npx disk create container-data ``` Export both for the steps below: ```bash theme={null} export ARCHIL_DISK_ID=dsk-... export ARCHIL_MOUNT_TOKEN=adt_... ``` ## Create the project Create a directory for the project and install Wrangler: ```bash theme={null} mkdir archil-container && cd archil-container npm init -y npm install --save-dev wrangler ``` The project needs four files: the image, a script the container runs, the Worker, and Wrangler's configuration. The image installs the `archil` client. Cloudflare runs `linux/amd64` images, and the installer picks the matching build: ```dockerfile Dockerfile theme={null} FROM debian:12-slim RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl \ && curl -fsSL https://archil.com/install | sh \ && rm -rf /var/lib/apt/lists/* \ && mkdir -p /mnt/archil COPY start.sh /usr/local/bin/start.sh CMD ["sh", "/usr/local/bin/start.sh"] ``` The container's main process waits for Cloudflare's `SIGTERM` and unmounts the disk before it exits, which sends every buffered write to the disk and releases it: ```sh start.sh theme={null} trap 'archil unmount /mnt/archil; exit 0' TERM sleep infinity & wait ``` The Worker routes each request to a Durable Object by name. The Durable Object starts its container, mounts the disk with the token from a Worker secret, and then runs a command in the mounted directory. The command appends a line to a file on the disk and prints the file: ```ts src/index.ts theme={null} import { DurableObject } from "cloudflare:workers"; interface Env { WORKSPACE: DurableObjectNamespace; ARCHIL_DISK_ID: string; ARCHIL_REGION: string; ARCHIL_MOUNT_TOKEN: string; } const MOUNTPOINT = "/mnt/archil"; const decoder = new TextDecoder(); export class Workspace extends DurableObject { private mounted?: Promise; async run(argv: string[], env: Record = {}) { const container = this.ctx.container!; if (!container.running) { container.start({ image: container.images.workspace, instance: "standard-1", enableInternet: true }); this.mounted = undefined; await container.setInactivityTimeout(10 * 60 * 1000); } this.mounted ??= this.mount(container).catch((error) => { this.mounted = undefined; throw error; }); await this.mounted; const process = await container.exec(argv, { cwd: MOUNTPOINT, env }); const output = await process.output(); return { exitCode: output.exitCode, stdout: decoder.decode(output.stdout) }; } async stop() { const container = this.ctx.container!; if (!container.running) return; const exited = container.monitor().catch(() => {}); container.signal(15); await exited; } private async mount(container: Container) { const process = await container.exec( ["sh", "-c", `mountpoint -q ${MOUNTPOINT} || archil mount "$ARCHIL_DISK_ID" ${MOUNTPOINT} --region "$ARCHIL_REGION"`], { env: { ARCHIL_DISK_ID: this.env.ARCHIL_DISK_ID, ARCHIL_REGION: this.env.ARCHIL_REGION, ARCHIL_MOUNT_TOKEN: this.env.ARCHIL_MOUNT_TOKEN, }, }, ); const output = await process.output(); if (output.exitCode !== 0) { throw new Error(`archil mount failed: ${decoder.decode(output.stdout)}${decoder.decode(output.stderr)}`); } } } export default { async fetch(request, env) { // Authenticate the request here, then pick the workspace for this user or task. const name = new URL(request.url).pathname.slice(1) || "default"; const workspace = env.WORKSPACE.getByName(name); if (request.method === "DELETE") { await workspace.stop(); return new Response(`Stopped ${name}\n`); } const { stdout } = await workspace.run( ["sh", "-c", 'echo "Hello from $NAME" >> visits.log && cat visits.log'], { NAME: name }, ); return new Response(stdout); }, } satisfies ExportedHandler; ``` `wrangler.jsonc` binds the Durable Object to the image and sets the disk ID and region: ```jsonc wrangler.jsonc theme={null} { "name": "archil-container", "main": "src/index.ts", "compatibility_date": "2026-09-02", "durable_objects": { "bindings": [{ "name": "WORKSPACE", "class_name": "Workspace" }] }, "containers": [ { "class_name": "Workspace", "scheduling_policy": "durable_object", "images": { "workspace": { "dockerfile": "./Dockerfile" } } } ], "exports": { "Workspace": { "type": "durable-object", "storage": "sqlite" } }, "vars": { "ARCHIL_DISK_ID": "dsk-...", "ARCHIL_REGION": "aws-us-east-1" } } ``` Replace the two values under `vars` with your disk ID and region. ## Deploy and try it Log in, deploy, and store the mount token as a Worker secret. The first deploy builds the image and pushes it to Cloudflare's registry: ```bash theme={null} npx wrangler login npx wrangler deploy printf '%s' "$ARCHIL_MOUNT_TOKEN" | npx wrangler secret put ARCHIL_MOUNT_TOKEN ``` The deploy prints the Worker's URL. Request a workspace by name: ```bash theme={null} export WORKER_URL=https://archil-container..workers.dev curl "$WORKER_URL/alice" ``` ```text theme={null} Hello from alice ``` The first request to a workspace starts its container and mounts the disk, so it takes a few seconds. Stop the container. The Durable Object sends it `SIGTERM`, `start.sh` unmounts the disk, and the request returns once the container has exited: ```bash theme={null} curl -X DELETE "$WORKER_URL/alice" ``` Request a different workspace. Its container mounts the same disk and finds the line the first one wrote: ```bash theme={null} curl "$WORKER_URL/bob" ``` ```text theme={null} Hello from alice Hello from bob ``` ## How the disk behaves in a container **One container holds the disk at a time.** A plain mount is exclusive. To mount the same disk in several containers at once, add `--conditional` to the `archil mount` command. Every container can then read and write anywhere on the disk; see [Sharing disks](/concepts/sharing-disks). To give each user their own storage instead, create a disk per user and pass its ID and token to that user's Durable Object. **Unmount when the container stops.** Cloudflare sends a container `SIGTERM` when its inactivity timeout expires, and `stop()` sends it with `signal()`. Unmounting in the `SIGTERM` handler sends the client's buffered writes to the disk and releases it for the next container. Stop containers with `signal()` rather than `destroy()`, which doesn't give the container time to unmount. If a container ever goes away with the disk still mounted, see [Release a stale lock](/guides/platforms/troubleshooting#release-a-stale-lock). ## Clean up Delete the Worker, its container application and image, and the disk once you no longer need them. `wrangler delete` leaves the container application and its image behind, so delete those by ID and tag: ```bash theme={null} npx wrangler delete npx wrangler containers list npx wrangler containers delete npx wrangler containers images list npx wrangler containers images delete : npx disk delete "$ARCHIL_DISK_ID" ``` ## Next steps * [TypeScript SDK](/sdks/typescript): read, write, and search the disk without mounting it. * [Sharing disks](/concepts/sharing-disks): mount one disk in many containers at once. * [`archil` CLI reference](/reference/archil-cli): every `archil mount` option. # Daytona Source: https://docs.archil.com/guides/platforms/daytona Already on Daytona? Mount an Archil disk in your Daytona sandboxes so agent workspaces outlive the sandbox and stay portable. [Daytona](https://daytona.io) sandboxes support FUSE, so an Archil disk mounts as a regular directory in a default sandbox, with no privileged mode or custom snapshot needed. Already running agents on Daytona? Keep their data portable. When you delete a Daytona sandbox, its files go with it. Mount an Archil disk in the sandbox and anything written there stays on the disk, where the next sandbox can mount it, your app can read it with the Archil SDK, and an [Archil sandbox](/compute/sandboxes/introduction) can boot with it attached. > [!NOTE] > > Archil also runs sandboxes itself. [Archil sandboxes](/compute/sandboxes/introduction) are Firecracker microVMs whose root > filesystem is an Archil disk, with an egress firewall and forks that branch the root disk at any size. This guide is > for teams that already run agents on Daytona and want their workspace data on storage they can take with them. ## Prerequisites Daytona sandboxes meet the [requirements for mounting](/mounting/linux#requirements): they provide `/dev/fuse`, and `archil` runs as root with `sudo`. If you restrict the sandbox's outbound traffic, allow TCP 8100 and 32050. Export your Archil API key and region, and your [Daytona API key](https://app.daytona.io/dashboard/keys), then install both SDKs: ```bash TypeScript theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export DAYTONA_API_KEY=dtn_... npm init -y && npm pkg set type=module npm install disk @daytona/sdk tsx ``` ```bash Python theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export DAYTONA_API_KEY=dtn_... pip install archil daytona ``` ## Mount a disk First, create a disk. You only do this once: ```typescript setup.ts theme={null} import * as archil from "disk"; const { disk, token } = await archil.createDisk({ name: "daytona-data" }); console.log(`export ARCHIL_DISK_ID=${disk.id} ARCHIL_MOUNT_TOKEN=${token}`); ``` ```python setup.py theme={null} import archil result = archil.create_disk(name="daytona-data") print(f"export ARCHIL_DISK_ID={result.disk.id} ARCHIL_MOUNT_TOKEN={result.token}") ``` Run it with `eval "$(npx tsx setup.ts)"` (or `eval "$(python setup.py)"`) to export the disk ID and the disk's [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only. Now start a sandbox, install the `archil` client, mount the disk at `/mnt/archil` with `sudo`, and write a file. The token is passed only to the mount command: ```typescript mount.ts theme={null} import { Daytona } from "@daytona/sdk"; const { ARCHIL_DISK_ID, ARCHIL_REGION, ARCHIL_MOUNT_TOKEN } = process.env; const sandbox = await new Daytona().create(); async function run(command: string, env?: Record) { const { exitCode, result } = await sandbox.process.executeCommand(command, undefined, env); if (exitCode !== 0) throw new Error(result); return result; } try { await run("sudo apt-get update -qq && curl -fsSL https://archil.com/install | sh > /dev/null"); await run( `sudo mkdir -p /mnt/archil && sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount ${ARCHIL_DISK_ID} /mnt/archil --region ${ARCHIL_REGION}`, { ARCHIL_MOUNT_TOKEN: ARCHIL_MOUNT_TOKEN! }, ); try { console.log(await run(`echo "Hello from ${sandbox.id}" | sudo tee -a /mnt/archil/hello.txt > /dev/null && cat /mnt/archil/hello.txt`)); } finally { // Flush buffered writes and release the disk before the sandbox is deleted await run("sudo archil sync /mnt/archil && sudo archil checkin /mnt/archil"); } } finally { await sandbox.delete(); } ``` ```python mount.py theme={null} import os from daytona import Daytona sandbox = Daytona().create() def run(command, env=None): response = sandbox.process.exec(command, env=env) if response.exit_code != 0: raise RuntimeError(response.result) return response.result try: run("sudo apt-get update -qq && curl -fsSL https://archil.com/install | sh > /dev/null") run( f"sudo mkdir -p /mnt/archil && sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount {os.environ['ARCHIL_DISK_ID']} /mnt/archil --region {os.environ['ARCHIL_REGION']}", env={"ARCHIL_MOUNT_TOKEN": os.environ["ARCHIL_MOUNT_TOKEN"]}, ) try: print(run(f'echo "Hello from {sandbox.id}" | sudo tee -a /mnt/archil/hello.txt > /dev/null && cat /mnt/archil/hello.txt')) finally: # Flush buffered writes and release the disk before the sandbox is deleted run("sudo archil sync /mnt/archil && sudo archil checkin /mnt/archil") finally: sandbox.delete() ``` The `archil` client buffers recent writes, so before the script deletes the sandbox, `archil sync` sends every buffered write to the disk and `archil checkin` releases the sandbox's lock on it. Those are the two steps `archil unmount` takes on other machines, but Daytona doesn't let the client detach the mount. Run `npx tsx mount.ts` (or `python mount.py`) twice. Each run starts a new sandbox, and the second one finds the line the first one wrote: ```text theme={null} Hello from 4fa2e3c5-2478-48e3-8a90-ff388946f767 Hello from 4fa2e3c5-2478-48e3-8a90-ff388946f767 Hello from 3f962e2a-1ee7-46e4-a683-5f3892d6643a ``` A plain mount is exclusive: one sandbox holds the disk at a time. To mount it in several sandboxes at once, add `--conditional`, so every sandbox can read and write without a checkout. See [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents) for how to hand files from one sandbox to another, and [Consistency](/details/consistency) for when each one sees the others' writes. To skip the install in every sandbox, run the same install command in a [Daytona snapshot](https://www.daytona.io/docs/en/snapshots) and create your sandboxes from it. ## Read it from anywhere Both sandboxes are gone, and the file is still on the disk. Read it from your own machine or app with the SDK: ```typescript read.ts theme={null} import * as archil from "disk"; const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!); console.log(new TextDecoder().decode(await disk.getObject("hello.txt"))); ``` ```python read.py theme={null} import os import archil disk = archil.get_disk(os.environ["ARCHIL_DISK_ID"]) print(disk.get_object("hello.txt").decode()) ``` If a sandbox is deleted or stopped before it releases the disk, see [Troubleshooting mounts on other platforms](/guides/platforms/troubleshooting). ## Clean up Delete the disk. The script already deleted each sandbox. ```bash TypeScript theme={null} npx disk delete "$ARCHIL_DISK_ID" ``` ```bash Python theme={null} python -c 'import os, archil; archil.get_disk(os.environ["ARCHIL_DISK_ID"]).delete()' ``` ## Next steps * [`archil` CLI](/reference/archil-cli): every mount option, plus `sync` and `checkin`. * [Persistent sandboxes](/compute/sandboxes/introduction): Archil's own sandboxes, with the disk mounted for you. # E2B Source: https://docs.archil.com/guides/platforms/e2b Already on E2B? Mount an Archil disk in your E2B sandboxes so agent workspaces outlive the sandbox and stay portable. Already running agents on [E2B](https://e2b.dev)? Keep their data portable. When an E2B sandbox is killed or times out, its files go with it. Mount an Archil disk in the sandbox and anything written there stays on the disk, where the next sandbox can mount it, your app can read it with the Archil SDK, and an [Archil sandbox](/compute/sandboxes/introduction) can boot with it attached. E2B sandboxes support FUSE, so the disk mounts as a regular directory. > [!NOTE] > > Archil also runs sandboxes itself. [Archil sandboxes](/compute/sandboxes/introduction) are Firecracker microVMs whose root > filesystem is an Archil disk, with an egress firewall and forks that branch the root disk at any size. This guide is > for teams that already run agents on E2B and want their workspace data on storage they can take with them. ## Prerequisites E2B sandboxes meet the [requirements for mounting](/mounting/linux#requirements): they provide `/dev/fuse`, and `archil` runs as root with `user: "root"`. If you restrict the sandbox's outbound traffic, allow TCP 8100 and 32050. Export your Archil API key and region, and your [E2B API key](https://e2b.dev/dashboard?tab=keys), then install both SDKs: ```bash TypeScript theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export E2B_API_KEY=e2b_... npm init -y && npm pkg set type=module npm install disk e2b tsx ``` ```bash Python theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export E2B_API_KEY=e2b_... pip install archil e2b ``` ## Mount a disk First, build an E2B template with the `archil` client installed and create a disk. You only do this once: ```typescript setup.ts theme={null} import * as archil from "disk"; import { Template } from "e2b"; const template = Template() .fromBaseImage() .runCmd("apt-get update && curl -fsSL https://archil.com/install | sh", { user: "root" }); await Template.build(template, "archil"); const { disk, token } = await archil.createDisk({ name: "e2b-data" }); console.log(`export ARCHIL_DISK_ID=${disk.id} ARCHIL_MOUNT_TOKEN=${token}`); ``` ```python setup.py theme={null} import archil from e2b import Template template = ( Template() .from_base_image() .run_cmd("apt-get update && curl -fsSL https://archil.com/install | sh", user="root") ) Template.build(template, "archil") result = archil.create_disk(name="e2b-data") print(f"export ARCHIL_DISK_ID={result.disk.id} ARCHIL_MOUNT_TOKEN={result.token}") ``` Run it with `eval "$(npx tsx setup.ts)"` (or `eval "$(python setup.py)"`) to export the disk ID and the disk's [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only. Now start a sandbox from the template, mount the disk at `/mnt/archil`, and write a file. `archil` needs root, so these commands run with `user: "root"`, and the token is passed only to the mount command: ```typescript mount.ts theme={null} import { Sandbox } from "e2b"; const { ARCHIL_DISK_ID, ARCHIL_REGION, ARCHIL_MOUNT_TOKEN } = process.env; const sandbox = await Sandbox.create("archil"); try { await sandbox.commands.run( `mkdir -p /mnt/archil && archil mount ${ARCHIL_DISK_ID} /mnt/archil --region ${ARCHIL_REGION}`, { user: "root", envs: { ARCHIL_MOUNT_TOKEN: ARCHIL_MOUNT_TOKEN! } }, ); try { const { stdout } = await sandbox.commands.run( `echo "Hello from ${sandbox.sandboxId}" >> /mnt/archil/hello.txt && cat /mnt/archil/hello.txt`, { user: "root" }, ); console.log(stdout); } finally { // Flush buffered writes and release the disk before the sandbox goes away await sandbox.commands.run("archil unmount /mnt/archil", { user: "root" }); } } finally { await sandbox.kill(); } ``` ```python mount.py theme={null} import os from e2b import Sandbox sandbox = Sandbox.create("archil") try: sandbox.commands.run( f"mkdir -p /mnt/archil && archil mount {os.environ['ARCHIL_DISK_ID']} /mnt/archil --region {os.environ['ARCHIL_REGION']}", user="root", envs={"ARCHIL_MOUNT_TOKEN": os.environ["ARCHIL_MOUNT_TOKEN"]}, ) try: result = sandbox.commands.run( f'echo "Hello from {sandbox.sandbox_id}" >> /mnt/archil/hello.txt && cat /mnt/archil/hello.txt', user="root", ) print(result.stdout) finally: # Flush buffered writes and release the disk before the sandbox goes away sandbox.commands.run("archil unmount /mnt/archil", user="root") finally: sandbox.kill() ``` The `archil` client buffers recent writes, so the script unmounts before it kills the sandbox: `archil unmount` returns once every write has reached the disk. Run `npx tsx mount.ts` (or `python mount.py`) twice. Each run starts a new sandbox, and the second one finds the line the first one wrote: ```text theme={null} Hello from ihd5w6hpg6f9zvh9cbf8w Hello from ihd5w6hpg6f9zvh9cbf8w Hello from iq88j4auz0rll06x5jgfh ``` A plain mount is exclusive: one sandbox holds the disk at a time. To mount it in several sandboxes at once, add `--conditional`, so every sandbox can read and write without a checkout. See [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents) for how to hand files from one sandbox to another, and [Consistency](/details/consistency) for when each one sees the others' writes. ## Read it from anywhere Both sandboxes are gone, and the file is still on the disk. Read it from your own machine or app with the SDK: ```typescript read.ts theme={null} import * as archil from "disk"; const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!); console.log(new TextDecoder().decode(await disk.getObject("hello.txt"))); ``` ```python read.py theme={null} import os import archil disk = archil.get_disk(os.environ["ARCHIL_DISK_ID"]) print(disk.get_object("hello.txt").decode()) ``` If a sandbox is killed before it unmounts, see [Troubleshooting mounts on other platforms](/guides/platforms/troubleshooting). ## Clean up Delete the disk with the [`disk` CLI](/reference/disk-cli), and the E2B template: ```bash theme={null} npx disk delete "$ARCHIL_DISK_ID" npx @e2b/cli template delete archil -y ``` ## Next steps * [`archil` CLI](/reference/archil-cli): every mount option, including `--conditional` and `--read-only`. * [Persistent sandboxes](/compute/sandboxes/introduction): Archil's own sandboxes, with the disk mounted for you. # exe.dev Source: https://docs.archil.com/guides/platforms/exe-dev Mount one Archil disk in many exe.dev VMs so they share files with each other and with the rest of your stack. An [exe.dev](https://exe.dev/docs) VM is a persistent Linux machine you create and reach over SSH. Its disk belongs to that one VM. Mount an Archil disk in your VMs and they can all read and write the same files, which your other machines and the Archil SDK can reach too. exe.dev VMs support FUSE and run systemd, so the disk mounts as a regular directory. Mount it with a systemd service, which mounts the disk when the VM boots and unmounts it when the VM shuts down. ## Prerequisites * An exe.dev account with a plan, and your SSH key added to it. `ssh exe.dev whoami` should print your account. * `jq` on your machine. * An Archil disk in the [Archil region](/reference/regions) of your choice. Set your Archil API key and region: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 ``` Create the disk with the [`disk` CLI](/reference/disk-cli). The output includes the disk ID and a one-time [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only: ```bash theme={null} npx disk create exe-data ``` Export both for the steps below: ```bash theme={null} export ARCHIL_DISK_ID=dsk-... export ARCHIL_MOUNT_TOKEN=adt_... ``` ## Mount the disk in a VM Save this unit on your machine as `archil.service`. It mounts the disk at `/mnt/archil`, gives the `exedev` user ownership of it, and unmounts it when the service stops: ```ini archil.service theme={null} [Unit] Description=Mount Archil disk at /mnt/archil Wants=network-online.target After=network-online.target [Service] Type=oneshot RemainAfterExit=yes EnvironmentFile=/etc/archil/disk.env ExecStartPre=mkdir -p /mnt/archil ExecStart=archil mount ${ARCHIL_DISK_ID} /mnt/archil --region ${ARCHIL_REGION} --conditional ExecStartPost=chown exedev:exedev /mnt/archil ExecStop=archil unmount /mnt/archil [Install] WantedBy=multi-user.target ``` Create a VM and install the `archil` client in it. Answer `yes` if SSH asks to trust the new VM's host key: ```bash theme={null} export VM=$(ssh exe.dev new --json | jq -r .ssh_dest) ssh "$VM" 'curl -fsSL https://archil.com/install | sh' ``` Write the disk ID, region, and token to a file only root can read, then install and start the service: ```bash theme={null} printf 'ARCHIL_DISK_ID=%s\nARCHIL_REGION=%s\nARCHIL_MOUNT_TOKEN=%s\n' "$ARCHIL_DISK_ID" "$ARCHIL_REGION" "$ARCHIL_MOUNT_TOKEN" \ | ssh "$VM" 'sudo install -D -m 600 /dev/stdin /etc/archil/disk.env' ssh "$VM" 'sudo install -m 644 /dev/stdin /etc/systemd/system/archil.service && sudo systemctl enable --now archil' < archil.service ``` Write a file to the disk: ```bash theme={null} ssh "$VM" 'echo "Hello from $(hostname)" >> /mnt/archil/visits.log && cat /mnt/archil/visits.log' ``` ```text theme={null} Hello from orange-and-halfopen ``` ## Share the disk between VMs Create a second VM and repeat the same steps: ```bash theme={null} export VM2=$(ssh exe.dev new --json | jq -r .ssh_dest) ssh "$VM2" 'curl -fsSL https://archil.com/install | sh' printf 'ARCHIL_DISK_ID=%s\nARCHIL_REGION=%s\nARCHIL_MOUNT_TOKEN=%s\n' "$ARCHIL_DISK_ID" "$ARCHIL_REGION" "$ARCHIL_MOUNT_TOKEN" \ | ssh "$VM2" 'sudo install -D -m 600 /dev/stdin /etc/archil/disk.env' ssh "$VM2" 'sudo install -m 644 /dev/stdin /etc/systemd/system/archil.service && sudo systemctl enable --now archil' < archil.service ssh "$VM2" 'echo "Hello from $(hostname)" >> /mnt/archil/visits.log && cat /mnt/archil/visits.log' ``` Both VMs have the disk mounted at the same time, and the second one finds the line the first one wrote: ```text theme={null} Hello from orange-and-halfopen Hello from nightstand-delta ``` ## How the disk behaves in a VM **The mount uses conditional mode.** With `--conditional`, every VM can read and write anywhere on the disk at the same time, and none of them holds a lock that another VM has to wait for. See [Sharing disks](/concepts/sharing-disks). **Restarts need no special handling.** When the VM shuts down, systemd stops the service, which unmounts the disk and sends every buffered write to it. When the VM boots, systemd mounts the disk again. **Stop the service to unmount.** `sudo systemctl stop archil` unmounts the disk until you run `sudo systemctl start archil`. To stop mounting it at boot, run `sudo systemctl disable --now archil`. ## Clean up Delete the VMs and the disk once you no longer need them: ```bash theme={null} ssh exe.dev rm "${VM%%.*}" "${VM2%%.*}" npx disk delete "$ARCHIL_DISK_ID" ``` ## Next steps * [TypeScript SDK](/sdks/typescript): read, write, and search the disk without mounting it. * [Sharing disks](/concepts/sharing-disks): how conditional mode and checkouts work. * [`archil` CLI reference](/reference/archil-cli): every `archil mount` option. # AWS Lambda MicroVMs Source: https://docs.archil.com/guides/platforms/lambda-microvms Mount an Archil disk in an AWS Lambda MicroVM so its files outlive the MicroVM. An [AWS Lambda MicroVM](https://docs.aws.amazon.com/lambda/latest/dg/microvms-how-it-works.html) is a Firecracker VM that Lambda builds from your Dockerfile, snapshots, and restores whenever you start one. When a MicroVM terminates, its files go with it. Mount an Archil disk in the MicroVM and anything written there stays on the disk, where the next MicroVM can mount it and your app can read it with the Archil SDK. Lambda MicroVMs support FUSE when the image is built with full OS capabilities, so the disk mounts as a regular directory. Mount it from the MicroVM's `run` lifecycle hook and unmount it from the `terminate` hook. ## Prerequisites * An Archil disk in an AWS region that also runs Lambda MicroVMs: `aws-us-east-1`, `aws-us-west-2`, `aws-eu-west-1`, or `aws-us-east-2` (in [preview](/reference/regions)). * AWS CLI 2.35.10 or later, which adds the `aws lambda-microvms` commands, and `jq`. * An S3 bucket in the same region to hold the image's build artifact. Set your Archil API key and region, and the matching AWS region for the AWS CLI: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 export AWS_REGION=us-west-2 ``` Create the disk with the [`disk` CLI](/reference/disk-cli). The output includes the disk ID and a one-time [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only: ```bash theme={null} npx disk create microvm-data ``` Export both for the steps below: ```bash theme={null} export ARCHIL_DISK_ID=dsk-... export ARCHIL_MOUNT_TOKEN=adt_... ``` ## Build the image The image needs the `archil` client and a process that serves the MicroVM's lifecycle hooks. Lambda MicroVMs run on ARM64, and the installer picks the matching build. ```dockerfile Dockerfile theme={null} FROM public.ecr.aws/lambda/microvms:al2023-minimal RUN dnf install -y python3 && dnf clean all RUN curl -fsSL https://archil.com/install | sh WORKDIR /app COPY server.py . CMD ["python3", "-u", "server.py"] ``` `server.py` serves the hooks on port 9000 and a small app on port 8080. The `run` hook mounts the disk using the disk ID, region, and token passed when the MicroVM starts. The `terminate` hook unmounts it. The app appends a line to a file on the disk for every request, and refuses requests while the disk isn't mounted, so it never writes to the MicroVM's local disk by mistake: ```python server.py theme={null} import json import os import subprocess from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from threading import Thread MOUNTPOINT = "/mnt/archil" HOOK_PREFIX = "/aws/lambda-microvms/runtime/v1/" microvm_id = None def archil(*args, env=None): # Stay inside the 30-second hook timeout set on the image. subprocess.run(["archil", *args], check=True, env=env, timeout=25) class Hooks(BaseHTTPRequestHandler): def do_POST(self): global microvm_id body = self.rfile.read(int(self.headers.get("Content-Length") or 0)) hook = self.path.removeprefix(HOOK_PREFIX) try: if hook == "run": request = json.loads(body) microvm_id = request["microvmId"] config = json.loads(request["runHookPayload"]) os.makedirs(MOUNTPOINT, exist_ok=True) archil("mount", config["disk"], MOUNTPOINT, "--region", config["region"], env={**os.environ, "ARCHIL_MOUNT_TOKEN": config["token"]}) elif hook == "terminate": archil("unmount", MOUNTPOINT) self.send_response(200) except Exception as error: print(f"{hook} hook failed: {error}") self.send_response(500) self.end_headers() class App(BaseHTTPRequestHandler): def do_GET(self): if not os.path.ismount(MOUNTPOINT): self.send_error(503, "Archil disk is not mounted") return with open(f"{MOUNTPOINT}/visits.log", "a+") as log: log.write(f"Hello from {microvm_id}\n") log.flush() os.fsync(log.fileno()) log.seek(0) content = log.read().encode() self.send_response(200) self.send_header("Content-Length", str(len(content))) self.end_headers() self.wfile.write(content) Thread(target=ThreadingHTTPServer(("0.0.0.0", 9000), Hooks).serve_forever, daemon=True).start() ThreadingHTTPServer(("0.0.0.0", 8080), App).serve_forever() ``` Zip both files and upload the zip to your bucket: ```bash theme={null} export BUCKET= export ACCOUNT=$(aws sts get-caller-identity --query Account --output text) zip artifact.zip Dockerfile server.py aws s3 cp artifact.zip "s3://$BUCKET/microvm/artifact.zip" ``` Lambda needs two IAM roles: one to build the image, which reads the artifact from S3, and one for the running MicroVM, which writes your app's output to CloudWatch Logs. Both trust `lambda.amazonaws.com`: ```bash theme={null} TRUST='{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"lambda.amazonaws.com"},"Action":["sts:AssumeRole","sts:TagSession"]}]}' aws iam create-role --role-name ArchilMicrovmBuild --assume-role-policy-document "$TRUST" aws iam put-role-policy --role-name ArchilMicrovmBuild --policy-name read-artifact --policy-document \ "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::$BUCKET/microvm/*\"}]}" aws iam create-role --role-name ArchilMicrovmExec --assume-role-policy-document "$TRUST" aws iam put-role-policy --role-name ArchilMicrovmExec --policy-name write-logs --policy-document \ '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Action":["logs:CreateLogGroup","logs:CreateLogStream","logs:PutLogEvents"],"Resource":"*"}]}' ``` Now create the image. `--additional-os-capabilities ALL` gives the MicroVM the `/dev/fuse` device and the capabilities that `archil mount` needs. The `--hooks` option enables the `run` and `terminate` hooks on port 9000, plus the `ready` hook that tells Lambda when the image is ready to snapshot. Set the hook timeouts explicitly: the defaults are only a few seconds, and Lambda terminates a MicroVM whose `run` hook doesn't finish in time: ```bash theme={null} export IMAGE_ARN=$(aws lambda-microvms create-microvm-image \ --name archil-microvm \ --base-image-arn "arn:aws:lambda:$AWS_REGION:aws:microvm-image:al2023-1" \ --build-role-arn "arn:aws:iam::$ACCOUNT:role/ArchilMicrovmBuild" \ --code-artifact "uri=s3://$BUCKET/microvm/artifact.zip" \ --additional-os-capabilities ALL \ --cpu-configurations architecture=ARM_64 \ --hooks 'port=9000,microvmHooks={run=ENABLED,runTimeoutInSeconds=30,terminate=ENABLED,terminateTimeoutInSeconds=30},microvmImageHooks={ready=ENABLED}' \ --query imageArn --output text) ``` The build takes a few minutes. Wait until the first version reports `SUCCESSFUL` before you start a MicroVM: ```bash theme={null} aws lambda-microvms get-microvm-image-version --image-identifier "$IMAGE_ARN" --image-version 1.0 --query state --output text ``` ## Start a MicroVM with the disk Pass the disk ID, region, and token in `--run-hook-payload`. Lambda delivers the payload to the `run` hook as each MicroVM starts: ```bash theme={null} PAYLOAD=$(jq -nc --arg disk "$ARCHIL_DISK_ID" --arg region "$ARCHIL_REGION" --arg token "$ARCHIL_MOUNT_TOKEN" \ '{disk: $disk, region: $region, token: $token}') MICROVM_ID=$(aws lambda-microvms run-microvm \ --image-identifier "$IMAGE_ARN" \ --execution-role-arn "arn:aws:iam::$ACCOUNT:role/ArchilMicrovmExec" \ --run-hook-payload "$PAYLOAD" \ --query microvmId --output text) ``` The MicroVM stays `PENDING` until its `run` hook returns, so wait for it to reach `RUNNING`. If the mount fails, the hook returns an error and Lambda terminates the MicroVM; `get-microvm` shows why in `stateReason`, and your app's output is in CloudWatch Logs under `/aws/lambda-microvms/archil-microvm`. ```bash theme={null} until [ "$(aws lambda-microvms get-microvm --microvm-identifier "$MICROVM_ID" --query state --output text)" != PENDING ]; do sleep 1 done aws lambda-microvms get-microvm --microvm-identifier "$MICROVM_ID" --query '[state,stateReason]' --output text ``` Request the app on port 8080 with an auth token for the MicroVM: ```bash theme={null} ENDPOINT=$(aws lambda-microvms get-microvm --microvm-identifier "$MICROVM_ID" --query endpoint --output text) AUTH=$(aws lambda-microvms create-microvm-auth-token --microvm-identifier "$MICROVM_ID" \ --expiration-in-minutes 60 --allowed-ports '[{"port":8080}]' --query 'authToken."X-aws-proxy-auth"' --output text) curl -H "X-aws-proxy-auth: $AUTH" "https://$ENDPOINT/" ``` ```text theme={null} Hello from microvm-cefc2a0c-f93b-36ca-ba38-babe9515a690 ``` Terminate the MicroVM. Its `terminate` hook unmounts the disk, which sends every buffered write to the disk and releases it: ```bash theme={null} aws lambda-microvms terminate-microvm --microvm-identifier "$MICROVM_ID" ``` Start a second MicroVM with the same commands and request the app again. It finds the line the first MicroVM wrote: ```text theme={null} Hello from microvm-cefc2a0c-f93b-36ca-ba38-babe9515a690 Hello from microvm-bd32b1eb-bba4-3db4-bae1-277799e7943d ``` If you would rather not pass the token at launch, store it in AWS Secrets Manager, pass the secret's ARN in the payload instead, and read the secret in the `run` hook. Grant the execution role `secretsmanager:GetSecretValue` on that secret. ## How the disk behaves in a MicroVM **Mount in the `run` hook, not while the image builds.** Lambda restores every MicroVM from one snapshot of the image, so a disk mounted before the snapshot would come back in every MicroVM as the same Archil client. Mounting in the `run` hook gives each MicroVM its own. **One MicroVM holds the disk at a time.** A plain mount is exclusive. To mount the same disk in several MicroVMs at once, add `--conditional` to the `archil mount` command. Every MicroVM can then read and write anywhere on the disk; see [Sharing disks](/concepts/sharing-disks). **Suspend and resume need no special handling.** When a MicroVM suspends, the mount is suspended with it. On resume, the client reconnects on its own and finishes any request that was in flight. **Unmount in the `terminate` hook.** Unmounting sends the client's buffered writes to the disk and releases it for the next MicroVM. If a MicroVM ever goes away with the disk still mounted, see [Release a stale lock](/guides/platforms/troubleshooting#release-a-stale-lock). ## Clean up Delete the image, its build artifact, the roles, and the disk once you no longer need them: ```bash theme={null} aws lambda-microvms delete-microvm-image --image-identifier "$IMAGE_ARN" aws s3 rm "s3://$BUCKET/microvm/artifact.zip" aws iam delete-role-policy --role-name ArchilMicrovmBuild --policy-name read-artifact aws iam delete-role --role-name ArchilMicrovmBuild aws iam delete-role-policy --role-name ArchilMicrovmExec --policy-name write-logs aws iam delete-role --role-name ArchilMicrovmExec npx disk delete "$ARCHIL_DISK_ID" ``` ## Next steps * [Python SDK](/sdks/python): read, write, and search the disk without mounting it. * [Sharing disks](/concepts/sharing-disks): mount one disk in many MicroVMs at once. * [`archil` CLI reference](/reference/archil-cli): every `archil mount` option. # Modal Source: https://docs.archil.com/guides/platforms/modal Mount an Archil disk in Modal Sandboxes so their files outlive the container. [Modal](https://modal.com) containers start fast, but their files disappear when they exit. Mount an Archil disk in a Modal Sandbox and anything written there stays on the disk, where the next Sandbox can mount it and your app can read it with the Archil SDK, without starting any compute. `archil mount` uses FUSE, which Modal supports in [VM Sandboxes](https://modal.com/docs/guide/vm-sandboxes) (`runtime="vm"`). Modal Functions and Sandboxes on the default runtime can't mount FUSE file systems. From those, read and write the disk with the [Python SDK](/sdks/python) (`get_object` and `put_object`, with `ARCHIL_API_KEY` and `ARCHIL_REGION` in a Modal Secret), the same way the last step below reads it from your machine. ## Prerequisites Modal VM Sandboxes meet the [requirements for mounting](/mounting/linux#requirements). Functions and Sandboxes on Modal's default gVisor runtime don't. If you restrict the sandbox's outbound traffic, allow TCP 8100 and 32050. Export your Archil API key and region, then install the Archil and Modal SDKs and sign in to Modal: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 pip install archil modal modal setup ``` ## Mount a disk First, create a disk and store its [mount token](/concepts/authorization#token-based-disk-authorization) in a [Modal Secret](https://modal.com/docs/guide/secrets). The token grants access to this disk only. You only do this once: ```python setup.py theme={null} import archil import modal result = archil.create_disk(name="modal-data") modal.Secret.objects.create("archil-mount", {"ARCHIL_MOUNT_TOKEN": result.token}) print(f"export ARCHIL_DISK_ID={result.disk.id}") ``` Run it with `eval "$(python setup.py)"` to export the disk ID. Now start a VM Sandbox from an image with the `archil` client installed. The Sandbox mounts the disk at `/mnt/archil`, writes a file, and unmounts on exit: ```python sandbox.py theme={null} import os import modal app = modal.App("archil-mount") image = modal.Image.debian_slim().apt_install("curl").run_commands("curl -fsSL https://archil.com/install | sh") SCRIPT = """ set -e mkdir -p /mnt/archil archil mount "$ARCHIL_DISK_ID" /mnt/archil --region "$ARCHIL_REGION" > /dev/null trap 'archil unmount /mnt/archil > /dev/null' EXIT echo "Hello from $MODAL_TASK_ID" >> /mnt/archil/hello.txt cat /mnt/archil/hello.txt """ @app.local_entrypoint() def main(): sandbox = modal.Sandbox.create( "bash", "-c", SCRIPT, app=app, image=image, runtime="vm", secrets=[modal.Secret.from_name("archil-mount")], env={"ARCHIL_DISK_ID": os.environ["ARCHIL_DISK_ID"], "ARCHIL_REGION": os.environ["ARCHIL_REGION"]}, timeout=5 * 60, ) sandbox.wait() if sandbox.returncode != 0: raise RuntimeError(sandbox.stderr.read()) print(sandbox.stdout.read()) ``` The `archil` client buffers recent writes, so the `trap` unmounts before the Sandbox exits, even if a command fails: `archil unmount` returns once every write has reached the disk. Run `modal run sandbox.py` twice. Each run starts a new Sandbox, and the second one finds the line the first one wrote: ```text theme={null} Hello from ta-01M55BF71CCMH321JDE0XTP6TV Hello from ta-01M55BF71CCMH321JDE0XTP6TV Hello from ta-01M55BFBF4D404NCTHT4ZCEYHV ``` A plain mount is exclusive: one Sandbox holds the disk at a time. To mount it in several Sandboxes at once, add `--conditional`, so every sandbox can read and write without a checkout. See [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents) for how to hand files from one sandbox to another, and [Consistency](/details/consistency) for when each one sees the others' writes. ## Read it from anywhere Both Sandboxes are gone, and the file is still on the disk. Read it from your own machine or app with the SDK: ```python read.py theme={null} import os import archil disk = archil.get_disk(os.environ["ARCHIL_DISK_ID"]) print(disk.get_object("hello.txt").decode()) ``` If a Sandbox is terminated before it unmounts, see [Troubleshooting mounts on other platforms](/guides/platforms/troubleshooting). ## Clean up Delete the disk and the Modal Secret with `python cleanup.py`. Apps started with `modal run` stop on their own. ```python cleanup.py theme={null} import os import archil import modal archil.get_disk(os.environ["ARCHIL_DISK_ID"]).delete() modal.Secret.objects.delete("archil-mount") ``` ## Next steps * [Python SDK](/sdks/python): read, write, and search the disk without mounting it. * [Persistent sandboxes](/compute/sandboxes/introduction): Archil's own sandboxes, with the disk mounted for you. # Fly.io Sprites Source: https://docs.archil.com/guides/platforms/sprites Mount one Archil disk in many Sprites so they share files with each other and with the rest of your stack. A [Sprite](https://docs.fly.io/sprites) is a persistent Linux VM on Fly.io that pauses when it's idle and wakes on the next request. A Sprite's own disk persists, but it belongs to that one Sprite. Mount an Archil disk in your Sprites and they can all read and write the same files, which your other machines and the Archil SDK can reach too. Sprites support FUSE, so the disk mounts as a regular directory. Run the mount as a Sprite service, which the Sprite's runtime starts again whenever the Sprite boots. ## Prerequisites * The [`sprite` CLI](https://docs.fly.io/sprites/cli/installation), signed in with `sprite login`. * An Archil disk in the [Archil region](/reference/regions) of your choice. Set your Archil API key and region: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 ``` Create the disk with the [`disk` CLI](/reference/disk-cli). The output includes the disk ID and a one-time [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only: ```bash theme={null} npx disk create sprite-data ``` Export both for the steps below: ```bash theme={null} export ARCHIL_DISK_ID=dsk-... export ARCHIL_MOUNT_TOKEN=adt_... ``` ## Mount the disk in a Sprite Create a Sprite and install the `archil` client in it: ```bash theme={null} sprite create archil-one sprite exec -s archil-one -- sh -c 'curl -fsSL https://archil.com/install | sh' ``` Save this script on your machine as `archil-mount.sh`. It mounts the disk at `/mnt/archil`, gives the `sprite` user ownership of it, and unmounts when the service is stopped: ```sh archil-mount.sh theme={null} sudo mkdir -p /mnt/archil sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount "$ARCHIL_DISK_ID" /mnt/archil --region "$ARCHIL_REGION" --conditional || exit 1 sudo chown "$(id -u):$(id -g)" /mnt/archil trap 'mountpoint -q /mnt/archil && sudo archil unmount /mnt/archil; exit 0' TERM INT sleep infinity & wait ``` Upload the script and create a service that runs it, passing the disk ID, region, and token in the service's environment: ```bash theme={null} sprite exec -s archil-one --file archil-mount.sh:/home/sprite/archil-mount.sh -- \ sprite-env services create archil --cmd sh --args /home/sprite/archil-mount.sh \ --env "ARCHIL_DISK_ID=$ARCHIL_DISK_ID,ARCHIL_REGION=$ARCHIL_REGION,ARCHIL_MOUNT_TOKEN=$ARCHIL_MOUNT_TOKEN" ``` The command streams the service's first few seconds of output, ending with `Successfully mounted`. Write a file to the disk: ```bash theme={null} sprite exec -s archil-one -- sh -c 'echo "Hello from $(hostname)" >> /mnt/archil/visits.log && cat /mnt/archil/visits.log' ``` ```text theme={null} Hello from archil-one ``` ## Share the disk between Sprites Create a second Sprite and repeat the same steps: ```bash theme={null} sprite create archil-two sprite exec -s archil-two -- sh -c 'curl -fsSL https://archil.com/install | sh' sprite exec -s archil-two --file archil-mount.sh:/home/sprite/archil-mount.sh -- \ sprite-env services create archil --cmd sh --args /home/sprite/archil-mount.sh \ --env "ARCHIL_DISK_ID=$ARCHIL_DISK_ID,ARCHIL_REGION=$ARCHIL_REGION,ARCHIL_MOUNT_TOKEN=$ARCHIL_MOUNT_TOKEN" sprite exec -s archil-two -- sh -c 'echo "Hello from $(hostname)" >> /mnt/archil/visits.log && cat /mnt/archil/visits.log' ``` Both Sprites have the disk mounted at the same time, and the second one finds the line the first one wrote: ```text theme={null} Hello from archil-one Hello from archil-two ``` ## How the disk behaves in a Sprite **The mount uses conditional mode.** With `--conditional`, every Sprite can read and write anywhere on the disk at the same time, and none of them holds a lock that another Sprite has to wait for. See [Sharing disks](/concepts/sharing-disks). **Pauses need no special handling.** When a Sprite pauses warm, the mount is suspended with it and the client reconnects on wake. When a Sprite boots cold, its runtime starts the `archil` service again, which mounts the disk before you use it. **Stop the service to unmount.** `sprite-env services stop archil` unmounts the disk and keeps it unmounted until you run `sprite-env services start archil`. ## Clean up Destroy the Sprites and delete the disk once you no longer need them: ```bash theme={null} sprite destroy -s archil-one --force sprite destroy -s archil-two --force npx disk delete "$ARCHIL_DISK_ID" ``` ## Next steps * [TypeScript SDK](/sdks/typescript): read, write, and search the disk without mounting it. * [Sharing disks](/concepts/sharing-disks): how conditional mode and checkouts work. * [`archil` CLI reference](/reference/archil-cli): every `archil mount` option. # Troubleshooting mounts on other platforms Source: https://docs.archil.com/guides/platforms/troubleshooting What happens when a sandbox goes away with an Archil disk mounted, how to release a stale lock, and notes for E2B, Daytona, Modal, and Vercel Sandbox. Mounting an Archil disk in an E2B, Daytona, Modal, or Vercel sandbox works much like it does on any Linux server, and needs the same things: `/dev/fuse`, root, permission to mount, and outbound TCP 8100 and 32050. See [Requirements](/mounting/linux#requirements) for the full list and which environments support mounting. The main difference is that the platform can stop the machine while the disk is still mounted, for example when a sandbox reaches its timeout. In [Archil sandboxes](/compute/sandboxes/introduction), Archil mounts disks for you, and a clean stop or pause flushes them. This page covers what happens then, how to recover, and a few platform-specific details. ## A sandbox stopped with the disk mounted The `archil` client buffers recent writes in memory and sends them to the disk in the background. A plain (exclusive) mount also holds a lock on the disk, called a [delegation](/concepts/sharing-disks#understanding-delegations), so that no other client writes to it at the same time. `archil unmount` sends every buffered write and then releases the lock. If the sandbox is killed, stopped, or times out before it unmounts: * **Recent unsynced writes can be lost.** Writes the client hadn't sent yet never reach the disk. Writes that `fsync` returned for are already durable. * **The disk stays locked.** The lock still belongs to the client in the old sandbox, so the next exclusive mount fails with: ```text theme={null} ✗ Unable to mount Archil disk to '/mnt/archil' because another client has an outstanding delegation to the root of the disk, or to any individual file on the disk. ``` To avoid both, unmount in a `finally` block or a shell `trap` before you stop the sandbox (on Daytona, run `archil sync` and `archil checkin` instead, as described below), and give the sandbox a timeout longer than the work it does. ## Release a stale lock Archil doesn't release the lock on its own, and a new sandbox that mounts the disk is a new client, so it can't release the old one's lock either. Once the old client stops responding, Archil marks its delegations as orphaned, usually within a minute. Revoke them with the SDK from your machine: ```typescript release-lock.ts theme={null} import * as archil from "disk"; const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!); for (const delegation of await disk.listDelegations()) { if (delegation.isOrphaned) { await disk.revokeDelegation(delegation); console.log(`Released ${delegation.path}`); } } ``` ```python release_lock.py theme={null} import os import archil disk = archil.get_disk(os.environ["ARCHIL_DISK_ID"]) for delegation in disk.list_delegations(): if delegation.is_orphaned: disk.revoke_delegation(delegation) print(f"Released {delegation.path}") ``` ```text theme={null} Released / ``` The next mount then succeeds. Revoking only orphaned delegations can't interrupt a client that's still connected. `archil mount --force` also takes the disk back, but it revokes every other client's delegations, including those of sandboxes that are still running. See [Forcing ownership](/concepts/sharing-disks#forcing-ownership). ## Platform notes ### E2B * Run `archil mount` and `archil unmount` with `user: "root"`. As the default `user`, the mount fails with `The mount command requires root privileges`. * E2B kills a sandbox when it reaches its timeout, whether or not the disk is unmounted. Set `timeoutMs` (`timeout` in Python) longer than the session. ### Daytona * `archil unmount` fails inside a Daytona sandbox with `Unable to unmount disk: GenericFailure("No such file or directory (os error 2)")`, because the container runtime doesn't let the client detach a FUSE mount (`umount` and `fusermount3 -u` fail too). Run `sudo archil sync && sudo archil checkin ` instead: `sync` sends every buffered write, and `checkin` releases the lock. Then delete the sandbox. * Daytona stops a sandbox after 15 minutes without activity by default. If a session can sit idle longer, raise `autoStopInterval` when you create the sandbox. * Pass the mount token as a regular environment variable. With a Daytona [secret](https://www.daytona.io/docs/en/secrets), the sandbox sees a placeholder instead of the token, and the mount fails with `Authentication failed`. ### Modal * `archil mount` only works in [VM Sandboxes](https://modal.com/docs/guide/vm-sandboxes) (`runtime="vm"`). In Modal Functions and in Sandboxes on the default gVisor runtime, it fails with `Operation not permitted (os error 1)`. From those, read and write the disk with the [Python SDK](/sdks/python) instead. * Modal terminates a Sandbox at its `timeout` without running a shell `trap`. Set `timeout` longer than the work. ### Vercel Sandbox * When a sandbox stops or reaches its `timeout`, the client usually sends its writes and releases the disk on the way out. Stopping it in the middle of heavy writes, such as an `npm install`, can still leave the disk locked. Unmount before `stop()`, and call `sandbox.extendTimeout()` for long jobs. * Create the sandbox in the Vercel region next to the disk's region (`iad1` for `aws-us-east-1`, `pdx1` for `aws-us-west-2`, `dub1` for `aws-eu-west-1`). In a distant region, every read that misses the cache crosses the country. # Vercel Sandbox Source: https://docs.archil.com/guides/platforms/vercel-sandbox Already on Vercel Sandbox? Mount an Archil disk so agent workspaces outlive the sandbox and stay portable. A [Vercel Sandbox](https://vercel.com/docs/vercel-sandbox) is a short-lived microVM: when it stops or its timeout runs out, its files go with it. Mount an Archil disk in the sandbox and anything written there stays on the disk, where the next sandbox can mount it and your app can read it with the Archil SDK, without starting a sandbox. Vercel Sandbox VMs support FUSE, so the disk mounts as a regular directory. > [!NOTE] > > Archil also runs sandboxes itself. [Archil sandboxes](/compute/sandboxes/introduction) are Firecracker microVMs whose root > filesystem is an Archil disk, with an egress firewall and forks that branch the root disk at any size. This guide is > for teams that already run agents on Vercel Sandbox and want their workspace data on storage they can take with them. ## Prerequisites Vercel Sandbox VMs meet the [requirements for mounting](/mounting/linux#requirements): they provide `/dev/fuse`, and `archil` runs as root with `sudo: true`. If you restrict the sandbox's outbound traffic, allow TCP 8100 and 32050. Export your Archil API key and region, install the Archil and Vercel Sandbox SDKs, and link the directory to a Vercel project with the [Vercel CLI](https://vercel.com/docs/cli). On your machine, `@vercel/sandbox` authenticates as the linked project. ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 npm init -y && npm pkg set type=module npm install disk @vercel/sandbox tsx vercel link ``` ## Mount a disk First, create a disk. You only do this once: ```typescript setup.ts theme={null} import * as archil from "disk"; const { disk, token } = await archil.createDisk({ name: "vercel-data" }); console.log(`export ARCHIL_DISK_ID=${disk.id} ARCHIL_MOUNT_TOKEN=${token}`); ``` Run it with `eval "$(npx tsx setup.ts)"` to export the disk ID and the disk's [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to this disk only. In a Vercel Function, store both as [environment variables](https://vercel.com/docs/environment-variables) instead. Now start a sandbox in the Vercel region next to the disk, install the `archil` client, mount the disk at `/mnt/archil`, and write a file. The commands run as root, and the token is passed only to the mount command: ```typescript mount.ts theme={null} import { Sandbox } from "@vercel/sandbox"; const { ARCHIL_DISK_ID, ARCHIL_REGION, ARCHIL_MOUNT_TOKEN } = process.env; const regions: Record = { "aws-us-east-1": "iad1", "aws-us-west-2": "pdx1", "aws-eu-west-1": "dub1" }; const sandbox = await Sandbox.create({ region: regions[ARCHIL_REGION!], persistent: false, timeout: 5 * 60_000 }); async function run(script: string, env: Record = {}) { const result = await sandbox.runCommand({ cmd: "bash", args: ["-c", script], env, sudo: true }); if (result.exitCode !== 0) throw new Error(await result.stderr()); return result.stdout(); } try { await run("apt-get update -qq && curl -fsSL https://archil.com/install | sh > /dev/null"); await run(`mkdir -p /mnt/archil && archil mount ${ARCHIL_DISK_ID} /mnt/archil --region ${ARCHIL_REGION}`, { ARCHIL_MOUNT_TOKEN: ARCHIL_MOUNT_TOKEN!, }); try { console.log(await run(`echo "Hello from ${sandbox.name}" >> /mnt/archil/hello.txt && cat /mnt/archil/hello.txt`)); } finally { // Send any writes not yet synced and release the disk before the sandbox stops await run("archil unmount /mnt/archil"); } } finally { await sandbox.stop(); } ``` A write is durable on Archil once `fsync` returns, stored redundantly across Availability Zones (see [Data durability](/details/architecture#data-durability)). The client buffers writes that haven't been synced and sends them in the background, so the script unmounts before it stops the sandbox: `archil unmount` returns once every write has reached the disk, and releases the disk for the next sandbox. Run `npx tsx mount.ts` twice. Each run starts a new sandbox, and the second one finds the line the first one wrote: ```text theme={null} Hello from sapphire-vocational-porpoise-kBFwUh Hello from sapphire-vocational-porpoise-kBFwUh Hello from coffee-causal-porcupine-4NZYwH ``` A plain mount is exclusive: one sandbox holds the disk at a time. To mount it in several sandboxes at once, add `--conditional`, so every sandbox can read and write without a checkout. See [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents) for how to hand files from one sandbox to another, and [Consistency](/details/consistency) for when each one sees the others' writes. ## Read it from anywhere Both sandboxes are gone, and the file is still on the disk. Read it from your own machine or app with the SDK: ```typescript read.ts theme={null} import * as archil from "disk"; const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!); console.log(new TextDecoder().decode(await disk.getObject("hello.txt"))); ``` If a sandbox stops before it unmounts, see [Troubleshooting mounts on other platforms](/guides/platforms/troubleshooting). ## Clean up Delete the disk with the [`disk` CLI](/reference/disk-cli). The sandboxes already stopped at the end of each run. ```bash theme={null} npx disk delete "$ARCHIL_DISK_ID" ``` ## Next steps * [TypeScript SDK](/sdks/typescript): read, write, and search the disk without mounting it. * [Persistent sandboxes](/compute/sandboxes/introduction): Archil's own sandboxes, with the disk mounted for you. # Evals and RL environments Source: https://docs.archil.com/guides/training/rl-environments Fork a task environment once per rollout, grade each agent run with a hidden grader, and collect the graded trajectories as a JSONL dataset on an Archil disk. Evals and reinforcement learning for agents run the same loop: put an agent in an environment, let it act, and grade the result. To measure a model or train one, you run that loop many times per task and keep every trajectory with its reward. That takes an environment that starts from the same state every time, a grader the agent can't tamper with, and a place to collect the results. This guide builds all three on Archil: * **Environment:** a stopped sandbox with a small task suite installed, kept as a template. Each rollout forks it, so every agent starts on an identical, isolated machine. The fork is the environment's `reset()`. * **Grader:** hidden checks that enter the fork only after the agent finishes, and turn the result into a reward. * **Dataset:** one JSONL record per rollout, appended to an Archil disk, then scored where it lives with a [serverless command](/compute/serverless-execution). [Parallel agents](/guides/agents/parallel-agents) races forks and keeps the best one. Here you keep every rollout, pass or fail: the output is a graded trajectory dataset for pass\@k reports, rejection sampling, or RL fine-tuning. ## Prerequisites Export your Archil API key, a region where sandboxes are available (`aws-us-east-1`, `aws-us-west-2`, or `aws-eu-west-1`), and an [AI Gateway](https://vercel.com/ai-gateway) key for the model: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-east-1 export AI_GATEWAY_API_KEY=... ``` Install the dependencies. The TypeScript agent uses the [AI SDK](https://ai-sdk.dev); the Python agent calls the same model through the gateway's OpenAI-compatible API. ```bash TypeScript theme={null} npm init -y && npm pkg set type=module npm install disk ai zod tsx ``` ```bash Python theme={null} pip install archil openai ``` ## Define the tasks The suite has three small, deterministic tasks: implement an expression evaluator from a spec, write a report from a messy CSV, and fix a buggy parser. This script runs once inside the template and creates a directory per task under `/work`, each with a `TASK.md` for the agent and the starting files. ```bash setup.sh theme={null} set -euo pipefail mkdir -p /work/calc /work/sales-report /work/duration cd /work/calc cat > TASK.md <<'EOF' Implement evaluate(expr) in calc.py. It takes an arithmetic expression with numbers, + - * / // % **, and parentheses, and returns the value Python would compute for it, with Python's precedence and associativity. Don't use eval, exec, compile, or the ast module. EOF printf 'def evaluate(expr):\n raise NotImplementedError\n' > calc.py cd /work/sales-report cat > TASK.md <<'EOF' Write report.py that reads sales.csv and prints one `region,total` line per region, where total is the sum of quantity * unit_price formatted with 2 decimals, sorted by total descending. Region names are case-insensitive; print them in lowercase. Skip rows with no quantity. EOF cat > sales.csv <<'EOF' region,quantity,unit_price West,3,$19.99 east,10,$4.50 West ,1,"$1,200.00" North,,$15.00 East,2,$120.00 north,4,$15.00 EOF cd /work/duration cat > TASK.md <<'EOF' Fix parse_duration in duration.py. It takes strings like "1h30m" and returns seconds. Units are d, h, m, s, in that order, each at most once, and at least one is required. Raise ValueError for anything else, including whitespace. EOF cat > duration.py <<'EOF' import re UNITS = {"d": 86400, "h": 3600, "m": 60, "s": 1} def parse_duration(text): total = 0 for amount, unit in re.findall(r"(\d+)([dhms])", text): total += int(amount) * UNITS[unit] return total EOF ``` The grader stays on your machine, so the agent can't read the checks or edit them to earn a reward it didn't. Like most real evals, it tests cases the spec implies but doesn't list: Python number literals such as `1e3`, and a CSV the agent never saw. It prints how many checks passed for one task. ```python grade.py theme={null} import builtins import importlib import json import subprocess import sys CALC = {"1 + 2 * 3": 7, "(1 + 2) * 3": 9, "2 ** 3 ** 2": 512, "-2 ** 2": -4, "2 ** -1": 0.5, "7 // -2": -4, "-7 % 3": 2, "10 / 4": 2.5, "2 * -3": -6, "--3": 3, ".5 + 1": 1.5, "1e3 / 8": 125.0, "1_000 // 7": 142} DURATIONS = {"1h30m": 5400, "45s": 45, "2d": 172800, "1d2h3m4s": 93784} BAD_DURATIONS = ["", "1x", "30m1h", "1h 30m", "h", "1h1h", "1H", "1.5h", "1h30", "-5m"] HIDDEN_CSV = 'region,quantity,unit_price\nSouth,2,"$1,000.50"\nsouth ,1,$0.99\nEast,,$5.00\nEast,3,$3.33\n' def load(module): try: return importlib.import_module(module) except Exception: return None def ok(check, raises=None): """True if check() returns True, or raises the expected exception.""" try: return check() is True and raises is None except Exception as err: return raises is not None and isinstance(err, raises) def report(csv): with open("sales.csv", "w") as f: f.write(csv) return subprocess.run([sys.executable, "report.py"], capture_output=True, text=True, timeout=30).stdout.split() def forbidden(*args, **kwargs): raise RuntimeError("not allowed") task = sys.argv[1] sys.path.insert(0, ".") if task == "calc": calc = load("calc") builtins.eval = builtins.exec = builtins.compile = forbidden # also blocks ast.parse results = [ok(lambda: calc.evaluate(expr) == value) for expr, value in CALC.items()] elif task == "duration": duration = load("duration") results = [ok(lambda: duration.parse_duration(text) == secs) for text, secs in DURATIONS.items()] results += [ok(lambda: duration.parse_duration(text), raises=ValueError) for text in BAD_DURATIONS] else: original = open("sales.csv").read() results = [ok(lambda: report(original) == ["west,1259.97", "east,285.00", "north,60.00"]), ok(lambda: report(HIDDEN_CSV) == ["south,2001.99", "east,9.99"])] print(json.dumps({"passed": sum(results), "total": len(results)})) ``` ## Build the environment template Create the disk for the trajectories, then a sandbox that runs `setup.sh`. The tasks live on the sandbox's root filesystem, which every fork copies, and the template doesn't mount the disk: the orchestrator writes to it through the API. Once the tasks are in place, the template is stopped, so it doesn't use compute while it waits. ```typescript prepare.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; await archil.createDisk({ name: "rl-trajectories" }); const template = await archil.createSandbox({ name: "rl-env-template", baseImage: "python:3.13", vcpuCount: 2, memSizeMiB: 2048, }); await template.files.uploadFile(await readFile("setup.sh"), "/root/setup.sh"); const setup = await template.exec("bash /root/setup.sh"); if (setup.exitCode !== 0) throw new Error(setup.stderr); await template.stop(); ``` ```python prepare.py theme={null} import archil archil.create_disk(name="rl-trajectories") template = archil.create_sandbox( name="rl-env-template", base_image="python:3.13", vcpu_count=2, mem_size_mib=2048, ) template.files.upload_file("setup.sh", "/root/setup.sh") setup = template.exec("bash /root/setup.sh") if setup.exit_code != 0: raise RuntimeError(setup.stderr) template.stop() ``` ## Run the rollouts The rollout script runs each task twice, at most four rollouts at a time. Each rollout: 1. Forks the template. A fork of a stopped sandbox comes up running, with a copy of its root filesystem, in about a second. 2. Runs an agent with a `bash` tool bound to its fork, capped at 10 steps. 3. Uploads `grade.py`, runs it, and uses the fraction of checks passed as the reward. 4. Appends one JSONL record to `trajectories/.jsonl` on the disk: task, sample, model, reward, steps, token usage, timings, and every message and tool call, with tool output truncated to 2,000 characters. 5. Deletes the fork, in a `finally` block so a failed rollout doesn't leave a sandbox behind. Concurrent appends to an existing file are safe, but two first appends that both create the file conflict, so the script creates the file before the rollouts start. ```typescript rollouts.ts theme={null} import { readFile } from "node:fs/promises"; import { ToolLoopAgent, isStepCount, tool } from "ai"; import * as archil from "disk"; import type { Sandbox } from "disk"; import { z } from "zod"; const TASKS = ["calc", "sales-report", "duration"]; const SAMPLES = 2; // rollouts per task const CONCURRENCY = 4; const MODEL = "anthropic/claude-haiku-5.5"; const template = (await archil.listSandboxes()).find((s) => s.name === "rl-env-template")!; const [disk] = await archil.listDisks({ name: "rl-trajectories" }); const grader = await readFile("grade.py"); const run = `run-${Date.now()}`; const dataset = `trajectories/${run}.jsonl`; await disk.putObject(dataset, ""); function bashTool(env: Sandbox, cwd: string) { return tool({ description: "Run a bash command. Returns the exit code and output.", inputSchema: z.object({ command: z.string() }), execute: async ({ command }) => { const r = await env.exec('bash -c "$COMMAND"', { cwd, env: { COMMAND: command }, timeoutSeconds: 60 }); return `exit ${r.exitCode ?? r.status}\n${r.stdout}${r.stderr}`.slice(-2000); }, }); } async function rollout(task: string, sample: number) { const t0 = Date.now(); const env = await template.fork({ name: `rl-rollout-${task}-${sample}` }); try { const t1 = Date.now(); const cwd = `/work/${task}`; const prompt = `You are in ${cwd}. Read TASK.md and complete the task. Hidden tests will grade your files, ` + "so test your work. Finish with a one-sentence summary."; const agent = new ToolLoopAgent({ model: MODEL, tools: { bash: bashTool(env, cwd) }, stopWhen: isStepCount(10) }); const result = await agent.generate({ prompt }); const t2 = Date.now(); await env.files.uploadFile(grader, "/grader/grade.py"); const { passed, total } = JSON.parse((await env.exec(`python3 /grader/grade.py ${task}`, { cwd })).stdout); const record = { run, task, sample, model: MODEL, reward: passed / total, passed, total, steps: result.steps.length, usage: { input: result.totalUsage.inputTokens, output: result.totalUsage.outputTokens }, fork_ms: t1 - t0, agent_ms: t2 - t1, messages: [{ role: "user", content: prompt }, ...result.responseMessages], }; await disk.appendObject(dataset, `${JSON.stringify(record)}\n`); console.log(`${task} #${sample}: reward ${record.reward.toFixed(2)} (${passed}/${total}) in ${record.steps} steps`); } finally { await env.stop(); await env.delete(); } } const jobs = TASKS.flatMap((task) => Array.from({ length: SAMPLES }, (_, sample) => ({ task, sample }))); const count = jobs.length; const started = Date.now(); await Promise.all( Array.from({ length: CONCURRENCY }, async () => { for (let job = jobs.shift(); job; job = jobs.shift()) { const { task, sample } = job; await rollout(task, sample).catch((err) => console.error(`${task} #${sample} failed: ${err}`)); } }), ); console.log(`${count} rollouts in ${Math.round((Date.now() - started) / 1000)}s -> ${dataset}`); ``` ```python rollouts.py theme={null} import json import os import time from concurrent.futures import ThreadPoolExecutor import archil from openai import OpenAI TASKS = ["calc", "sales-report", "duration"] SAMPLES = 2 # rollouts per task CONCURRENCY = 4 MODEL = "anthropic/claude-haiku-5.5" MAX_STEPS = 10 PROMPT = ( "You are in {cwd}. Read TASK.md and complete the task. Hidden tests will grade your files, " "so test your work. Finish with a one-sentence summary." ) BASH_TOOL = {"type": "function", "function": { "name": "bash", "description": "Run a bash command. Returns the exit code and output.", "parameters": {"type": "object", "properties": {"command": {"type": "string"}}, "required": ["command"]}, }} client = OpenAI(base_url="https://ai-gateway.vercel.sh/v1", api_key=os.environ["AI_GATEWAY_API_KEY"]) template = next(s for s in archil.list_sandboxes() if s.name == "rl-env-template") disk = archil.list_disks(name="rl-trajectories")[0] run = f"run-{int(time.time() * 1000)}" dataset = f"trajectories/{run}.jsonl" disk.put_object(dataset, b"") def bash(env, cwd, command): r = env.exec('bash -c "$COMMAND"', cwd=cwd, env={"COMMAND": command}, timeout_seconds=60) status = r.status if r.exit_code is None else r.exit_code return f"exit {status}\n{r.stdout}{r.stderr}"[-2000:] def run_agent(env, cwd): messages = [{"role": "user", "content": PROMPT.format(cwd=cwd)}] usage = {"input": 0, "output": 0} for step in range(1, MAX_STEPS + 1): response = client.chat.completions.create(model=MODEL, messages=messages, tools=[BASH_TOOL]) usage["input"] += response.usage.prompt_tokens usage["output"] += response.usage.completion_tokens reply = response.choices[0].message messages.append(reply.model_dump(exclude_none=True)) if not reply.tool_calls: break for call in reply.tool_calls: command = json.loads(call.function.arguments)["command"] messages.append({"role": "tool", "tool_call_id": call.id, "content": bash(env, cwd, command)}) return messages, step, usage def rollout(job): task, sample = job t0 = time.time() env = template.fork(name=f"rl-rollout-{task}-{sample}") try: t1 = time.time() cwd = f"/work/{task}" messages, steps, usage = run_agent(env, cwd) t2 = time.time() env.files.upload_file("grade.py", "/grader/grade.py") graded = json.loads(env.exec(f"python3 /grader/grade.py {task}", cwd=cwd).stdout) passed, total = graded["passed"], graded["total"] record = { "run": run, "task": task, "sample": sample, "model": MODEL, "reward": passed / total, "passed": passed, "total": total, "steps": steps, "usage": usage, "fork_ms": round((t1 - t0) * 1000), "agent_ms": round((t2 - t1) * 1000), "messages": messages, } disk.append_object(dataset, (json.dumps(record) + "\n").encode()) print(f"{task} #{sample}: reward {record['reward']:.2f} ({passed}/{total}) in {steps} steps") except Exception as err: print(f"{task} #{sample} failed: {err}") finally: env.stop() env.delete() jobs = [(task, sample) for task in TASKS for sample in range(SAMPLES)] started = time.time() with ThreadPoolExecutor(max_workers=CONCURRENCY) as pool: list(pool.map(rollout, jobs)) print(f"{len(jobs)} rollouts in {round(time.time() - started)}s -> {dataset}") ``` ```bash TypeScript theme={null} npx tsx prepare.ts && npx tsx rollouts.ts ``` ```bash Python theme={null} python prepare.py && python rollouts.py ``` Rollouts finish in any order. Most of the run time is the agents working; each `fork_ms` in the dataset was around a second. One run looked like this: ```text theme={null} sales-report #1: reward 1.00 (2/2) in 4 steps sales-report #0: reward 1.00 (2/2) in 4 steps duration #0: reward 1.00 (14/14) in 4 steps duration #1: reward 1.00 (14/14) in 5 steps calc #0: reward 0.92 (12/13) in 6 steps calc #1: reward 0.92 (12/13) in 7 steps 6 rollouts in 151s -> trajectories/run-1792249424509.jsonl ``` ## Score the run The dataset is plain JSONL on the disk, so you can score it without downloading it. This [jq](https://jqlang.org) program groups the records by task and computes the number solved (every check passed), pass\@1, pass\@2, and the mean reward. pass\@2 uses the unbiased estimator, so it needs at least two samples per task and stays correct when you collect more. ```jq score.jq theme={null} def r: . * 100 | round / 100; group_by(.task) | map({task: .[0].task, n: length, c: (map(select(.reward == 1)) | length), reward: (map(.reward) | add / length)}) | map(. + {pass1: (.c / .n), pass2: (1 - ((.n - .c) * (.n - .c - 1)) / (.n * (.n - 1)))}) | (["task", "rollouts", "solved", "pass@1", "pass@2", "mean_reward"] | @tsv), (.[] | [.task, .n, .c, (.pass1 | r), (.pass2 | r), (.reward | r)] | @tsv), (["overall", (map(.n) | add), (map(.c) | add), (map(.pass1) | add / length | r), (map(.pass2) | add / length | r), (map(.reward) | add / length | r)] | @tsv) ``` The score script uploads the program next to the data and runs it with `disk.exec`, in a short-lived VM with the disk mounted. It scores every run on the disk, so samples from later runs add to the totals. ```typescript score.ts theme={null} import { readFile } from "node:fs/promises"; import * as archil from "disk"; const [disk] = await archil.listDisks({ name: "rl-trajectories" }); await disk.putObject("score.jq", await readFile("score.jq")); const { stdout } = await disk.exec("jq -rsf score.jq trajectories/*.jsonl | column -t"); console.log(stdout); ``` ```python score.py theme={null} import archil disk = archil.list_disks(name="rl-trajectories")[0] with open("score.jq", "rb") as f: disk.put_object("score.jq", f.read()) print(disk.exec("jq -rsf score.jq trajectories/*.jsonl | column -t").stdout) ``` ```text theme={null} task rollouts solved pass@1 pass@2 mean_reward calc 2 0 0 0 0.92 duration 2 2 1 1 1 sales-report 2 2 1 1 1 overall 6 4 0.67 0.67 0.97 ``` Results vary from run to run. In this one, every `duration` and `sales-report` rollout passed all its checks, including the CSV the agent never saw. Both `calc` rollouts passed 12 of 13 checks: they handled precedence and `.5` but rejected `1_000`, a number Python accepts. pass\@1 counts those as failures, while the mean reward shows they were close. That partial credit is what lets RL tell a near miss from a wrong answer, and the trajectories show exactly where each agent went wrong. ## Scale it up * **More samples:** raise `SAMPLES` and `CONCURRENCY`. Each fork is an independent machine, so rollouts don't share state no matter how many run at once. * **More tasks:** add directories to `setup.sh` and checks to `grade.py`. Build a new template, such as `rl-env-template-v2`, instead of changing the old one, so earlier results stay reproducible. * **Use the data:** the records are ordinary JSONL. Filter for `reward == 1` to build a fine-tuning set, compare models by grouping on `.model` as well as `.task`, or read the files from any S3 client through the [S3-compatible API](/protocols/s3-api). ## Clean up Rollout forks delete themselves, so the template and the disk are what's left. The script also deletes forks left behind by an interrupted run; they have to go before the template they came from. ```typescript cleanup.ts theme={null} import * as archil from "disk"; const sandboxes = (await archil.listSandboxes()).filter((s) => s.name.startsWith("rl-rollout-")); sandboxes.push(...(await archil.listSandboxes()).filter((s) => s.name === "rl-env-template")); for (const sandbox of sandboxes) { await sandbox.stop(); await sandbox.delete(); } const [disk] = await archil.listDisks({ name: "rl-trajectories" }); await disk.delete(); ``` ```python cleanup.py theme={null} import archil sandboxes = [s for s in archil.list_sandboxes() if s.name.startswith("rl-rollout-")] sandboxes += [s for s in archil.list_sandboxes() if s.name == "rl-env-template"] for sandbox in sandboxes: sandbox.stop() sandbox.delete() archil.list_disks(name="rl-trajectories")[0].delete() ``` ## Next steps > [card] [Parallel agents](/guides/agents/parallel-agents) > > Race several agents on forks of one sandbox and keep the best result. > [card] [Persistent sandboxes](/compute/sandboxes/introduction) > > Lifecycle, services, network access, and forking in detail. > [card] [Serverless execution](/compute/serverless-execution) > > Run commands directly on a disk without managing a sandbox. > [card] [S3-compatible API](/protocols/s3-api#appending) > > Append to objects and read them from any S3 client. # AI SDK Source: https://docs.archil.com/integrations/ai-sdk Context your agent can understand Agents are extremely capable of performing tasks, but they are limited by the amount of information available in their context. The Archil AI SDK integration gives agents the ability to store and retrieve information from an infinite filesystem, giving them access to the entirety of your data. It exposes an interface they are already extremely familiar with -- the filesystem. It allows your agent to read files, write files, list directories, search with grep, and execute commands in a sandbox via [serverless execution](/compute/serverless-execution). **1. Create an Archil disk** Go to the [Archil console](https://console.archil.com) and create a new disk in an [AWS region with serverless execution](/reference/regions#compute-availability). Also, [create an API key](https://console.archil.com/api-keys) in the same region as your disk. Add the following environment variables to your project: ```sh theme={null} ARCHIL_API_KEY=key-... ARCHIL_REGION= ARCHIL_DISK_ID= AI_GATEWAY_API_KEY= ``` This example uses [Vercel AI Gateway](https://vercel.com/docs/ai-gateway). Create a gateway API key and keep it, along with your Archil API key, out of source control. **2. Install the SDK** ```bash theme={null} npm install ai@7 disk @archildata/ai-sdk ``` **3. Create tools** Create tools for your agent to interact with the disk. ```ts theme={null} import { generateText, isStepCount } from "ai"; import * as archil from "disk"; import { createDiskTools } from "@archildata/ai-sdk"; const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!); const result = await generateText({ model: "anthropic/claude-sonnet-4.6", prompt: "Read the quarterly reports and write a summary of the key findings for each one.", tools: createDiskTools(disk), stopWhen: isStepCount(30), }); console.log(result.text); ``` # eve Source: https://docs.archil.com/integrations/eve Context your agent can understand eve agents are built around the concept of filesystems as a core primitive. Archil now automatically provides the layer connecting them to your data by exposing tools for eve agents to read, write, and execute commands against data, as if it was a local filesystem. **1. Create an Archil disk** Go to the [Archil console](https://console.archil.com) and create a new disk. Also, [create an API key](https://console.archil.com/api-keys) in the same region as your disk. Create a `.env.local` file in the root of your project with: ```sh theme={null} ARCHIL_API_KEY= ARCHIL_REGION= ARCHIL_DISK_ID= ``` **2. Install the SDK** ```bash theme={null} npm install disk @archildata/eve ``` **3. Create tools** Give your agent tools to read/write from an existing Archil disk. Create `agent/tools/disk.ts`: ```ts theme={null} import * as archil from "disk"; import { createDiskTools } from "@archildata/eve"; const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!); export default createDiskTools(disk); ``` Optionally, you can add Archil as the compute and persistent storage provider for eve agent sessions. Create `agent/sandbox/sandbox.ts`: ```ts theme={null} import { defineSandbox } from "eve/sandbox"; import { archilBackend } from "@archildata/eve"; export default defineSandbox({ backend: archilBackend({ disk: process.env.ARCHIL_DISK_ID!, }), }); ``` # just-bash Source: https://docs.archil.com/integrations/just-bash [just-bash](https://justbash.dev) is a terminal emulator from Vercel designed to provide agents with an in-process bash emulator with a limited set of tools. We offer the ability to use Archil disks from `just-bash` for agent environments where you do not have the ability to mount (such as AWS Lambda), and do not want to use a sandbox. For all users of `just-bash`, we recommend moving to [Serverless execution](/compute/serverless-execution), which gives you a full Linux environment to run commands like bash, node, and python on your disk data. Our `just-bash` integration is built by linking our Typescript package against our native Linux client. As a result, it's only available on `x86_64` and `aarch64` Linux machines. ## Usage You use the Archil `just-bash` integration through the exported `ArchilFs` file system. In order to give your agent the ability to use the `archil` CLI to manage delegations, we recommend also adding `archil` as a custom command with `createArchilCommand`. ```typescript theme={null} import { ArchilClient } from '@archildata/client'; import { ArchilFs, createArchilCommand } from "@archildata/just-bash"; import { Bash } from "just-bash"; const client = await ArchilClient.connect({ region: 'aws-us-east-1', diskName: 'myaccount/agent-workspace', }); const fs = await ArchilFs.create(client); const bash = new Bash({ fs, customCommands: [createArchilCommand(client, fs)], }); const result = await bash.exec("ls -la /"); console.log(result.stdout); await bash.exec('echo "hello world" > /greeting.txt'); const cat = await bash.exec("cat /greeting.txt"); console.log(cat.stdout); // "hello world" ``` ## Delegations Writes through `ArchilFs` and the bash executor go through Archil's [delegation system](/concepts/sharing-disks). `writeFile` and `mkdir` handle delegations automatically when possible; for explicit control: ```typescript theme={null} const inodeId = await fs.resolveInodeId("/mydir"); await client.checkout(inodeId); await bash.exec('echo "hello" > /mydir/newfile.txt'); await client.checkin(inodeId); ``` # Subprocessors Source: https://docs.archil.com/legal/subprocessors Third parties Archil engages to process data on your behalf *Last updated: October 17, 2026.* Archil engages the subprocessors listed below to deliver the service. This page is referenced from our [Data Processing Agreement](/details/security#data-protection-gdpr-&-ccpa) and is the authoritative subprocessor list for DPA purposes. ## Customer data subprocessors These subprocessors process disk contents and associated metadata. | Subprocessor | Purpose | Location of processing | Entity jurisdiction | | - | - | - | - | | Amazon Web Services, Inc. | Disk metadata hosting for every disk[1](#note-metadata); active disk data hosting when AWS is the selected cloud | [Region you select](/reference/regions) | United States | | Google LLC | Active disk data hosting when Google Cloud is the selected cloud | [Region you select](/reference/regions) | United States | | Microsoft Corporation | Active disk data hosting for private Azure deployments | Region agreed for the deployment; no public Azure regions | United States | > [!NOTE] > > Each disk's active data is processed by **exactly one** of AWS, Google Cloud, or Microsoft Azure > — determined by the public or preview region you select, or the cloud and region agreed for > a private deployment. Azure has no public regions. The other two do not process that disk's active data. **1.** "Disk metadata" means the disk's name, its list of authorized users, and billing information — and nothing more. Your file data, file paths, and file attributes are always stored exclusively in the cloud and region you select for the disk; they are never sent to AWS unless AWS is the cloud you selected. The metadata hosted in AWS does not include the names or contents of files within a disk. ## Regional contracting entities Where a customer's contracting entity is in the EEA or UK, the applicable subprocessor contracting entity may be a regional affiliate instead of the US entity listed above: * **Amazon Web Services, Inc.** → Amazon Web Services EMEA SARL (Luxembourg) * **Google LLC** → Google Cloud EMEA Limited (Ireland) * **Microsoft Corporation** → Microsoft Ireland Operations Limited ## Transfers and change notifications Applicable cross-border transfer mechanisms (such as the Standard Contractual Clauses) and subprocessor change-notification commitments are set out in the DPA. To be notified of new or replaced subprocessors, [contact us](mailto:security@archil.com). ## Customer-controlled data sources If you configure a [data source](/concepts/data-sources), the data source provider (for example, AWS S3, Google Cloud Storage, or Cloudflare R2) stores the synchronized copy of your data in the bucket you own. You contract directly with that provider — they are not Archil subprocessors. # Mount in a container Source: https://docs.archil.com/mounting/containers Mount Archil into a container when you need data that survives individual container lifetimes, or if you need to share data across multiple containers. Containers, unlike regular Linux servers, require special permissions in order to mount external file systems. A container that runs `archil mount` needs `/dev/fuse`, root, and the `CAP_SYS_ADMIN` capability. See [Requirements](/mounting/linux#requirements) for the full list, including network ports. The best way to mount Archil into a container depends on the platform that you're using and the permissions that you have: * **Local docker containers** support mounting Archil if they are started with `--privileged`, or mount to the host machine and add Archil to your container as a Docker volume. * **Kubernetes pods** support mounting Archil via the [CSI driver](/reference/csi-driver), follow instructions to install and use the CSI driver on K8s deployments * **Cloud container platforms** (such as AWS Lambda, ECS Fargate, or Modal container sandboxes) often do not support mounting Archil because they do not give users sufficient permission to mount external file systems. For these users, we recommend using [Serverless execution](/compute/serverless-execution) to send code that needs the disk to Archil directly. * **Cloud sandboxes** (such as Daytona, E2B, and Modal VM sandboxes) often support mounting Archil directly. See the [Daytona](/guides/platforms/daytona), [E2B](/guides/platforms/e2b), and [Modal](/guides/platforms/modal) guides, or consult your provider's documentation for the best way to mount Archil. The remainder of this article is about how to mount into a local Docker container. For Docker Compose files, mount propagation, sharing a disk across containers and hosts, and clean shutdown, see [Docker and Docker Compose](/mounting/docker-compose). ## Mounting into a privileged container You can easily mount an Archil disk into a privileged container, because it has full access to the host's Linux kernel, and has permissions to mount external file systems. These steps are the same as mounting on a regular Linux machine. **1. Add the client to your Dockerfile** Add the following lines to your Dockerfile to automatically install the Archil CLI to mount Archil disks, and create the folder where you plan to mount Archil. ```bash theme={null} RUN curl -s https://archil.com/install | sh RUN mkdir /mnt/archil ``` **2. Start your privileged container** Start your container with the `--privileged` flag to give your container permission to mount external file systems. ```bash theme={null} docker run --privileged ``` **3. Authorize your server** Each container that connects to the Archil disk needs to be authorized. You can managed which servers are authorized in the Archil console, by navigating to the disk that you want to use, then selecting **Connections** in the left-bar, and clicking **Add User** to authorize a new server. Select **Disk Token**, and save the token somewhere you can retrieve it, you will not be able to see it again. **4. Mount the disk** Using the token that you just saved from the console, you can now mount the disk with `archil mount`. ```bash theme={null} export ARCHIL_MOUNT_TOKEN="" sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount /mnt/archil --region ``` ## Mounting as a Docker volume **1. Mount your disk to the host** Follow the instructions for either [mounting to Linux](/mounting/linux) or [mounting to macOS](/mounting/macos) to mount your Archil disk to the host running docker. For example, for Linux, assuming you have the Archil CLI installed: ```bash theme={null} export ARCHIL_MOUNT_TOKEN="" sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount /mnt/archil --region ``` **2. Run your container** Once your disk is mounted to the host, you can run your container with the `docker run` command, using the `--volume` flag to bind-mount the disk to the container. The `rslave` option lets the container see the disk even if the host mounts or remounts it after the container starts. ```bash theme={null} docker run --volume /mnt/archil:/mnt/archil:rslave ``` ## Comparisons * [Archil vs Amazon S3 Files for agent workspaces](https://archil.com/article/archil-vs-amazon-s3-files-for-agent-workspaces) * [Archil vs Amazon EFS for AI agents](https://archil.com/article/archil-vs-amazon-efs-for-ai-agents) # Docker and Docker Compose Source: https://docs.archil.com/mounting/docker-compose Give Docker containers persistent, shared storage with an Archil disk, using a bind mount from the host or a mount inside the container. A Docker volume lives on one host's disk, so its data goes away with the host and other hosts can't reach it. An Archil disk gives your containers storage that outlives every container and host, and that many containers on many hosts can use at the same time. Archil doesn't ship a Docker volume plugin. Instead, use the `archil` client in one of two ways: * **[Mount on the host and bind into containers](#mount-on-the-host-and-bind-into-containers).** The containers need no extra privileges. Use this when you control the host. * **[Mount inside the container](#mount-inside-the-container).** Each container mounts the disk itself, which needs `/dev/fuse` and the `SYS_ADMIN` capability. Use this when the container should carry its own mount, such as replicas spread across hosts. You need a disk, its region, and a [disk token](/concepts/authorization#token-based-disk-authorization). To create a token, open the disk in the Archil console, select **Connections**, click **Add User**, and select **Disk Token**. ## Mount on the host and bind into containers Install the client on the host and mount the disk, as described in [Mount on Linux](/mounting/linux): ```bash theme={null} curl -s https://archil.com/install | sh sudo mkdir -p /mnt/archil export ARCHIL_MOUNT_TOKEN="" sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount /mnt/archil --region ``` Then bind the mountpoint into your containers with `rslave` propagation: ```yaml compose.yaml theme={null} services: app: image: alpine:3.20 command: ["sleep", "infinity"] volumes: - type: bind source: /mnt/archil target: /data bind: propagation: rslave ``` ```bash theme={null} docker compose up -d docker compose exec app ls /data ``` With `docker run`, add `:rslave` to the volume: ```bash theme={null} docker run --rm -v /mnt/archil:/data:rslave alpine:3.20 ls /data ``` `rslave` makes the container follow the host's mounts. Without it, Docker takes a private copy of the bind mount when the container starts: * A container that starts before the disk is mounted sees an empty directory until you restart it. * A container that starts while the disk is mounted keeps its own copy of the mount, and `archil unmount` on the host waits until that container stops. With `rslave`, the disk appears in running containers as soon as the host mounts it, and a remount on the host shows up without restarting them. While the disk is unmounted, `/data` is the empty directory underneath. Every container on the host shares the host's single mount, so any number of containers on one host can read and write the disk with no extra flags. > [!WARNING] > > Docker's `read_only` (`:ro`) option doesn't apply to a mount that propagates in after the container starts, so a > container can still write to the disk. To give a container read-only access, mount the disk inside it with > `--read-only`. ## Mount inside the container The container needs three things to mount a disk: * The `/dev/fuse` device. * The `SYS_ADMIN` capability, which allows the `mount` call. * On hosts that use AppArmor, such as Ubuntu and Debian, `apparmor:unconfined`. Docker's default AppArmor profile blocks `mount`. This is a smaller grant than `--privileged`. Install the client in your image: ```dockerfile Dockerfile theme={null} FROM debian:bookworm-slim RUN apt-get update \ && apt-get install -y --no-install-recommends curl ca-certificates \ && curl -fsSL https://archil.com/install | sh \ && rm -rf /var/lib/apt/lists/* COPY entrypoint.sh /usr/local/bin/entrypoint.sh ENTRYPOINT ["/usr/local/bin/entrypoint.sh"] CMD ["sleep", "infinity"] ``` The entrypoint reads the token, mounts the disk, and runs your command. When Docker stops the container, it stops your command and unmounts the disk before exiting: ```sh entrypoint.sh theme={null} #!/bin/sh set -eu if [ -z "${ARCHIL_MOUNT_TOKEN:-}" ] && [ -f /run/secrets/archil_mount_token ]; then ARCHIL_MOUNT_TOKEN="$(cat /run/secrets/archil_mount_token)" export ARCHIL_MOUNT_TOKEN fi mkdir -p /mnt/archil archil mount "$ARCHIL_DISK" /mnt/archil --region "$ARCHIL_REGION" ${ARCHIL_MOUNT_FLAGS:-} "$@" & app=$! trap 'kill -TERM "$app" 2>/dev/null' TERM INT status=0 wait "$app" || status=$? if kill -0 "$app" 2>/dev/null; then wait "$app" || status=$? fi archil unmount /mnt/archil exit "$status" ``` Make it executable with `chmod +x entrypoint.sh`. The Compose file grants the device and capability, and passes the token as a Docker secret: ```yaml compose.yaml theme={null} services: app: build: . devices: - /dev/fuse cap_add: - SYS_ADMIN security_opt: - apparmor:unconfined environment: ARCHIL_DISK: ${ARCHIL_DISK} ARCHIL_REGION: ${ARCHIL_REGION} ARCHIL_MOUNT_FLAGS: ${ARCHIL_MOUNT_FLAGS:-} secrets: - archil_mount_token stop_grace_period: 1m secrets: archil_mount_token: file: ./archil_mount_token.txt ``` Keep the token file out of the build context: ```text .dockerignore theme={null} archil_mount_token.txt .env ``` Save your disk token in `archil_mount_token.txt`, then start the service: ```bash theme={null} chmod 600 archil_mount_token.txt ARCHIL_DISK= ARCHIL_REGION= docker compose up -d --build docker compose exec app ls /mnt/archil ``` > [!NOTE] > > You can also put `ARCHIL_DISK` and `ARCHIL_REGION` in a `.env` file next to `compose.yaml`. ## Credentials Keep disk tokens out of your images. Don't put them in a Dockerfile `ENV` or `ARG`, and don't `COPY` them into the image. * **Docker secrets** (recommended). Compose mounts the file at `/run/secrets/archil_mount_token`, and the entrypoint above reads it. The token doesn't appear in `docker inspect`. * **Environment variable.** Pass `ARCHIL_MOUNT_TOKEN` from the shell that runs Compose. This is simpler, but anyone who can run `docker inspect` on the container can read the token. ```yaml compose.yaml theme={null} services: app: environment: ARCHIL_MOUNT_TOKEN: ${ARCHIL_MOUNT_TOKEN} ``` For a host mount, only the host needs the token. The containers never see it. ## Share one disk across containers and hosts On a single host, mount the disk once on the host and bind it into every container that needs it, as shown [above](#mount-on-the-host-and-bind-into-containers). When several containers or hosts each mount the disk, choose a mount mode. By default, `archil mount` takes exclusive ownership of the disk, so only one client can mount it for writing. To let many clients mount it at once, use one of: * `--conditional`: every client can write anywhere on the disk without `archil checkout`. Use this when several replicas write to the same disk. * `--shared`: the disk is read-only to each client until it runs `archil checkout `, which gives that client exclusive write access to the path. The entrypoint above passes `ARCHIL_MOUNT_FLAGS` to `archil mount`. To run two replicas of the service against one disk: ```bash theme={null} ARCHIL_MOUNT_FLAGS=--conditional docker compose up -d --scale app=2 ``` With `--shared`, check out a directory before writing to it, and check it in when you're done: ```bash theme={null} docker compose exec --index 1 app sh -c ' mkdir -p /mnt/archil/worker-1 && archil checkout /mnt/archil/worker-1 && echo hello > /mnt/archil/worker-1/out.txt && archil checkin /mnt/archil/worker-1' ``` Each client caches file and directory information, so a file that one client creates can take up to 30 seconds to appear on another. See [Consistency](/details/consistency) for how to tune this, and [Sharing disks](/concepts/sharing-disks) for how the modes compare and how they handle two clients writing the same file. ## Shut down cleanly Always unmount with `archil unmount`, which flushes data that hasn't reached the disk yet and releases the disk for other clients. * **Mounted inside the container.** `docker stop` and `docker compose down` send `SIGTERM` only to the container's main process, and everything else in the container is killed when it exits. The entrypoint above handles `SIGTERM` by stopping your command and running `archil unmount` first. Set `stop_grace_period` long enough for the unmount to finish. * **Mounted on the host.** Stop the containers first, then unmount: ```bash theme={null} docker compose down sudo archil unmount /mnt/archil ``` A container whose bind mount doesn't use `rslave` keeps the disk mounted, and `archil unmount` waits until it stops. # Mount on Linux Source: https://docs.archil.com/mounting/linux You can use the `archil` CLI to mount an Archil disk locally to an already running Linux server -- such as an Amazon EC2 instance, a microVM sandbox, or a VPS. Archil is compatible out-of-the-box with all programs that run on Linux, such as git, SQLite, or ffmpeg. For more details, see [Application compatibility](/details/support). > [!NOTE] > > This article is for servers which are running their own Linux kernel. If you're trying to mount Archil into a container > such as Docker, Podman, or a container platform (such as Amazon ECS) see [Mount in a container](/mounting/containers). ## How the client mounts The Linux client is a **FUSE** file system. `archil mount` runs in user space, serves the mount through the kernel's FUSE device, and talks to Archil's servers over Archil's own file protocol, encrypted with TLS. It doesn't use NFS or SMB, so no NFS client or kernel module other than FUSE is needed. On macOS, the client mounts through Apple's FSKit instead; see [Mount on macOS](/mounting/macos). ## Requirements Any Linux host can mount an Archil disk with `archil mount` if it meets these requirements. This applies to VMs, containers, and other providers' sandboxes alike. | Requirement | Details | | - | - | | CPU architecture | x86-64 or arm64. | | Linux distribution | The installer supports distributions with `apt` (Debian, Ubuntu), RPM-based distributions, and Alpine (`apk`). It installs the client and the FUSE library it uses (`libfuse2`, `fuse-libs`, or `fuse`). | | FUSE | The kernel's FUSE support, exposed as `/dev/fuse`. If `/dev/fuse` is missing, the installer runs `modprobe fuse`, and warns if the device still isn't there. `archil mount` doesn't check the kernel version. | | Root | `archil mount` must run as root (uid 0), for example with `sudo`. Otherwise it exits with `The mount command requires root privileges`. | | Permission to mount | The process must be allowed to mount a filesystem. In a container, that means access to `/dev/fuse` and the `CAP_SYS_ADMIN` capability, for example with `docker run --device /dev/fuse --cap-add SYS_ADMIN` or `--privileged`. Without it, the mount fails with `the kernel returned EINVAL (invalid argument)`. | | Outbound network | TCP 8100 to the region's mount endpoint, such as `mount.green.us-east-1.aws.prod.archil.com`, and TCP 32050 to Archil's file servers, whose addresses the mount endpoint returns. Both use TLS. To install or upgrade the client, HTTPS to `s3.amazonaws.com`. No inbound ports are needed. | A mounted disk persists only what is written under its mount point. The rest of the host's filesystem, including packages you installed, stays on the host. In [Archil sandboxes](/compute/sandboxes/introduction#root-filesystem-and-mounted-disks), the root filesystem is itself on an Archil disk. ### Where mounting works | Environment | Mounts? | Notes | | - | - | - | | Linux VMs and bare metal | Yes | Run `archil mount` with `sudo`. | | [Archil sandboxes](/compute/sandboxes/introduction#attaching-an-archil-disk) | Yes | Pass `mounts` when you create the sandbox; Archil mounts the disk for you. | | Docker | With `--privileged` | Or mount on the host and bind-mount into the container. See [Mount in a container](/mounting/containers). | | Kubernetes | Yes | Use the [CSI driver](/reference/csi-driver). | | [E2B](/guides/platforms/e2b) | Yes | Run the commands with `user: "root"`. | | [Daytona](/guides/platforms/daytona) | Yes | Run with `sudo`. `archil unmount` doesn't work; run `archil sync` and `archil checkin` instead. | | [Vercel Sandbox](/guides/platforms/vercel-sandbox) | Yes | Run the commands with `sudo: true`. | | [Modal](/guides/platforms/modal) | VM Sandboxes only | Modal Functions and Sandboxes on the default gVisor runtime fail with `Operation not permitted`. | | [Cloud Run](/guides/platforms/cloud-run) | Second-generation execution environment only | Deploy with `--execution-environment gen2`. | | [Cloudflare Containers](/guides/platforms/cloudflare-containers), [Sprites](/guides/platforms/sprites), [exe.dev](/guides/platforms/exe-dev) | Yes | | | [AWS Lambda MicroVMs](/guides/platforms/lambda-microvms) | Yes | Build the image with `--additional-os-capabilities ALL`. | | AWS Lambda functions, Amazon ECS on Fargate | No | These don't let you mount a filesystem. Use [serverless execution](/compute/serverless-execution) or the SDKs instead. | Where you can't mount, you can still use the disk without FUSE or root: run commands next to it with [serverless execution](/compute/serverless-execution), or read and write files with the [TypeScript](/sdks/typescript#reading-and-writing-objects) or [Python](/sdks/python#reading-and-writing-objects) SDK or the [S3 API](/protocols/s3-api). ## Mount a disk **1. Install the client** The `archil` client provides support for mounting Archil disks on Linux. First, install it on your server: ```bash theme={null} curl -s https://archil.com/install | sh ``` **2. Authorize your server** Each server that connects to the Archil disk needs to be authorized. You can managed which servers are authorized in the Archil console, by navigating to the disk that you want to use, then selecting **Connections** in the left-bar, and clicking **Add User** to authorize a new server. Select **Disk Token**, and save the token somewhere you can retrieve it, you will not be able to see it again. > [!NOTE] > > Archil disks also support *IAM authentication* for users mounting from within AWS. For more details on how disks > authenticate users, see [Disk users](/concepts/disk-users). **3. Mount the disk** First, create the directory that you intend to use to mount Archil: ```bash theme={null} sudo mkdir -p /mnt/archil ``` Using the token that you just saved from the console, you can now mount the disk with `archil mount`. ```bash theme={null} export ARCHIL_MOUNT_TOKEN="" sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount /mnt/archil --region ``` You can now use any Linux program against your Archil disk by navigating to `/mnt/archil`, and running commands. ## Upgrading the client The `archil` CLI prints a notice when a newer release is available. To upgrade, run: ```bash theme={null} sudo archil upgrade ``` To install patch releases automatically, see [`archil config`](/reference/archil-cli#config). ## Disk ownership By default, new Archil disks are owned by the root user (Linux `uid:0`). When you mount your disk to a folder such as `/mnt/archil`, that folder will become owned by the root user. When not running as the root user, you will see `EACCES: Permission denied` for file operations. You can change the owner of the root directory with the command: `sudo chown "$(id -u):$(id -g)" /mnt/archil` The ownership change persists on the disk. ## Mounting on multiple hosts By default, your disk is mounted in **exclusive-ownership** mode, and a second server can only mount it with `--read-only`. To write to the disk from several servers, mount it on each one with `--conditional` or `--shared`. See [Sharing Disks](/concepts/sharing-disks) for how the modes compare. When you mount an Archil disk with `--conditional`, every server can write to any part of the disk without calling `archil checkout`. See [Conditional mode](/concepts/sharing-disks#conditional-mode) for how writes from several servers interact. When you mount an Archil disk with `--shared`, each server can only write to paths it holds a delegation on. Creating a file or directory, or opening a file for writing, takes one automatically; for anything else, call `archil checkout ` before writing and `archil checkin ` after. Otherwise, the operation fails with `EROFS: Read-only file system`. Each server caches what it reads, so it can briefly serve an older version of a file another server changed. See [Consistency](/details/consistency#between-archil-clients) for the exact rules. ## Ensuring that data is persisted on shutdown Like all local file systems, Archil automatically stages recently written data in-memory until an explicit `fsync` or a periodic background flush job writes it to durable storage. This means that recently-written data may not be persisted if a machine is unexpectedly powered off. Archil sandboxes automatically manage this persistence on shutdown. On other sandbox providers, make sure that you explicitly call `archil unmount` before powering off the sandbox. See [Crashes and forced ownership](/details/consistency#crashes-and-forced-ownership) for what a client that stops without unmounting loses and leaves behind. We recommend **always** using `archil unmount` instead of the built-in Linux `umount` command, which does not automatically flush data to Archil's servers. ## Mount automatically on startup If your service automatically launches new instances, such as in a load-based auto scaling group, you'll want to ensure that new servers automatically mount Archil at startup. In addition to service startup scripts, you can achieve this by adding Archil to the server's `/etc/fstab` with a line like the following: ```bash theme={null} archil _netdev,region=,defaults 0 0 ``` **Parameters:** * ``: An owner-qualified disk name (`organization/disk-name` or `employee@email.com/disk-name`) or disk ID (`dsk-0123456789abcdef`). * ``: Local directory where the disk should be mounted (e.g., `/mnt/archil`). * `archil`: Filesystem type for Archil disks. * `_netdev`: Tells systemd this is a network filesystem that needs network connectivity before mount. * `region=`: The region your disk is in (e.g., `aws-us-east-1`). Example: ```bash theme={null} # /etc/fstab myorg/mydisk /mnt/archil archil _netdev,region=aws-us-east-1,defaults 0 0 ``` # Mount on macOS Source: https://docs.archil.com/mounting/macos In addition to mounting on Linux servers, Archil also supports mounting locally to a macOS machine for occassionally debugging what is happening on the disk. Mounting Archil on macOS is supported on macOS 26 (Tahoe) and later. When you mount an Archil disk to a laptop or desktop in an office, you're often introducing >100ms of network latency between your computer and the Archil service. As a result, this configuration has reduced performance compared to mounting Archil from an instance in the same cloud region as the Archil file system, and you should not use it for benchmarking how your application runs against the service. **1. Install the macOS client** Mounting Archil on macOS is via a menu bar application. Install it by running the following: ```bash theme={null} curl -s https://archil.com/install | sh ``` **2. Sign in** Click the Archil icon in your menu bar and select **Sign in to Archil**. Enter the email address associated with your Archil account, then enter the verification code we send you. Your session persists across restarts. **3. Pick a disk** Once you're signed in, the menu bar lists every disk you have access to — both your personal disks and any disks shared with you through an [organization](/administration/organizations). Disks are grouped by region. If you don't have a disk yet, create one in the [Archil console](https://console.archil.com) and click **Refresh Disks** in the menu. **4. Mount it** Click any disk and select **Mount**. Archil mounts the disk as a native macOS volume, opens it in Finder, and adds a shortcut to your Finder sidebar. From there, you can browse, edit, and copy files just like any other folder. **5. Unmount when you're done** To unmount cleanly, click the disk in the menu bar and select **Unmount**. Archil waits for any pending writes to flush to your data source before disconnecting. Quitting the app unmounts every active volume automatically. ## Checking out files The macOS client mounts the disk in a shared mode, so it can be simultaneously used by other computers. In this mode, calling `archil checkout` and `archil checkin` are not required, but should be used to improve the performance of metadata-heavy workloads such as `git clone` or `npm install`. For more information on the semantics of `checkin` and `checkout`, see [Shared disks](/concepts/shared-disks). You can "check-out" a file or folder for writing by right-clicking it in Finder and selecting "Check out". An indicator will appear on the file in Finder to show that it's locked exclusively for writing. When you're done working with the folder or file, you can right-click it and select "Check in". ## Mount via command line After installing the application and signing in to your Archil account, you can also mount and unmount disks from the command line. You can mount a disk on macOS by running the following: ```bash theme={null} sudo /usr/local/bin/archil-mount \ 'archil:////?token=&shared=true' \ /Volumes/ ``` On macOS, disks *must* be mounted underneath the `/Volumes` directory. You'll need to specify the `region`, `owner`, `disk-name`, and `disk-token` to mount. To unmount the disk, you can call: ```bash theme={null} sudo umount /Volumes/ ``` > [!WARNING] > > The command-line interface on macOS is **unstable and subject to change** — the helper path, URL format, and flags may change in any release. We're actively working to make this a first-class, supported interface. For anything you depend on, prefer the menu bar, or mount from a [Linux client](/mounting/linux) where the [`archil` CLI](/reference/archil-cli) is fully supported. ## Sharing data with Linux servers Disks mounted on your Mac are the same disks you can mount from Linux servers using the [`archil` CLI](/mounting/linux). Once writes are committed to Archil, other clients can read them from the server. A mounted client may continue to serve an older view until its cache refreshes; see [Consistency](/details/consistency). # S3 API Source: https://docs.archil.com/protocols/s3-api Read and write Archil disks over an S3-compatible HTTP API — list, download, upload, copy, and delete objects with any S3 client, no mount required. Archil exposes every disk as an **S3-compatible bucket**. Point `aws s3`, `boto3`, `s5cmd`, or any other S3 client at the Archil endpoint, present per-disk SigV4 credentials, and read or write the disk's contents over HTTPS without installing the [Archil client](/mounting/linux) or running an [`exec`](/compute/serverless-execution). This works on disks that use Archil-managed storage (the default) and on disks synchronized to your own bucket. This S3 view talks directly to the disk and is **[strongly read-after-write consistent](/details/consistency)** with committed writes. Writes made through a [mount](/mounting/linux) become visible to the S3 API after `fsync` or a background flush commits them to Archil. S3 reads do not wait for synchronization to the backing bucket. Mounted clients may still serve an older view from their own caches until those caches refresh. We also recommend using the S3 API to accelerate access to a disk: requests are served from Archil's high-performance, in-region SSD storage rather than going all the way to object storage, making it lower-latency than a general-purpose S3 bucket. ```bash theme={null} aws s3 ls s3://dsk-abc123/ \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` For workloads that need full POSIX semantics (renames, locks, fast random reads), [mount the disk](/mounting/linux) instead — the S3 API isn't optimised for high-throughput streaming. ## Endpoint The S3 API is served from a per-region endpoint. Use the endpoint that matches the region your disk lives in: | Region | Code | Endpoint | | - | - | - | | AWS US East (N. Virginia) | `aws-us-east-1` | `https://s3.green.us-east-1.aws.prod.archil.com` | | AWS US West (Oregon) | `aws-us-west-2` | `https://s3.green.us-west-2.aws.prod.archil.com` | | AWS EU West (Ireland) | `aws-eu-west-1` | `https://s3.green.eu-west-1.aws.prod.archil.com` | | GCP US Central (Iowa) | `gcp-us-central1` | `https://s3.blue.us-central1.gcp.prod.archil.com` | The endpoint uses **path-style** addressing (`https:////`); virtual-host style is not supported. Configure your client as shown below. The settings are explained in [Client configuration](#client-configuration). ```bash AWS CLI theme={null} export AWS_REQUEST_CHECKSUM_CALCULATION=when_required aws s3 ls s3://dsk-abc123/ \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` ```python boto3 theme={null} import boto3 from botocore.config import Config s3 = boto3.client( "s3", endpoint_url="https://s3.green.us-east-1.aws.prod.archil.com", aws_access_key_id="", aws_secret_access_key="", region_name="us-east-1", config=Config( signature_version="s3v4", s3={"addressing_style": "path"}, request_checksum_calculation="when_required", ), ) ``` ```typescript AWS SDK for JavaScript theme={null} import { S3Client } from "@aws-sdk/client-s3"; const s3 = new S3Client({ endpoint: "https://s3.green.us-east-1.aws.prod.archil.com", region: "us-east-1", forcePathStyle: true, requestChecksumCalculation: "WHEN_REQUIRED", credentials: { accessKeyId: "", secretAccessKey: "", }, }); ``` ```bash s5cmd theme={null} s5cmd \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com \ ls s3://dsk-abc123/ ``` ## Client configuration Most S3 clients work against Archil with two settings: path-style addressing and request checksums set to "when required". Large copies need a third. | Setting | Why | AWS CLI | boto3 | AWS SDK for JavaScript v3 | | - | - | - | - | - | | Path-style addressing | The endpoint serves `https:////` only. | Automatic with `--endpoint-url` | Automatic with `endpoint_url`; set `addressing_style: "path"` to be explicit | `forcePathStyle: true` (the default is virtual-host style) | | Request checksums when required | Archil doesn't accept `aws-chunked` upload bodies yet. | `AWS_REQUEST_CHECKSUM_CALCULATION=when_required`, or `request_checksum_calculation = when_required` in `~/.aws/config` | `Config(request_checksum_calculation="when_required")` | `requestChecksumCalculation: "WHEN_REQUIRED"` | | Multipart threshold for copies | `UploadPartCopy` isn't supported. Only needed for server-side copies larger than 8 MB. | `aws configure set default.s3.multipart_threshold 5GB` | `TransferConfig(multipart_threshold=5 * 1024**3)` passed to `copy()` | Use `CopyObjectCommand` | **Why the checksum setting matters.** Recent AWS SDKs add a CRC checksum to uploads and send the body as `aws-chunked` with the checksum in a trailer, signed as `x-amz-content-sha256: STREAMING-UNSIGNED-PAYLOAD-TRAILER`. boto3 1.36 and later and current AWS CLI v2 releases do this for every `PutObject` and `UploadPart` sent over HTTPS. The AWS SDK for JavaScript v3 does it for stream bodies from version 3.729.0. Archil returns `501 NotImplemented` for any `STREAMING-` payload mode, so these uploads, including every `aws s3 cp` upload, fail unless the setting is `when_required`. With `when_required`, the SDK sends a plain body signed with its SHA-256 or with `UNSIGNED-PAYLOAD`, and Archil accepts it. The AWS CLI, boto3, and the AWS SDK for JavaScript v3 all read the `AWS_REQUEST_CHECKSUM_CALCULATION` environment variable, so one setting covers all three. Clients that always sign the body in chunks (`STREAMING-AWS4-HMAC-SHA256-PAYLOAD`), such as some configurations of the AWS SDK for Java and Hadoop `s3a`, can't upload to Archil yet. **Uploads of any size work.** The AWS CLI and boto3 switch to multipart upload above 8 MB by default and upload 8 MB parts, which fits Archil's 5 MiB minimum part size. No multipart setting is needed for uploads. **Server-side copies above 8 MB need a change.** `aws s3 cp` and `aws s3 mv` between two keys on the disk, and boto3's managed `copy()`, copy objects above the multipart threshold with `UploadPartCopy`, which returns `501 NotImplemented`. Raise the threshold as shown in the table, or call `CopyObject` directly. See [Copying](#copying). ## Authentication Each disk has its own set of S3 credentials — there are no account-wide S3 keys. To get credentials, add an **S3 user** to the disk: 1. Open the disk's **Details** page in the [Archil console](https://console.archil.com). 2. Under **Authorized Users**, click **Add User** and select **S3 credentials**. 3. The console shows the **Access Key ID** and **Secret Access Key** exactly once. Save them somewhere secure — the secret cannot be retrieved again. You can also create S3 credentials with the API by calling [Add Disk User](/api-reference/disk-users/add-user) with `type: "s3"`: ```bash theme={null} curl -X POST \ -H "Authorization: Bearer ${ARCHIL_API_KEY}" \ -H "Content-Type: application/json" \ -d '{"type": "s3", "nickname": "uploads", "ttl": "24h"}' \ "https://control.green.us-east-1.aws.prod.archil.com/api/disks/dsk-abc123/users" ``` The response includes the access key ID and secret, and the secret is shown only once. Add `ttl` to make the keys expire, or `readOnly: true` to limit them to reads. Credentials are scoped to a single disk: presenting them on a request to a different bucket returns `AccessDenied`. To rotate, add a new S3 user and remove the old one. The S3 API uses standard **AWS Signature Version 4** — any S3 SDK signs correctly out of the box. The endpoint also accepts your Archil API key as `Authorization: Bearer ` for disks your account owns. This is how the [TypeScript](/sdks/typescript) and [Python](/sdks/python) SDKs call it, and it lets you use `curl` without SigV4 signing. Standard S3 clients can't send a bearer token, so use S3 credentials with them. > [!NOTE] > > S3 credentials are distinct from [disk tokens](/concepts/authorization#token-based-disk-authorization) (used by the Archil client when mounting) and from [Control Plane API keys](/api-reference/introduction) (account-level, used to manage disks). They cannot be used in place of either one. ## Bucket name The bucket name is the **disk ID** as shown in the console (`dsk-…`). ```bash theme={null} aws s3 ls s3://dsk-abc123/ ``` To address a [branch](/concepts/branches-and-checkpoints) instead of the disk's root, append the branch name to the disk ID with a `.` separator. The same disk credentials authorize every branch. ```bash theme={null} aws s3 ls s3://dsk-abc123.branch-xyz/ ``` ## Supported operations | Operation | Method | Notes | | - | - | - | | `HeadBucket` | `HEAD /` | Verifies the disk exists and the credentials can read it. | | `GetBucketLocation` | `GET /?location` | Returns the disk's region. | | `ListObjectsV2` | `GET /?list-type=2` | Supports `prefix`, `delimiter`, `max-keys`, and `continuation-token`. | | `GetObject` | `GET //` | Streams the object body. Supports HTTP `Range`, `If-Match`, and the `response-*` header overrides. Add `versionId=` to read from a checkpoint; see [Reading from a checkpoint](#reading-from-a-checkpoint). | | `HeadObject` | `HEAD //` | Returns size, content type, and last-modified. Supports `If-Match` and `versionId=`. | | `PutObject` | `PUT //` | Creates or overwrites the file at ``. Send `If-None-Match: *` to create only if the key doesn't exist. See [Writing](#writing). | | `DeleteObject` | `DELETE //` | Removes the file at ``. Idempotent. | | `DeleteObjects` | `POST /?delete` | Bulk-deletes up to 1,000 keys in one request; returns per-key results. | | `AppendObject` | `PUT //?append=true` | Appends to an existing file (or creates it). Up to 1 MiB per call — see [Appending](#appending). | | `CopyObject` | `PUT //` with `x-amz-copy-source` | Copies an object to another key on the same disk, up to 5 GiB. See [Copying](#copying). | | Multipart upload | `CreateMultipartUpload` / `UploadPart` / `CompleteMultipartUpload` / `AbortMultipartUpload` / `ListParts` / `ListMultipartUploads` | Upload large objects in parts — see [Large uploads](#large-uploads). Each part (except the last) must be ≥ 5 MiB. | `ListBuckets` and `UploadPartCopy` return `NotImplemented`. For atomic renames or any other POSIX operation, mount the disk as a file system or use [`disk.exec`](/compute/serverless-execution). `AppendObject` is the only Archil-specific operation on this endpoint. [Serverless execution](/compute/serverless-execution), [parallel search](/compute/search-files), and [share URLs](#share-urls) are part of the control plane API, so a plain S3 client can't call them. Use the [SDKs](/sdks/typescript) or the [API](/api-reference/introduction) for those. ### Conditional requests `GetObject` and `HeadObject` honor `If-Match` and return `412 Precondition Failed` when the ETag doesn't match. `PutObject` and `CompleteMultipartUpload` honor `If-None-Match: *`, which writes only if the key doesn't exist yet. Any other conditional header, including `If-None-Match` with an ETag, `If-Modified-Since`, and `If-Unmodified-Since`, returns `501 NotImplemented`. ### Listing `ListObjectsV2` accepts the standard query parameters: ```bash theme={null} aws s3api list-objects-v2 \ --bucket dsk-abc123 \ --prefix logs/ \ --delimiter / \ --max-keys 100 \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` The only delimiter that's accepted is `/`. Any other delimiter value returns a `400 InvalidArgument`. This matches how Archil clients see the underlying directory tree. `delimiter=/` collapses each directory into a single `CommonPrefixes` entry, exactly as a real S3 bucket would. Pagination uses the standard `NextContinuationToken`. Pass it back as `continuation-token` to fetch the next page. Only `ListObjectsV2` is implemented. A bucket `GET` without `list-type=2` still gets a `ListObjectsV2` response, and the version 1 `marker` parameter and the `start-after` parameter are ignored. Clients that page with `ListObjects` (version 1) therefore see the first page again instead of the next one. `aws s3 ls`, `aws s3api list-objects-v2`, and the boto3 and JavaScript `ListObjectsV2` paginators use version 2 and page correctly. ### Range requests `GetObject` honours the HTTP `Range` header end-to-end and returns `206 Partial Content` with a `Content-Range` header: ```bash theme={null} # First 1 KiB curl -H "Authorization: Bearer ${ARCHIL_API_KEY}" -H "Range: bytes=0-1023" \ https://s3.green.us-east-1.aws.prod.archil.com/dsk-abc123/data.bin # Last 500 bytes curl -H "Authorization: Bearer ${ARCHIL_API_KEY}" -H "Range: bytes=-500" \ https://s3.green.us-east-1.aws.prod.archil.com/dsk-abc123/data.bin # Open-ended curl -H "Authorization: Bearer ${ARCHIL_API_KEY}" -H "Range: bytes=1048576-" \ https://s3.green.us-east-1.aws.prod.archil.com/dsk-abc123/data.bin ``` `Accept-Ranges: bytes` is set on every response so clients can discover range support without an explicit probe. ### Content type `Content-Type` is derived from the object key's extension when one is recognised; otherwise Archil sniffs the first 512 bytes. `HEAD` and `GET` always agree on the same content type for a given key. ### Writing `PutObject` creates or overwrites the file at the given key, creating any parent directories as needed. The write commits to the disk before the response returns, so subsequent reads from the server see it immediately. Mounted clients may need to refresh their caches; see [Consistency](/details/consistency). When the disk is synchronized to a [data source](/concepts/data-sources), the new object is flushed back to the underlying bucket like any other write. `DeleteObject` is idempotent — deleting a key that doesn't exist still returns success — and if you send a `Content-MD5` header on a `PutObject`, Archil validates the body against it and rejects a mismatch. To remove many keys at once, `DeleteObjects` (`POST /?delete`) deletes up to 1,000 per request and reports per-key results; `aws s3 rm --recursive` and `s5cmd rm` use it automatically. ```bash theme={null} # Upload a file aws s3 cp report.json s3://dsk-abc123/reports/2026-01/data.json \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com # Delete it aws s3 rm s3://dsk-abc123/reports/2026-01/data.json \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` ### Large uploads Multipart uploads are supported, so large files need no extra multipart settings beyond the [client configuration](#client-configuration) every upload needs. The AWS CLI and boto3 automatically switch to multipart above their threshold (8 MB by default), uploading parts in parallel and assembling them server-side. `CreateMultipartUpload`, `UploadPart`, `CompleteMultipartUpload`, `AbortMultipartUpload`, `ListParts`, and `ListMultipartUploads` are all available. Each part except the last must be at least **5 MiB**, and an upload may have at most **10,000 parts**. ```bash theme={null} # Large files upload via multipart automatically # (with AWS_REQUEST_CHECKSUM_CALCULATION=when_required set) aws s3 cp big.tar s3://dsk-abc123/backups/big.tar \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` A single `PutObject` body is capped at **5 GiB**; larger objects (up to **5 TiB**) must go through multipart. ### Appending `AppendObject` appends bytes to the end of an existing object, or creates it if the key doesn't exist yet. This is useful for log-style writes that grow a file over time without rewriting it. It's a `PutObject` with an `?append=true` query parameter: ```http theme={null} PUT /dsk-abc123/logs/app.log?append=true ``` Each call may append at most **1 MiB**; to grow a file further, append in successive calls. Like every other write, an append commits before the response returns and is immediately visible to reads that reach the server. Mounted clients may need to refresh their caches. The standard CLI tools don't expose append, so it's easiest to use through the SDKs — `append_object` ([Python](/sdks/python#reading-and-writing-objects)) / `appendObject` ([TypeScript](/sdks/typescript#reading-and-writing-objects)). ### Copying `CopyObject` copies an object to another key on the same disk, creating any parent directories as needed. If the destination exists, it is overwritten. Like every other write, the copy commits before the response returns. The response is a standard `CopyObjectResult` with the new object's `ETag` (the MD5 of its contents) and `LastModified`. The new file keeps the source file's permissions and owner. ```bash theme={null} aws s3api copy-object \ --bucket dsk-abc123 \ --copy-source dsk-abc123/reports/2026-01/data.json \ --key archive/2026-01/data.json \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` A copy is not a rename: the source is left in place. To move an object, copy it and then delete the source, or rename it atomically through a [mount](/mounting/linux). `CopyObject` supports only the following, and returns an error for anything else: * **Same bucket only.** The source must be in the bucket named in the request. A source on another disk, or on another [branch](/concepts/branches-and-checkpoints) of the same disk, returns `400 InvalidRequest`. Copying an object onto itself also returns `400 InvalidRequest`. * **Up to 5 GiB.** A larger source returns `400 EntityTooLarge`. * **Metadata is always copied.** `x-amz-metadata-directive` and `x-amz-tagging-directive` accept only `COPY` (the default). `REPLACE` returns `501 NotImplemented`. * **No conditional copies.** The `x-amz-copy-source-if-match`, `x-amz-copy-source-if-none-match`, `x-amz-copy-source-if-modified-since`, and `x-amz-copy-source-if-unmodified-since` headers, and the destination `If-*` headers, return `501 NotImplemented`. * **No source versions.** A `versionId` on the copy source returns `501 NotImplemented`. * **Files and directory markers don't mix.** A file can be copied to a file key and a directory marker (`key/`) to a directory marker key. Copying between the two returns `400 InvalidRequest`. `UploadPartCopy` is not supported, so a multipart copy fails with `501 NotImplemented`. `aws s3 cp` and `aws s3 mv` between two keys on the disk, and boto3's managed `copy()`, switch to multipart copy above their multipart threshold (8 MB by default). For larger objects, call `CopyObject` directly (`aws s3api copy-object` or boto3's `copy_object`), or raise the client's multipart threshold above the object size: ```bash theme={null} # Copies up to 5 GiB now use a single CopyObject aws configure set default.s3.multipart_threshold 5GB ``` ### Reading from a checkpoint `GetObject` and `HeadObject` accept a `versionId` query parameter that names a [checkpoint](/concepts/branches-and-checkpoints). The response returns the object as it was when that checkpoint was taken, and echoes the checkpoint name in `x-amz-version-id`. ```bash theme={null} aws s3api get-object \ --bucket dsk-abc123 \ --key reports/data.json \ --version-id my-checkpoint \ data.json \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` A checkpoint name that doesn't exist on the disk, or that hasn't finished committing, returns `404 NoSuchVersion`. `versionId=null` reads the live object, as it does on an unversioned S3 bucket. Listings always show the live disk; `ListObjectVersions` isn't supported. To list a checkpoint's contents, [create a branch](/concepts/branches-and-checkpoints) from it and list the branch bucket. ## What gets exposed The S3 API exposes the disk's filesystem as it currently exists. Every file written through any client (FUSE mount, `disk.exec`, the Console file browser, or `PutObject` over this API) appears as an object with the key set to its POSIX path relative to the disk root. Objects written here show up identically on a mount. Newly written files are visible to the next S3 request after the write commits, with the same [read-after-write consistency](/details/consistency) as any other Archil client. Symlinks, FIFOs, sockets, and device nodes appear in listings as zero-byte objects and can be deleted, but `GetObject`, `HeadObject`, and `CopyObject` on them return `501 NotImplemented`; read them through a mount. Directories appear as `CommonPrefixes` when you list with `delimiter=/`. There is no separate set of "S3 objects" — the bucket is just a different protocol view of the same data. ## Presigned URLs The S3 API accepts standard SigV4 presigned URLs for `GetObject` and `PutObject`. Generate them with any S3 SDK, using the disk's S3 credentials and the path-style client shown in [Endpoint](#endpoint). Anyone holding the URL can make that request until it expires, without credentials of their own. To let a browser upload or download a file directly, sign the URL on your server and send the browser only the URL; the credentials themselves reach the whole disk, read-write unless they were created with `readOnly`. Requests from a web page also need your site's origin allowed for CORS. [Contact us](mailto:support@archil.com) to add it to your account's allowlist. ```python boto3 theme={null} download_url = s3.generate_presigned_url( "get_object", Params={"Bucket": "dsk-abc123", "Key": "reports/data.json"}, ExpiresIn=900, ) upload_url = s3.generate_presigned_url( "put_object", Params={"Bucket": "dsk-abc123", "Key": "uploads/photo.jpg"}, ExpiresIn=900, ) ``` ```bash theme={null} curl -o data.json "$DOWNLOAD_URL" curl -X PUT --upload-file photo.jpg "$UPLOAD_URL" ``` ## Share URLs For one-off, ad-hoc downloads, Archil can mint **signed share URLs** that don't require any credentials. Generate one from the console's file browser ("Share" in the row menu), or via the [Control Plane API](/api-reference/disks/create-share-url): ```bash theme={null} curl -X POST \ -H "Authorization: Bearer ${ARCHIL_API_KEY}" \ -H "Content-Type: application/json" \ -d '{"key": "path/to/file.csv", "expiresIn": 3600}' \ "https://control.green.us-east-1.aws.prod.archil.com/api/disks/dsk-abc123/share" ``` `expiresIn` is in seconds and accepts any integer from `1` to `604800` (7 days). The default is `86400` (24 hours). The response contains the share URL: ```json theme={null} { "success": true, "data": { "url": "https://control.green.us-east-1.aws.prod.archil.com/api/shared/", "expiresIn": 3600 } } ``` The returned URL can be opened in any browser — no credentials needed. It streams the file with a sensible `Content-Type` and `Content-Disposition` so files render inline where the browser supports it. HTTP `Range` requests are supported for partial downloads. Share URLs are signed by Archil's internal KMS key, so they survive control-plane restarts and work across replicas. After the link expires, requests return `401 Unauthorized`. Deleting the underlying file causes the link to return `404 Not Found`. ## Limitations * **`PutObject` is capped at 5 GiB per request.** Larger objects (up to 5 TiB) must use [multipart](#large-uploads). * **`CopyObject` is capped at 5 GiB and same-bucket only.** `UploadPartCopy` isn't supported. See [Copying](#copying). * **Reserved paths are rejected.** Keys under `.archil/` are managed by Archil and can't be written or deleted. * **Chunked (`aws-chunked`) upload bodies aren't supported.** Any request whose `x-amz-content-sha256` starts with `STREAMING-`, including the `STREAMING-UNSIGNED-PAYLOAD-TRAILER` that recent AWS SDKs send by default, returns `501 NotImplemented`. Set request checksums to `when_required` as shown in [Client configuration](#client-configuration) so the client sends `UNSIGNED-PAYLOAD` or a full-body SHA-256. * **Path-style addressing only.** Virtual-host style requests (`.`) are rejected. * **`ListObjects` version 1 isn't supported.** Use `ListObjectsV2`; see [Listing](#listing). * **Few conditional headers.** Only `If-Match` on reads and `If-None-Match: *` on writes; see [Conditional requests](#conditional-requests). * **Browser access needs a CORS allowlist.** There is no `PutBucketCors`. [Contact us](mailto:support@archil.com) to allow your site's origin; see [Presigned URLs](#presigned-urls). * **Active mount conflicts return `409`.** If a FUSE mount holds a write [delegation](/concepts/sharing-disks#understanding-delegations) and is actively writing a file, a conflicting `PutObject` or `DeleteObject` returns `409 Conflict` rather than racing the mounted client. * **One bucket per credential.** S3 credentials are disk-scoped; cross-disk access requires a separate set of credentials. * **`/` is the only supported delimiter** for `ListObjectsV2`. * **Keys must fit the data source's key limit.** On a disk synchronized to a data source, a `PutObject` or `CreateMultipartUpload` key longer than the data source allows (1,024 bytes for Amazon S3, including the disk's bucket prefix) returns `400 KeyTooLongError`. * **Non-empty directory markers can't be deleted.** `DeleteObject` on `key/` while that directory still has contents returns `400 InvalidRequest`. * **No `ListBuckets`.** With disk-scoped credentials there is at most one bucket to list, so the operation isn't meaningful. * **No S3 versioning, object lock, ACLs, tags, lifecycle rules, or bucket policies.** A disk is a filesystem exposed over S3, not a full S3 bucket. `PutObject` overwrites the previous contents at that key in place, and prior versions are not retained. These APIs aren't supported: * Object subresources: `GET`, `HEAD`, or `PUT` on an object with `?acl`, `?tagging`, `?retention`, `?legal-hold`, `?attributes`, or `?torrent` returns `501 NotImplemented`. * `ListObjectVersions` returns `501 NotImplemented`. * Bucket writes: `PUT` and `DELETE` on the bucket itself, including `CreateBucket`, `DeleteBucket`, `PutBucketVersioning`, `PutBucketPolicy`, `PutBucketLifecycleConfiguration`, and `PutBucketCors`, return `405 MethodNotAllowed`. * Bucket configuration reads such as `GetBucketVersioning` and `GetBucketPolicy` aren't recognized. The bucket answers them with an object listing instead of an error, so don't rely on their results. To preserve point-in-time state, use [checkpoints and branches](/concepts/branches-and-checkpoints) instead. A checkpoint captures the entire disk at a moment in time and is immutable once created, and you can read a file from it over this API with [`versionId`](#reading-from-a-checkpoint). * **Throughput.** The S3 API is optimised for low latency and convenience. For sustained, high-throughput reads or writes use the [Archil client](/mounting/linux) — it talks the native protocol and benefits from local caching. ## Related * [Branches & Checkpoints](/concepts/branches-and-checkpoints) — fork a disk and read a branch over S3 with `dsk-….` * [Disk Users](/concepts/disk-users) — how authorization works for the disk's other access paths * [Mounting on Linux](/mounting/linux) — the POSIX-protocol path, with full read/write support * [Serverless Execution](/compute/serverless-execution) — run code directly on the disk instead of pulling data out # `archil` CLI Source: https://docs.archil.com/reference/archil-cli Reference for the Linux system CLI that mounts Archil disks as local folders. The `archil` CLI is the Linux system client you install with `curl -s https://archil.com/install | sh` to mount Archil disks on a Linux server. For a step-by-step walkthrough, see [Mount on Linux (FUSE)](/mounting/linux). To verify a versioned Linux artifact before installing it, see [Verify Linux downloads](/reference/verify-linux-downloads). This page is a reference for every command and flag it accepts. To mount a disk on macOS, use the separate [Archil for macOS](/mounting/macos) app. It's an FSKit-based menu bar app, not the `archil` CLI. ## Global options ```bash theme={null} archil [OPTIONS] ``` **Global options:** * `--log-level `: Set log level (trace, debug, info, warn, error) \[default: info] * `--quiet, -q`: Suppress informational output, such as progress spinners * `--verbose, -v`: Show additional detail in command output * `--version`: Print version information * `--help`: Show help message ## Environment variables The following environment variables can be used to configure the Archil CLI: | Variable | Description | | - | - | | `ARCHIL_MOUNT_TOKEN` | The [disk token](/concepts/authorization#token-based-disk-authorization) used to authenticate a mount when not using AWS IAM. Generate one from the disk's **Details** page in the Archil console. Pair with `sudo --preserve-env=ARCHIL_MOUNT_TOKEN` to pass it securely. | **Example:** ```bash theme={null} export ARCHIL_MOUNT_TOKEN="" sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount /mnt/archil --region aws-us-east-1 ``` > [!NOTE] > > Using `--preserve-env=ARCHIL_MOUNT_TOKEN` passes the disk token through sudo's environment rather than the command line, preventing it from being visible in process listings (`ps aux`). > [!NOTE] > > `ARCHIL_MOUNT_TOKEN` is **not** an API key. It grants access to a single disk for mounting; API keys authenticate requests to the [Control Plane API](/api-reference/introduction) and are separate. ## mount Mount a disk to a local directory. ```bash theme={null} archil mount --region [OPTIONS] ``` **Arguments:** * ``: An owner-qualified disk name (`organization/disk-name|employee@email.com/disk-name`) or disk ID (`dsk-0123456789abcdef`). Optionally append `:/path` to mount a [subdirectory](#subdirectory-mounts) as the root (e.g., `myorg/mydisk:/data`). * ``: Local directory to mount the disk **Options:** **Connection:** * `--region `: Region where the disk is located (e.g., `aws-us-east-1`, `gcp-us-central1`) **Cache configuration:** * `--max-cache-mb `: Maximum cache size in MB \[default: 1/4 of available RAM or 2048 MiB, whichever is smaller] * `--target-cache-mb `: Target cache size in MB \[default: 75% of the maximum cache size] * `--pre-fetch `: Comma-separated list of paths to pre-fetch into the cache after the mount completes (e.g., `models,data/2024`). Equivalent to running [`archil prefetch`](#prefetch) once the disk is mounted. * `--read-ahead `: Maximum window the client reads ahead of sequential reads; `0` disables read-ahead \[default: 4194304 (4 MiB)] * `--cache-expiry-secs `: How long the client trusts cached file attributes and directory entries before checking the disk again \[default: 30]. On a `--shared` or `--conditional` mount, lower it to see files that other clients create sooner; `1` lets a client read a file that another client wrote a second earlier. See [When another client sees it](/details/consistency#when-another-client-sees-it). **Behavior options:** * `--shared`: Enable sharing the disk with multiple clients by skipping the checkout of the root directory \[default: false]. Without this flag, the client claims exclusive ownership of the root, and any attempt to mount the same disk from a second location will fail. See [Shared Disks](/concepts/sharing-disks). * `--conditional`: Send writes to the server without requiring a delegation \[default: false]. You can still use `archil checkout` for exclusive write access and higher performance. Cannot be combined with `--shared`. See [Conditional mode](/concepts/sharing-disks#conditional-mode). * `--read-only`: Mount without write access \[default: false]. * `--enable-xattrs`: Enable extended attributes \[default: false] * `--statistics`: Enable statistics collection \[default: false] * `--force, -f`: Force claim ownership of the root directory, immediately revoking the delegations of any other clients \[default: false] * `--branch `: Mount a specific [branch](#branches) of the disk instead of the root > [!WARNING] > > Force claiming ownership of the root directory will immediately revoke the delegations of any other clients. > This can result in data loss for any outstanding writes from other clients to the disk. **Process & logging:** * `--no-fork`: Launch without forking \[default: false] * `--log-dir `: Log to directory instead of journald * `--max-log-size-mb `: Maximum log file size in MB \[default: 100] **Examples:** ```bash theme={null} # Basic mount archil mount /mnt/archil --region aws-us-east-1 # Mount a subdirectory as the root archil mount :/data /mnt/archil --region aws-us-east-1 # Mount with custom cache settings archil mount /mnt/archil --region aws-us-east-1 --max-cache-mb 2048 --target-cache-mb 1024 # Mount in shared mode with statistics archil mount /mnt/archil --region aws-us-east-1 --shared --statistics # Mount and warm the cache with specific paths archil mount /mnt/archil --region aws-us-east-1 --pre-fetch models,data/2024 # Mount with custom logging archil mount /mnt/archil --region aws-us-east-1 --log-dir /var/log/archil --max-log-size-mb 50 ``` > [!CAUTION] > > Always use `archil unmount` to unmount your Archil disks, not the builtin `umount` command. Unlike `umount`, > the `archil unmount` command does not exit until all pending data is flushed to the backing disk. This is particularly > important on system shutdown, in which the Linux kernel will, by default, call `umount` on all devices and shutdown > without waiting for pending Archil writes to be synced. ### Subdirectory mounts You can mount a subdirectory of a disk instead of the entire root by appending `:/path` to the disk name, following the same convention as NFS: ```bash theme={null} # Mount the /data directory as the local root sudo archil mount myorg/mydisk:/data /mnt/archil --region aws-us-east-1 # Mount a nested subdirectory sudo archil mount myorg/mydisk:/projects/alpha /mnt/alpha --region aws-us-east-1 ``` The client resolves the path at mount time by walking from the disk root to the target directory, following symlinks along the way. The resolved directory becomes the root of the FUSE mount: your application sees it as `/` and cannot traverse above it via `..`. The target path must be absolute (starting with `/` after the `:`) and must resolve to a directory that already exists on the disk. **Delegations with subdirectory mounts** When mounting exclusively (without `--shared`), the client checks out ownership of the subdirectory rather than the disk root. This means different clients can mount different subdirectories of the same disk simultaneously without [delegation](/concepts/sharing-disks) conflicts: ```bash theme={null} # Client A mounts /models exclusively sudo archil mount myorg/mydisk:/models /mnt/models --region aws-us-east-1 # Client B mounts /data exclusively — no conflict with Client A sudo archil mount myorg/mydisk:/data /mnt/archil --region aws-us-east-1 ``` > [!NOTE] > > The subdirectory must exist on the disk before mounting. Create it by first mounting the disk root, > running `mkdir`, and then unmounting with `archil unmount`. ## unmount Unmount a disk. ```bash theme={null} archil unmount [OPTIONS] ``` **Arguments:** * ``: Directory where the disk is mounted **Options:** * `--force, -f`: Force unmount operation, skipping fsyncing outstanding writes and releasing any ownership claims \[default: false] > [!WARNING] > > Force unmounting a disk will skip fsyncing outstanding writes. This will result in data loss for any outstanding writes. **Examples:** ```bash theme={null} # Normal unmount archil unmount /mnt/archil # Force unmount archil unmount /mnt/archil --force ``` ## version Display version information. ```bash theme={null} archil version ``` ## upgrade Upgrade the CLI to the latest release. ```bash theme={null} sudo archil upgrade [OPTIONS] ``` **Options:** * `--force, -f`: Reinstall even if the latest release is already installed \[default: false] Once a day, the CLI checks for a newer release and prints a notice when one is available. With [automatic upgrades](#config) enabled, it installs patch releases (such as `v0.9.1` to `v0.9.2`) on its own when run as root. Minor and major releases always require `archil upgrade`. You can also upgrade by rerunning the installer: `curl -s https://archil.com/install | sh`. ## config Read or set a CLI setting. Settings apply to every user on the server. ```bash theme={null} archil config [VALUE] ``` **Arguments:** * ``: Setting to read or set. The only supported key is `upgrade.automatic`, which installs patch releases automatically \[default: false] * `[VALUE]`: New value, `true` or `false`. Setting a value requires root. **Examples:** ```bash theme={null} # Check whether automatic upgrades are enabled archil config upgrade.automatic # Enable automatic upgrades sudo archil config upgrade.automatic true ``` ## checkout Request an ownership claim on a file or directory. ```bash theme={null} archil checkout [OPTIONS] ``` **Arguments:** * ``: Path to the file to check out **Options:** * `--force, -f`: Force checkout, revoking an existing delegation from another client \[default: false] > [!NOTE] > > Force checkout will revoke ownership that other clients have on parents of the affected file or children of the affected file. > For more details, see the [Shared Disks](/concepts/sharing-disks) page. > [!WARNING] > > Force checkout will revoke any existing delegation from another client. > This can result in data loss for any outstanding writes from other clients to the affected file. **Examples:** ```bash theme={null} # Normal checkout archil checkout /mnt/archil/myfile.txt # Force checkout (with confirmation prompt) archil checkout /mnt/archil/myfile.txt --force ``` ## checkin Check in a file, releasing ownership of a file or directory. ```bash theme={null} archil checkin ``` **Arguments:** * ``: Path to the file to check in **Examples:** ```bash theme={null} # Check in a file archil checkin /mnt/archil/myfile.txt ``` ## delegations View the list of ownership claims that this client has on the disk mounted at ``. ```bash theme={null} archil delegations ``` **Arguments:** * ``: Directory where the disk is mounted **Examples:** ```bash theme={null} # View delegations archil delegations /mnt/archil ``` ## status Display status information for a mounted disk, for example, whether or not the disk has failed and is not accepting writes from this client. ```bash theme={null} archil status ``` **Arguments:** * ``: Directory where the disk is mounted **Examples:** ```bash theme={null} # View disk status archil status /mnt/archil ``` ## sync Flush every pending write on a mounted disk to durable storage. Fails if the mount has failed or is read-only, since unacknowledged writes cannot be flushed. Run it before you tell another client that files are ready; see [Sharing a disk between agents](/concepts/sharing-disks#sharing-a-disk-between-agents). ```bash theme={null} archil sync ``` **Arguments:** * ``: Directory where the disk is mounted ## update-token Replace the disk token used by a live, token-authenticated mount. The new token is read from stdin so it never appears in the process list. ```bash theme={null} archil update-token ``` **Arguments:** * ``: Directory where the disk is mounted **Examples:** ```bash theme={null} echo "$NEW_ARCHIL_TOKEN" | archil update-token /mnt/archil ``` ## set-cache-expiry Set how long this client serves a directory listing, and the names it learned from that listing, from its cache instead of listing the directory on the server again. The default is 300 seconds (five minutes). Pass a directory to set it for that directory, or the mount root to set it for every directory on the mount (client v0.8.31 and later). See [Consistency](/details/consistency#when-another-client-sees-it) for how long other cached data can lag. ```bash theme={null} archil set-cache-expiry --readdir-expiry ``` **Arguments:** * ``: Path to the directory where cache expiry should be set, or the mount root for the whole mount **Options:** * `--readdir-expiry `: Cache expiry duration in seconds **Notes:** * This command only applies to directories, not files * Setting expiry to 0 seconds disables caching for the specified directory * Cache settings help balance performance and data freshness * Longer expiry times improve performance but may show stale directory contents * The path must be on an Archil-mounted filesystem **Examples:** ```bash theme={null} # Set directory cache expiry to 5 minutes (300 seconds) archil set-cache-expiry /mnt/archil/my-directory --readdir-expiry 300 # Disable caching for a directory archil set-cache-expiry /mnt/archil/real-time-data --readdir-expiry 0 # Set cache expiry to 1 hour (3600 seconds) archil set-cache-expiry /mnt/archil/static-content --readdir-expiry 3600 ``` ## invalidate-cache *Available in client `v0.8.11` and later.* Drop this client's cached entries (and the kernel's matching attribute, page, and dentry caches) so the next read goes back to the server. Pass the mount root to refresh everything, or a single file path to refresh just that file. Non-root directory paths are rejected. In-flight (dirty) writes are always preserved. ```bash theme={null} archil invalidate-cache ``` **Arguments:** * ``: An Archil mount root (whole-mount form) or a regular file inside an Archil mount (per-file form). **Examples:** ```bash theme={null} # Whole mount archil invalidate-cache /mnt/archil # One file (siblings keep their cache) archil invalidate-cache /mnt/archil/shared-data/report.csv ``` > [!NOTE] > > Stale data on a `--shared`, `--conditional`, or `--read-only` mount typically means another client has updated files on the disk and your client is still serving cached entries. See [Consistency](/details/consistency#when-another-client-sees-it) for how long each kind of cached data can lag. ## set-immutable Mark a disk immutable. Its files can no longer change, so any number of clients can read them at full speed at the same time. Use it for data that's final, such as a dataset release or a dated partition. ```bash theme={null} archil set-immutable archil set-mutable archil list-immutable ``` **Arguments:** * ``: The root of an Archil mount. `set-immutable` and `set-mutable` need a mount with write access. Once a disk is immutable, it rejects writes, and clients mount it with `--read-only` or `--shared`. `archil set-mutable` lets the disk accept writes again; clients that mounted it while it was immutable keep treating it as immutable until they remount. `archil list-immutable` shows whether the disk is immutable. **Example:** ```bash theme={null} # Mark a release immutable once, then mount it read-only everywhere sudo archil mount /mnt/release --region aws-us-west-2 sudo archil set-immutable /mnt/release sudo archil unmount /mnt/release sudo archil mount /mnt/release --region aws-us-west-2 --read-only ``` ## prefetch *Available in client `v0.8.11` and later.* Warm this client's cache by reading paths into it ahead of time, so later reads are served locally instead of fetching from the server. Pass one or more files or directories; directories are fetched recursively. The command queues the work and returns immediately. Fetching continues in the background. ```bash theme={null} archil prefetch ... ``` **Arguments:** * ``: Directory where the disk is mounted * `...`: One or more paths to pre-fetch. Each path may be relative to the mount root (e.g., `data/report.csv`) or a full path under the mountpoint (e.g., `/mnt/archil/data/report.csv`). The mountpoint prefix is stripped automatically, so shell tab-completion works. **Examples:** ```bash theme={null} # Pre-fetch a single file archil prefetch /mnt/archil data/report.csv # Pre-fetch several paths at once (files and/or directories, fetched recursively) archil prefetch /mnt/archil models config.yaml data/2024 # Full paths under the mountpoint also work archil prefetch /mnt/archil /mnt/archil/models ``` > [!NOTE] > > Pre-fetched data is read into the same client cache as regular reads, so it counts against your cache size limits (`--max-cache-mb`). If you pre-fetch more than the cache can hold, earlier entries may be evicted before you read them. To warm the cache automatically as soon as a disk is mounted, use [`mount --pre-fetch`](#mount). ## checkpoints Manage checkpoints: named, point-in-time snapshots of a disk's filesystem state. A [branch](#branches) is always forked from a specific checkpoint. For details, see [Branches & Checkpoints](/concepts/branches-and-checkpoints). ### checkpoints create Create a checkpoint capturing the current filesystem state of a mounted disk. ```bash theme={null} archil checkpoints create ``` **Arguments:** * ``: Local directory where the disk is mounted * ``: Name for the new checkpoint (must be unique within the current branch) **Examples:** ```bash theme={null} # Checkpoint the current state of the disk mounted at /mnt/archil archil checkpoints create /mnt/archil pre-migration ``` ### checkpoints list List checkpoints on a branch (or on the root if `--branch` is omitted). ```bash theme={null} archil checkpoints list --region [OPTIONS] ``` **Arguments:** * ``: Disk ID to list checkpoints for (e.g., `dsk-0123456789abcdef`) **Options:** * `--region `: Region where the disk is located (e.g., `aws-us-east-1`) * `--branch `: Branch name to list checkpoints from. If omitted, lists checkpoints on the root. * `--json`: Output as JSON \[default: false] **Examples:** ```bash theme={null} # List checkpoints on the root archil checkpoints list dsk-0123456789abcdef --region aws-us-east-1 # List checkpoints on a specific branch archil checkpoints list dsk-0123456789abcdef --region aws-us-east-1 --branch experiment-1 # Machine-readable output archil checkpoints list dsk-0123456789abcdef --region aws-us-east-1 --json ``` ## branches Manage branches: independent, writable forks of a disk created from a [checkpoint](#checkpoints). Branches can be mounted with [`archil mount --branch`](#mount). For details, see [Branches & Checkpoints](/concepts/branches-and-checkpoints). `branches` subcommands require a `--region` flag: ```bash theme={null} archil branches --region ``` **Options (apply to all `branches` subcommands):** * `--region `: Region where the disk is located (e.g., `aws-us-east-1`) ### branches create Create a new branch forking a specific checkpoint. ```bash theme={null} archil branches --region create --from-checkpoint [--from-branch ] ``` **Arguments:** * ``: Disk ID to branch from (e.g., `dsk-0123456789abcdef`) * ``: Name for the new branch **Options:** * `--from-checkpoint `: Name of the checkpoint to branch from (required) * `--from-branch `: Name of the branch the checkpoint is on. Defaults to the root. **Examples:** ```bash theme={null} # Create a branch from a checkpoint on the root archil branches --region aws-us-east-1 create dsk-0123456789abcdef experiment-1 \ --from-checkpoint pre-migration # Create a branch from a checkpoint that lives on another branch archil branches --region aws-us-east-1 create dsk-0123456789abcdef experiment-2 \ --from-checkpoint v1 --from-branch experiment-1 ``` ### branches list List branches on a disk as a tree. Each node shows the branch name and the checkpoint it was forked from. ```bash theme={null} archil branches --region list [OPTIONS] ``` **Arguments:** * ``: Disk ID to list branches for (e.g., `dsk-0123456789abcdef`) **Options:** * `--limit `: Cap the number of branches shown in the tree. Excess branches are dropped newest-leaf-first, and a `... N more` marker is rendered beneath the closest still-visible parent to indicate how many direct children were truncated. Ignored when `--json` is set. * `--json`: Output the full list as JSON, with no truncation \[default: false] **Examples:** ```bash theme={null} # Render the full branch tree archil branches --region aws-us-east-1 list dsk-0123456789abcdef # Cap the tree to the 10 oldest branches archil branches --region aws-us-east-1 list dsk-0123456789abcdef --limit 10 # Machine-readable output (full list, never truncated) archil branches --region aws-us-east-1 list dsk-0123456789abcdef --json ``` # File systems on one host Source: https://docs.archil.com/reference/benchmarks/file-systems How we compared Archil with Amazon EFS, JuiceFS, Amazon S3 Files, and Mountpoint for Amazon S3 from one EC2 host, and the commands to rerun each test. Each benchmark on this page mounts every file system on the same EC2 host, so the compute and the network are the same and only the file system differs. Run the same commands from your own host to check our numbers. ## Archil vs Amazon EFS The results are in [Benchmark against Amazon EFS](/details/comparison#benchmark-against-amazon-efs) and on [archil.com/compare/amazon-efs](https://archil.com/compare/amazon-efs). ### Method * **Host:** AWS c7gn.2xlarge (Graviton3, 8 vCPU, 15 GiB), us-east-1, Amazon Linux 2023. Both file systems were in the same region and tested from the same host on 2026-10-15. * **Archil:** a new Archil disk with no data source, mounted with Archil client v0.9.2 and default mount options. * **Amazon EFS:** a new file system with General Purpose performance mode, Elastic throughput and Standard storage, mounted with amazon-efs-utils 3.0.1 (TLS, NFSv4.1, rsize/wsize 1 MiB). Amazon S3 Files is built on the same service. * **No network fetches:** git clones came from local bare mirrors, and npm used `npm ci --prefer-offline` with a prefilled cache. Only the file system under test was measured. * **Timing:** times are wall-clock time until the command returns. * **fio:** fio 3.32, buffered I/O, 4 GiB file. 4 KiB random I/O used libaio, iodepth 16, 4 jobs. Cold reads were taken after unmount, `drop_caches` and remount. * **Tool versions:** git 2.50.1, Node 24.18.0 / npm 11.16.0, SQLite 3.40.0. ### Run it Mount both file systems, then run every test below 3 times on each mount and take the median. Set `MNT` to the mount under test. Inputs and caches stay on local disk. ```bash theme={null} archil mount dsk-... /mnt/archil --region aws-us-east-1 mount -t efs -o tls :/ /mnt/efs ``` Between steps, except between a clone and its `git status`, we flushed the mount (`archil sync /mnt/archil` on Archil, `sync -f` on EFS). The flush isn't part of the times in the table. **git.** Mirror both repositories once, pinned to the commits we used: ```bash theme={null} git clone --bare https://github.com/expressjs/express /data/mirrors/express.git git -C /data/mirrors/express.git update-ref refs/heads/master a419aa7350389a275ab45fde2e849ed1f02449ef git clone --bare https://github.com/facebook/react /data/mirrors/react.git git -C /data/mirrors/react.git update-ref refs/heads/main 99c106c43439ad7af05d42f66582d8d05190e9ff ``` Then, for each repository (old tag `3.21.2` for express, `v16.0.0` for react): ```bash theme={null} mkdir -p "$MNT/git" && cd "$MNT/git" time git clone -q --no-hardlinks /data/mirrors/react.git react time git -C react status time git -C react checkout -q v16.0.0 time rm -rf react ``` **npm.** Create the project and fill the cache once: ```bash theme={null} export npm_config_cache=/data/npm-cache cd /data && npx -y create-next-app@15.5.4 app --ts --eslint --tailwind --app --src-dir --no-turbopack \ --import-alias '@/*' --use-npm --skip-install --disable-git --yes (cd /data/app && npm install --package-lock-only --no-audit --no-fund) mkdir /data/prime && cp /data/app/package*.json /data/prime && (cd /data/prime && npm ci --no-audit --no-fund) && rm -rf /data/prime ``` Our lockfile resolved 598 packages. Yours resolves the newest matching versions on the day you create it, so expect the count to differ a little. Then: ```bash theme={null} rm -rf "$MNT/npm" && mkdir "$MNT/npm" && cp /data/app/package*.json "$MNT/npm" && cd "$MNT/npm" time npm ci --prefer-offline --no-audit --no-fund time rm -rf node_modules ``` **SQLite.** `python3 sqlite_bench.py "$MNT/sqlite"` prints the time in seconds for each of the four tests. The script has three parts, `full`, `normal`, and `bulk`; `bulk` runs the 1M-row insert and then the point reads against that table in the same process, with no flush between them. On EFS we ran each part on its own with a 10-minute limit, for example `timeout 600 python3 sqlite_bench.py "$MNT/sqlite" bulk`, and the point reads hit that limit. > [accordion] sqlite_bench.py > > ```python sqlite_bench.py theme={null} > import os, random, sqlite3, sys, time > > d = sys.argv[1] > os.makedirs(d, exist_ok=True) > PAYLOAD = "x" * 100 > rng = random.Random(42) > > > def fresh(name, sync): > p = os.path.join(d, name) > for suf in ("", "-wal", "-shm"): > try: > os.remove(p + suf) > except FileNotFoundError: > pass > c = sqlite3.connect(p, isolation_level=None) > c.execute("PRAGMA journal_mode=WAL") > c.execute(f"PRAGMA synchronous={sync}") > c.execute("CREATE TABLE t(id INTEGER PRIMARY KEY, a INTEGER, b TEXT)") > return c, p > > > def single_txn_inserts(sync, n=10_000): > c, _ = fresh(f"single_{sync}.db", sync) > s = time.perf_counter() > for i in range(n): > c.execute("INSERT INTO t(a, b) VALUES (?, ?)", (i, PAYLOAD)) > e = time.perf_counter() - s > c.close() > return e > > > def bulk(n=1_000_000): > c, p = fresh("bulk.db", "FULL") > s = time.perf_counter() > c.execute("BEGIN") > c.executemany("INSERT INTO t(id, a, b) VALUES (?, ?, ?)", > ((i, i, PAYLOAD) for i in range(1, n + 1))) > c.execute("COMMIT") > e = time.perf_counter() - s > c.execute("PRAGMA wal_checkpoint(TRUNCATE)") > c.close() > return e, p, n > > > def point_reads(p, n_rows, n=100_000): > keys = [rng.randint(1, n_rows) for _ in range(n)] > s = time.perf_counter() > c = sqlite3.connect(p, isolation_level=None) > cur = c.cursor() > for k in keys: > cur.execute("SELECT a, b FROM t WHERE id = ?", (k,)) > if cur.fetchone() is None: > raise SystemExit(f"missing row {k}") > e = time.perf_counter() - s > c.close() > return e > > > which = sys.argv[2:] or ["full", "normal", "bulk"] > if "full" in which: > print(f"sqlite_10k_txn_full {single_txn_inserts('FULL'):.3f}", flush=True) > if "normal" in which: > print(f"sqlite_10k_txn_normal {single_txn_inserts('NORMAL'):.3f}", flush=True) > if "bulk" in which: > e, p, n = bulk() > print(f"sqlite_bulk_1m {e:.3f}", flush=True) > print(f"sqlite_100k_point_reads {point_reads(p, n):.3f}", flush=True) > ``` **fio.** The four jobs in the table, in the order they ran, all on the same 4 GiB file. The MB/s and IOPS are fio's `bw_bytes` and `iops`. Warm reads also ran after the sequential write; they aren't in the table. ```bash theme={null} F="--filename=$MNT/fio/file --size=4g --direct=0 --group_reporting --output-format=json" mkdir -p "$MNT/fio" fio --name=seqwrite_1m $F --rw=write --bs=1m --ioengine=psync --end_fsync=1 fio --name=randwrite_4k $F --rw=randwrite --bs=4k --ioengine=libaio --iodepth=16 --numjobs=4 --time_based --runtime=30 --end_fsync=1 # Unmount, run `sync; echo 3 > /proc/sys/vm/drop_caches`, and mount again before each cold read fio --name=seqread_1m_cold $F --rw=read --bs=1m --ioengine=psync fio --name=randread_4k_cold $F --rw=randread --bs=4k --ioengine=libaio --iodepth=16 --numjobs=4 --time_based --runtime=30 ``` ## Archil vs JuiceFS vs Amazon S3 Files The developer-workload numbers on [archil.com/compare/juicefs](https://archil.com/compare/juicefs) and [archil.com/compare/amazon-s3-files](https://archil.com/compare/amazon-s3-files) come from one AWS c7gn.2xlarge (arm64) in us-east-1 with every file system mounted on the same host: ```bash theme={null} archil mount dsk-... /mnt/archil --region aws-us-east-1 juicefs mount -d redis://:6379/1 /mnt/jfs # JuiceFS defaults juicefs mount -d --writeback redis://:6379/1 /mnt/jfs # JuiceFS writeback mount -t s3files :/ /mnt/s3files # amazon-efs-utils ``` JuiceFS Community Edition kept data in Amazon S3 and metadata in ElastiCache for Redis in the same Availability Zone. Inputs sit on local disk, so GitHub and the npm registry aren't timed, and each run is timed until its writes are durable. Run each workload three times per file system, delete its output between runs, and take the median: ```bash theme={null} # Untimed, once: a local Django mirror, a Next.js 15 app, and a filled npm cache git clone --mirror https://github.com/django/django.git /data/django.git npx create-next-app@15 /data/next --yes --skip-install (cd /data/next && npm install --cache /data/npm-cache --ignore-scripts && rm -rf node_modules) # Timed, with MNT set to the mount under test time (git clone -q file:///data/django.git "$MNT/django" && sync -f "$MNT/django") mkdir "$MNT/app" && cp /data/next/package.json /data/next/package-lock.json "$MNT/app" time (cd "$MNT/app" && npm ci --cache /data/npm-cache --prefer-offline --ignore-scripts --no-audit --no-fund && sync -f .) ``` On Archil, replace `sync -f ` with `archil sync /mnt/archil`. With `--writeback`, JuiceFS acknowledges writes once they are staged on local disk, so also wait until every staged block has uploaded: ```bash theme={null} until [ "$(awk '/^juicefs_staging_(writing_)?blocks /{s+=$2} END{print s+0}' /mnt/jfs/.stats)" = 0 ]; do sleep 0.05; done ``` Our app installed 18,602 files. The count depends on the Next.js release `create-next-app@15` resolves, so expect yours to differ a little. ## Agent workspace: Archil vs Amazon S3 Files vs JuiceFS The results are in the [agent workspace benchmark](/details/comparison#agent-workspace-benchmark). ### Method * **Host:** AWS c7gn.2xlarge (Graviton3, 8 vCPU, 15 GiB), us-east-1, Amazon Linux 2023, kernel 6.1. Every file system was tested from this host on 2026-11-03, except JuiceFS with SQLite on 2026-11-04. * **Archil:** Archil client v0.10.1, a default (exclusive) mount with directory and attribute caches kept for an hour. * **Amazon S3 Files:** amazon-efs-utils 3.0.1, NFS 4.2, with the mount target in the host's Availability Zone. * **JuiceFS:** Community Edition 1.4.2 with default mount options (1 s attribute and entry cache, 100 GiB local cache), data in Amazon S3, and metadata in one of: * Amazon ElastiCache for Redis 7.1 in the host's Availability Zone, mounted with and without `--writeback`. * Amazon RDS for PostgreSQL 17 in the host's Availability Zone. * A SQLite file on the host's EBS volume. * **Concurrency:** one client per file system. 16 threads stand in for concurrent agents. * **Cold and warm:** before each cold test we flushed every write, unmounted, deleted the JuiceFS cache directory, ran `sync; echo 3 > /proc/sys/vm/drop_caches`, and mounted again. The warm test repeats the same operations right after. Only the client is cold: Archil and Amazon S3 Files keep recently written data in a server-side tier, which was warm from the initial copy, as it would be for a workspace in active use. JuiceFS cold reads go to S3. * **Latency:** p50 and p99 over every call, timed from the start of the call until it returns, with 200 to 4,800 calls per cell. Scans are single timed runs. ```bash theme={null} archil mount dsk-... /mnt/archil --region aws-us-east-1 --cache-expiry-secs 3600 mount -t s3files :/ /mnt/s3files juicefs mount -d redis://:6379/1 /mnt/jfs juicefs mount -d --writeback redis://:6379/1 /mnt/jfs juicefs mount -d postgres://@:5432/ /mnt/jfs juicefs mount -d sqlite3:///data/jfs/meta.db /mnt/jfs ``` ### Workspace `python3 gen_workspace.py /data/ws` writes the workspace to local disk: 40 projects of 4,000 files each, directories of 30,000 and 15,000 entries, and four 250 MiB files, 205,004 files and 4.3 GB in total. About 0.5% of the text files hold the string `ARCHIL_NEEDLE_7f3a`, so `grep` has matches. We copied it into each file system with 32 threads and flushed it before testing. > [accordion] gen_workspace.py > > ```python gen_workspace.py theme={null} > import math, os, random, sys > > ROOT = sys.argv[1] > rng = random.Random(42) > WORDS = ("revenue forecast quarter agent workspace report slide table chart budget customer " > "pipeline import def return class self value index html div span total region " > "summary growth margin cost target plan draft notes meeting todo result data").split() > TEXT = " ".join(rng.choice(WORDS) for _ in range(8_000_000)).encode() > BIN = os.urandom(32 << 20) > TEXT_EXT = ["html", "py", "csv", "md", "json"] > BIN_EXT = ["xlsx", "pptx", "png"] > > > def size(): > # Lognormal, median 10 KiB, clipped to [256 B, 64 KiB) > return int(min(65535, max(256, math.exp(rng.gauss(math.log(10240), 1.0))))) > > > def body(ext, n): > if ext in BIN_EXT: > o = rng.randrange(len(BIN) - n) > return BIN[o:o + n] > o = rng.randrange(len(TEXT) - n) > b = bytearray(TEXT[o:o + n]) > if rng.random() < 0.005 and n > 64: > m = b"ARCHIL_NEEDLE_7f3a" > p = rng.randrange(n - len(m)) > b[p:p + len(m)] = m > return bytes(b) > > > def write(rel, data): > os.makedirs(os.path.dirname(f"{ROOT}/{rel}"), exist_ok=True) > with open(f"{ROOT}/{rel}", "wb") as f: > f.write(data) > > > for p in range(40): > for i in range(4000): > area = ("src", "out", "reports", "scratch", "data")[i % 5] > ext = rng.choice(TEXT_EXT * 3 + BIN_EXT) > write(f"projects/p{p:02d}/{area}/s{(i // 5) % 20:02d}/f{i:05d}.{ext}", body(ext, size())) > for name, n in (("d30k", 30000), ("d15k", 15000)): > for i in range(n): > ext = rng.choice(TEXT_EXT + BIN_EXT) > write(f"bigdirs/{name}/item-{i:06d}.{ext}", body(ext, size())) > for i in range(4): > write(f"media/large{i}.bin", os.urandom(250 << 20)) > ``` ### Tests `WS` is the workspace on the mount under test. Reads used a random sample of small files outside `projects/p00`, the same sample on every file system, and read each whole file with unbuffered 1 MiB reads. | Test | How | | - | - | | `stat` | `os.stat` of 3,000 files, 16 threads | | Small-file read | Open, read and close 4,000 files with 16 threads, or 500 files with 1 thread | | 250 MiB sequential read | 1 MiB reads of `media/large0.bin` | | `grep -rl` | `grep -rl ARCHIL_NEEDLE_7f3a $WS/projects/p00` (4,000 files) | | List a 30,000-entry directory | `os.listdir($WS/bigdirs/d30k)` | | `ls -l` of a 30,000-entry directory | `os.scandir` of `bigdirs/d30k` and `stat` of every entry | | `find` | `find $WS -type f -name '*.py'` | | Create a small file | Create, write and close 4,800 new files in 16 directories, 16 threads, sizes drawn from the same distribution as the workspace | | Make those files durable | Archil: `archil sync`. Others: `sync -f`, and with `--writeback`, wait until the `juicefs_staging_blocks` and `juicefs_staging_writing_blocks` counters in `.stats` reach 0 | | Office save | Write a 256 KiB temporary file in 64 KiB writes, fsync, close, and rename it over one of 16 existing documents. 200 saves with 1 thread, then 400 with 8 threads | | Scratch file | Create, write 16 KiB, close, read back and delete; 3,200 times with 16 threads | | 250 MiB sequential write | 250 writes of 1 MiB to a new file, then fsync; throughput includes the fsync | | Read files written before a restore | Flush, make the client cold, then read 2,000 of the 4,800 files from the create test with 16 threads | ## Archil vs Amazon S3 Files vs Mountpoint for Amazon S3 The latency, visibility, and new-file-system numbers on [archil.com/compare/amazon-s3-files](https://archil.com/compare/amazon-s3-files) and [archil.com/compare/mountpoint-for-amazon-s3](https://archil.com/compare/mountpoint-for-amazon-s3) come from one AWS c7gn.2xlarge (arm64) in us-east-1. Each file system had two independent mounts on that host, a writer A and a reader B, each with its own cache: ```bash theme={null} archil mount dsk-... /mnt/archil-a --region aws-us-east-1 # Archil client v0.9.0, A archil mount --shared dsk-... /mnt/archil-b --region aws-us-east-1 # B mount -t s3files -o nosharecache :/ /mnt/s3files-a # amazon-efs-utils 3.0.1, NFS 4.2 mount-s3 --region us-east-1 --allow-overwrite --allow-delete /mnt/mountpoint-a # Mountpoint 1.22.3 ``` A default Archil mount is exclusive, so B used `--shared`. The S3 Files mount target was in the host's Availability Zone, on a versioned bucket. Mountpoint ran without `--cache`. | Result | How | | - | - | | Mount an existing file system | Wall time of the mount command; median of 10 | | Create a file, write 4 KiB, fsync, close | On A, 300 new files; median | | Second client opens and reads that file | On B, the first open, 4 KiB read and close of 300 files A wrote; median | | Second client sees a newly created file | B lists the directory first, so it is cached. A creates a file and fsyncs it; B checks for it every 10 ms. Time from the start of A's write; median of 8 | | Read 1 GiB sequentially | On B, 1 MiB reads of a 1 GiB file A wrote | | New file system to first durable write | Create the file system, mount it, then write 4 KiB and fsync. Archil: create the disk with the API, median of 5. S3 Files: create a versioned bucket, the file system and its mount target, median of 3. Mountpoint: create a bucket, median of 3 | AWS API calls for the new-file-system test ran from a separate machine, which adds about 0.1 to 0.5 s per call. Archil's API call and every mount and write ran on the host. The measurement script will follow. # Latency Source: https://docs.archil.com/reference/benchmarks/latency One script that measures disk create, delete, mount, checkpoint, branch, 4 KiB cache-miss reads, and durable 4 KiB writes on your own account. These are the numbers in the [latency table on the Limits page](/reference/limits#latency). You can measure every latency row yourself with one script that uses the public API, the `archil` CLI, and `fio`. Run it in the same cloud region as the disk, since a cross-region round trip is larger than most of these figures. It needs `curl`, `jq`, `fio`, `journalctl`, the `archil` CLI, and an [API key](/api-reference/introduction#authentication) in `ARCHIL_API_KEY`. Set `API` to your region's [control plane endpoint](/api-reference/introduction#base-url) and `REGION` to match, then run it as root with `sudo -E bash latency.sh`. It takes about two minutes, creates disks named `latency-test-*`, and deletes them when it finishes. ```bash latency.sh theme={null} #!/usr/bin/env bash set -euo pipefail API="https://" REGION=aws-us-east-1 MNT=/mnt/latency-test N=40 T=$(date +%s) cd "$(mktemp -d)" pct() { sort -n | awk -v p=$1 '{ a[NR] = $1 } END { print a[int((NR * p + 99) / 100)] }'; } stats() { local v=$(cat); echo "p50 $(pct 50 <<< "$v"), p95 $(pct 95 <<< "$v")"; } ms() { local s=$(date +%s%N); "$@" > /dev/null; echo $(( ($(date +%s%N) - s) / 1000000 )); } api() { curl -sf -H "Authorization: Bearer $ARCHIL_API_KEY" -H 'Content-Type: application/json' "$@"; } fiostat() { jq -r ".jobs[0].$1.clat_ns.percentile | \"p50 \(.[\"50.000000\"] / 1e6), p95 \(.[\"95.000000\"] / 1e6)\""; } # Create and delete: request time after the TLS handshake for i in $(seq $N); do api -o create.json -w '%{time_appconnect} %{time_total}\n' -X POST "$API/api/disks" \ -d "{\"name\":\"latency-test-$T-$i\"}" >> create.txt api -o /dev/null -w '%{time_appconnect} %{time_total}\n' -X DELETE \ "$API/api/disks/$(jq -r .data.diskId create.json)" >> delete.txt done echo "create ms: $(awk '{ print ($2 - $1) * 1000 }' create.txt | stats)" echo "delete ms: $(awk '{ print ($2 - $1) * 1000 }' delete.txt | stats)" api -o disk.json -X POST "$API/api/disks" -d "{\"name\":\"latency-test-$T\"}" DISK=$(jq -r .data.diskId disk.json) export ARCHIL_MOUNT_TOKEN=$(jq -r '.data.authorizedUsers[0].token' disk.json) mkdir -p $MNT # Mount: wall time of `archil mount`, and the time the client logs for itself START=$(date '+%F %T') for i in $(seq $N); do ms archil mount -q $DISK $MNT --region $REGION >> mount.txt archil unmount $MNT > /dev/null done echo "mount ms, wall: $(stats < mount.txt)" echo "mount ms, client log: $(journalctl -t "archil-$DISK" --since "$START" \ | grep -o 'Mounted disk in [0-9]*' | awk '{ print $4 }' | stats)" # Checkpoint and branch: wall time of the CLI archil mount -q $DISK $MNT --region $REGION > /dev/null for i in $(seq $N); do ms archil checkpoints create $MNT cp-$i >> checkpoint.txt; done for i in $(seq $N); do ms archil branches create $DISK br-$i --from-checkpoint cp-$i --region $REGION >> branch.txt done echo "checkpoint ms: $(stats < checkpoint.txt)" echo "branch ms: $(stats < branch.txt)" # Durable 4 KiB writes, one at a time FIO="fio --name=t --bs=4k --iodepth=1 --direct=1 --time_based --runtime=30 --output-format=json" echo "O_SYNC write ms: $($FIO --filename=$MNT/w --size=1G --rw=randwrite --sync=1 | fiostat write)" # 4 KiB reads that miss the client cache: a fresh mount has a cold client cache, and the server # still holds the file it just received. Read-ahead off so no read hits locally. fio --name=fill --filename=$MNT/r --size=4G --rw=write --bs=1M --end_fsync=1 > /dev/null archil unmount $MNT > /dev/null archil mount -q $DISK $MNT --region $REGION --read-ahead 0 > /dev/null echo "cache-miss read ms: $($FIO --filename=$MNT/r --size=4G --rw=randread | fiostat read)" archil unmount $MNT > /dev/null api -o /dev/null -X DELETE "$API/api/disks/$DISK" ``` Our run, from a c7gn.2xlarge in AWS `us-east-1` on October 27, 2026: | Script line | p50 | p95 | | - | - | - | | `create ms` | 9.3 ms | 11 ms | | `delete ms` | 8.1 ms | 10 ms | | `mount ms, client log` | 41 ms | 49 ms | | `mount ms, wall` | 50 ms | 58 ms | | `checkpoint ms` | 13 ms | 19 ms | | `branch ms` | 33 ms | 36 ms | | `O_SYNC write ms` | 2.0 ms | 8.2 ms | | `cache-miss read ms` | 0.50 ms | 0.75 ms | * Create and delete leave out the TLS handshake, about 10 ms on a new connection. * `mount ms, wall` includes starting the CLI. The client log line doesn't. * Checkpoint and branch are CLI wall times, so they include starting the CLI, and branch also opens a new TLS connection. * The read test mounts with `--read-ahead 0`. With the default, random reads also pull in neighboring blocks, so about a third of them hit the client cache. # Benchmarks Source: https://docs.archil.com/reference/benchmarks/overview Every benchmark Archil publishes, what it compares, and where to find the script or commands to rerun it. We publish each benchmark with the script or commands that produced it, so you can run it yourself. The numbers use only Archil's public API, released clients, and standard tools, and we include the tests Archil loses. | Benchmark | What it compares | Method and script | Compare pages | | - | - | - | - | | Archil latency | Disk create, delete, mount, checkpoint and branch, 4 KiB cache-miss reads, and durable 4 KiB writes, on your own account | [Latency](/reference/benchmarks/latency) | [Limits](/reference/limits#latency) | | Sandbox storage | An Archil disk and the provider's own storage, mounted in the same sandbox | [Sandbox storage](/reference/benchmarks/sandbox-storage) | [Modal](https://archil.com/compare/modal), [Vercel Sandbox](https://archil.com/compare/vercel-sandbox), [Daytona](https://archil.com/compare/daytona), [sandbox providers](https://archil.com/compare/sandbox-providers) | | Vercel Drive fsync durability | Whether a Drive write can be read from a snapshot, a fork, or a new sandbox once `fsync` returns | [Vercel Drive fsync durability](/reference/benchmarks/vercel-drive-durability) | [Vercel Sandbox](https://archil.com/compare/vercel-sandbox) | | Archil vs Amazon EFS | git, npm, SQLite, and fio on one EC2 host | [File systems on one host](/reference/benchmarks/file-systems#archil-vs-amazon-efs) | [Amazon EFS](https://archil.com/compare/amazon-efs) | | Archil vs JuiceFS vs Amazon S3 Files | git clone and npm ci on one EC2 host, timed until durable | [File systems on one host](/reference/benchmarks/file-systems#archil-vs-juicefs-vs-amazon-s3-files) | [JuiceFS](https://archil.com/compare/juicefs), [Amazon S3 Files](https://archil.com/compare/amazon-s3-files) | | Archil vs Amazon S3 Files vs Mountpoint for Amazon S3 | Mount, write and read latency, cross-client visibility, and time to a new file system | [File systems on one host](/reference/benchmarks/file-systems#archil-vs-amazon-s3-files-vs-mountpoint-for-amazon-s3); script to follow | [Amazon S3 Files](https://archil.com/compare/amazon-s3-files), [Mountpoint for Amazon S3](https://archil.com/compare/mountpoint-for-amazon-s3) | If your results differ from ours, email [support@archil.com](mailto:support@archil.com) with your output and setup. # Sandbox storage Source: https://docs.archil.com/reference/benchmarks/sandbox-storage One Node script and a recipe per sandbox provider to rerun the storage numbers on the Archil compare pages yourself. The storage numbers on our compare pages for [Modal](https://archil.com/compare/modal), [Vercel Sandbox](https://archil.com/compare/vercel-sandbox), [Daytona](https://archil.com/compare/daytona), and [sandbox providers](https://archil.com/compare/sandbox-providers), including Fly.io Sprites, come from the script on this page. In each provider's sandbox, we mounted the provider's own storage and an Archil disk side by side and ran the same script against both, so the compute is identical and only the storage differs. You can run it on your own accounts with the recipes below. ## What it measures | Metric | How | Runs | | - | - | - | | Create a 4 KiB file and fsync it | `open`, `write`, `fsync`, `close` of a new file | 100 in a row; p50 and p90 | | Write 1 GiB sequentially and fsync | 1 MiB writes of random data, then one `fsync` | 3; median | | `npm ci`, 10,538 files | `npm ci --offline` from a local npm cache, so the registry isn't timed | 3; median | | Delete that tree | `rm -rf node_modules` | 3; median | | Read the 1 GiB back from a new sandbox | `read` mode in a sandbox that didn't write the file; sha256 checked | 1 in each of 3 new sandboxes; median | | Another sandbox sees a new file | `watch` in one sandbox while `mark` writes the file in another; 60 s limit | 5 rounds; median | | Create the storage | The provider's create call, from your machine, until the store can be attached | 5; median | | Fork | Checkpoint the Archil disk, branch it, mount the branch, read the 1 GiB file | 3; median | The recipes run the visibility, provisioning, and fork steps once; we repeated them for the counts above. Daytona visibility and provisioning used 10 rounds. Everything except provisioning, the branch call, and the branch mount is timed inside the sandbox, so the provider's API round trips aren't counted. Setup is untimed: the script writes a 1 GiB random source file to local disk, resolves the lockfile, and fills the npm cache, which is its only network use. Pinned to a registry date, `npm` resolves the same 501 packages every time. Some stores acknowledge `fsync` before data leaves the host. An Archil `fsync` returns once the write is durable on Archil's replicated storage. Our compare pages mark the rows where the other store's write was not durable when `fsync` returned, and [Vercel Drive fsync durability](/reference/benchmarks/vercel-drive-durability) tests it directly. ## The script Save it as `bench.mjs`. It needs Node 18 or later and npm, and nothing else. ```javascript bench.mjs theme={null} // Sandbox storage benchmark: https://docs.archil.com/reference/benchmarks/sandbox-storage // Pure Node (18+) and npm, no other dependencies. Every mode prints one JSON object. // node bench.mjs 4 KiB create + fsync, 1 GiB write + fsync, npm ci, rm -rf // node bench.mjs read read the 1 GiB file back and check it (run in a new sandbox) // node bench.mjs mark write a small file, for the visibility test // node bench.mjs watch wait up to --timeout seconds for that file to appear // Options: --runs 3 --creates 100 --timeout 60 --commit "" import { execFileSync } from "node:child_process" import crypto from "node:crypto" import fs from "node:fs" import os from "node:os" import path from "node:path" const args = process.argv.slice(2) const opt = (name, def) => { const i = args.indexOf(`--${name}`); return i < 0 ? def : args.splice(i, 2)[1] } const RUNS = Number(opt("runs", 3)), CREATES = Number(opt("creates", 100)), TIMEOUT = Number(opt("timeout", 60)) const COMMIT = opt("commit", "") const [dir, mode = "write", name = "visible.txt"] = args if (!dir) { console.error("usage: node bench.mjs [write | read | mark | watch ]"); process.exit(2) } // Pinned to this registry date, npm resolves the same 501 packages (10,538 files) every time. const PACKAGES = { "@babel/core": "7.26.0", "@babel/preset-env": "7.26.0", eslint: "8.57.1", jest: "29.7.0", lodash: "4.17.21", react: "18.3.1", "react-dom": "18.3.1", typescript: "5.6.3", webpack: "5.96.1", "webpack-cli": "5.1.4" } const NPM_BEFORE = "2026-11-02T17:10:00Z" const MiB = 1 << 20, GiB = 1 << 30 const work = path.join(dir, "archil-bench"), seqFile = path.join(work, "seq.bin"), shaFile = path.join(work, "seq.sha256") const tmp = path.join(os.tmpdir(), "archil-bench") const bin = path.dirname(process.execPath), env = { ...process.env, PATH: `${bin}:${process.env.PATH}` } const NPM = fs.existsSync(path.join(bin, "npm")) ? path.join(bin, "npm") : "npm" const now = () => performance.now() const tag = () => crypto.randomBytes(4).toString("hex") const sorted = (xs) => [...xs].sort((a, b) => a - b) const median = (xs) => { const s = sorted(xs), m = s.length >> 1; return s.length % 2 ? s[m] : (s[m - 1] + s[m]) / 2 } const round = (x, d = 2) => Number(x.toFixed(d)) const log = (msg) => console.error(`[bench] ${msg}`) // Some stores persist only on an explicit commit (a Modal Volume: `sync `), so it runs inside the timing. const commit = () => { if (COMMIT) execFileSync("sh", ["-c", COMMIT], { stdio: "inherit" }) } const npm = (cwd, argv, timeout) => execFileSync(NPM, [...argv, "--cache", path.join(tmp, "npm-cache"), "--ignore-scripts", "--no-audit", "--no-fund", "--loglevel=error"], { cwd, env, stdio: "pipe", timeout }) const writeDurable = (file, data) => { const fd = fs.openSync(file, "w"); fs.writeSync(fd, data); fs.fsyncSync(fd); fs.closeSync(fd); commit() } function readAll(file, onChunk) { const fd = fs.openSync(file, "r"), buf = Buffer.alloc(MiB) let total = 0 for (let n; (n = fs.readSync(fd, buf, 0, MiB, null)) > 0; total += n) onChunk?.(buf.subarray(0, n)) fs.closeSync(fd) return total } // Untimed: a 1 GiB random source on local disk, and a lockfile plus a filled npm cache (the only network use). function setup() { fs.mkdirSync(tmp, { recursive: true }) const src = path.join(tmp, "rand.bin"), app = path.join(tmp, "app") if (!fs.existsSync(src) || fs.statSync(src).size !== GiB) { log("writing a 1 GiB random source file to local disk") const fd = fs.openSync(src, "w") for (let i = 0; i < 1024; i++) fs.writeSync(fd, crypto.randomBytes(MiB)) fs.closeSync(fd) } if (!fs.existsSync(path.join(app, "package-lock.json"))) { log("resolving the lockfile and filling the npm cache") fs.mkdirSync(app, { recursive: true }) fs.writeFileSync(path.join(app, "package.json"), JSON.stringify({ name: "bench-app", version: "1.0.0", private: true, dependencies: PACKAGES })) npm(app, ["install", `--before=${NPM_BEFORE}`]) fs.rmSync(path.join(app, "node_modules"), { recursive: true, force: true }) } const hash = crypto.createHash("sha256") readAll(src, (b) => hash.update(b)) return { src, app, sha: hash.digest("hex") } } const result = { dir, mode, node: process.version } if (mode === "write" || mode === "mark") fs.mkdirSync(work, { recursive: true }) if (mode === "write") { const { src, app, sha } = setup() log(`creating ${CREATES} files of 4 KiB, each with fsync`) const createDir = path.join(work, `create-${tag()}`), page = crypto.randomBytes(4096), ms = [] fs.mkdirSync(createDir) for (let i = 0; i < CREATES; i++) { const t = now() const fd = fs.openSync(path.join(createDir, `f${i}`), "wx") fs.writeSync(fd, page); fs.fsyncSync(fd); fs.closeSync(fd); commit() ms.push(now() - t) } result.create_4k_fsync_ms = { n: CREATES, p50: round(median(ms), 3), p90: round(sorted(ms)[Math.floor(CREATES * 0.9)], 3) } const mbps = [] for (let r = 0; r < RUNS; r++) { log(`writing 1 GiB with fsync, run ${r + 1} of ${RUNS}`) const t = now(), fd = fs.openSync(seqFile, "w") const total = readAll(src, (b) => fs.writeSync(fd, b)) fs.fsyncSync(fd); fs.closeSync(fd); commit() mbps.push(total / 1e6 / ((now() - t) / 1000)) } writeDurable(shaFile, sha) result.write_1gib_fsync_mb_s = { runs: mbps.map((x) => round(x, 0)), median: round(median(mbps), 0) } const installs = [], removes = [] for (let r = 0; r < RUNS; r++) { log(`npm ci from the local cache, run ${r + 1} of ${RUNS}`) const d = path.join(work, `app-${tag()}`) fs.mkdirSync(d) // Plain read and write: some FUSE stores reject copy_file_range, which copyFileSync uses. for (const f of ["package.json", "package-lock.json"]) fs.writeFileSync(path.join(d, f), fs.readFileSync(path.join(app, f))) let t = now() try { npm(d, ["ci", "--offline"], Number(process.env.NPM_TIMEOUT_S || 900) * 1000) } catch (e) { result.npm_ci_s = { failed: true, timed_out: e.signal === "SIGTERM", seconds: round((now() - t) / 1000), error: String(e.stderr || e.message).slice(0, 400) } break } commit() installs.push((now() - t) / 1000) result.npm_files = Number(execFileSync("sh", ["-c", `find "${d}/node_modules" -type f | wc -l`]).toString().trim()) t = now() fs.rmSync(path.join(d, "node_modules"), { recursive: true, force: true }); commit() removes.push((now() - t) / 1000) fs.rmSync(d, { recursive: true, force: true }) } if (installs.length === RUNS) { result.npm_ci_s = { runs: installs.map((x) => round(x)), median: round(median(installs)) } result.rm_rf_s = { runs: removes.map((x) => round(x)), median: round(median(removes)) } } fs.rmSync(createDir, { recursive: true, force: true }) } else if (mode === "read") { const want = fs.readFileSync(shaFile, "utf8").trim() const t = now(), total = readAll(seqFile) const s = (now() - t) / 1000 const hash = crypto.createHash("sha256") readAll(seqFile, (b) => hash.update(b)) result.read_1gib_mb_s = round(total / 1e6 / s, 0) result.sha_ok = hash.digest("hex") === want } else if (mode === "mark") { writeDurable(path.join(work, name), String(Date.now())) result.written_at_ms = Date.now() } else if (mode === "watch") { const file = path.join(work, name), limit = Date.now() + TIMEOUT * 1000 for (;;) { const body = fs.existsSync(file) ? fs.readFileSync(file, "utf8") : "" // Lag uses both sandboxes' wall clocks, so it carries their clock skew (usually a few ms). if (body) { result.visible_after_ms = Date.now() - Number(body); break } if (Date.now() > limit) { result.visible_after_ms = null; result.not_seen_within_s = TIMEOUT; break } await new Promise((r) => setTimeout(r, 50)) } } else { console.error(`unknown mode ${mode}`); process.exit(2) } console.log(JSON.stringify(result)) ``` ```bash theme={null} node bench.mjs /vol # the provider's storage node bench.mjs /mnt/archil # an Archil disk in the same sandbox node bench.mjs /mnt/archil read # in a new sandbox: read the 1 GiB file back ``` It writes under `/archil-bench/` and prints one JSON object per run: ```json theme={null} { "dir": "/mnt/archil", "mode": "write", "node": "v24.22.1", "create_4k_fsync_ms": { "n": 100, "p50": 6.463, "p90": 13.602 }, "write_1gib_fsync_mb_s": { "runs": [ 371, 409, 421 ], "median": 409 }, "npm_files": 10538, "npm_ci_s": { "runs": [ 3.49, 3.48, 3.47 ], "median": 3.48 }, "rm_rf_s": { "runs": [ 0.81, 0.79, 0.83 ], "median": 0.81 } } ``` If a store persists only on an explicit commit, pass the commit command with `--commit`; it runs inside every timed write. A Modal Volume needs `--commit 'sync /vol'`. ## Mount an Archil disk Create the disk from your own machine, so your API key stays there. The sandbox only gets the disk's [mount token](/concepts/authorization#token-based-disk-authorization), which grants access to that disk alone: ```bash theme={null} export ARCHIL_API_KEY=key-... export ARCHIL_REGION=aws-us-west-2 npx disk create bench # prints the disk ID and a mount token ``` Then, in the sandbox, as root: ```bash theme={null} curl -fsSL https://archil.com/install | sh mkdir -p /mnt/archil ARCHIL_MOUNT_TOKEN=adt_... archil mount dsk-... /mnt/archil --region aws-us-west-2 ``` Put the disk in the [Archil region](/reference/regions) closest to the sandbox. We publish Archil side by side only where a TCP connection from the sandbox to Archil took about 20 ms or less; Daytona's `us` region and Fly.io Sprites were farther away, so those pages show the provider's storage alone. ## Recipes Each recipe creates the stores, runs the script, and deletes everything it created. ### Vercel Sandbox and Vercel Drive Reproduces every row on [archil.com/compare/vercel-sandbox](https://archil.com/compare/vercel-sandbox): one sandbox with 2 vCPUs in `pdx1` (AWS us-west-2), a Vercel Drive in `pdx1`, and an Archil disk in `aws-us-west-2`. ```bash theme={null} npm init -y && npm pkg set type=module npm install disk@^1.11.0 @vercel/sandbox tsx vercel link && vercel env pull export ARCHIL_API_KEY=key-... ARCHIL_REGION=aws-us-west-2 npx tsx --env-file=.env.local vercel.ts ``` ```typescript vercel.ts theme={null} import { readFileSync } from "node:fs"; import { Drive, Sandbox } from "@vercel/sandbox"; import * as archil from "disk"; const REGION = "pdx1"; // AWS us-west-2, next to an Archil disk in aws-us-west-2 const ARCHIL_REGION = process.env.ARCHIL_REGION!; // aws-us-west-2 const script = readFileSync("bench.mjs"); const seconds = (t: number) => ((performance.now() - t) / 1000).toFixed(3); // Provisioning, timed from your machine let t = performance.now(); const drive = await Drive.getOrCreate({ name: `bench-${Date.now()}`, region: REGION }); console.log("create Drive s", seconds(t)); t = performance.now(); const { disk, token } = await archil.createDisk({ name: `bench-${Date.now()}` }); console.log("create Archil disk s", seconds(t)); async function run(sandbox: Sandbox, cmd: string, env: Record = {}) { const result = await sandbox.runCommand({ cmd: "bash", args: ["-c", cmd], env, sudo: true }); if (result.exitCode !== 0) throw new Error(await result.stderr()); return (await result.stdout()).trim(); } // A new sandbox with the Drive (or a snapshot of it) at /vol and the Archil disk at /mnt/archil const sandboxes: Sandbox[] = []; const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); // A Drive takes one read-write mount at a time, so wait until the previous sandbox has released it const released = async () => { while ((await Drive.get({ name: drive.name })).currentSandboxName) await sleep(2000); }; async function start(vol: Drive | { drive: string; mode: "snapshot" }, flags = "") { if (vol instanceof Drive) await released(); const sandbox = await Sandbox.create({ region: REGION, resources: { vcpus: 2 }, mounts: { "/vol": vol }, persistent: false, timeout: 45 * 60_000 }); sandboxes.push(sandbox); await sandbox.writeFiles([{ path: "/tmp/bench.mjs", content: script }]); await run(sandbox, "curl -fsSL https://archil.com/install | sh > /dev/null"); await run(sandbox, `mkdir -p /mnt/archil && archil mount ${disk.id} /mnt/archil --region ${ARCHIL_REGION} ${flags}`, { ARCHIL_MOUNT_TOKEN: token, }); return sandbox; } try { // 4 KiB creates, 1 GiB write, npm ci and rm -rf on each store, in the same sandbox const a = await start(drive); console.log(await run(a, "node /tmp/bench.mjs /vol")); console.log(await run(a, "node /tmp/bench.mjs /mnt/archil")); // Visibility: a second sandbox watches while the first writes a file const b = await start(drive.snapshot(), "--shared"); for (const dir of ["/vol", "/mnt/archil"]) { const name = `visible-${Date.now()}`; const watch = await b.runCommand({ cmd: "bash", args: ["-c", `node /tmp/bench.mjs ${dir} watch ${name}`], sudo: true, detached: true }); await sleep(2000); await run(a, `node /tmp/bench.mjs ${dir} mark ${name}`); console.log((await (await watch.wait()).stdout()).trim()); } await run(b, "archil unmount /mnt/archil"); await b.stop(); await run(a, "archil unmount /mnt/archil"); await a.stop(); // Cold reads: three new sandboxes each read the 1 GiB file back for (let i = 0; i < 3; i++) { const c = await start(drive); console.log(await run(c, "node /tmp/bench.mjs /vol read")); console.log(await run(c, "node /tmp/bench.mjs /mnt/archil read")); if (i === 2) { // Fork: checkpoint the disk, branch it from your machine, mount the branch, read the file const ms = await run(c, "s=$(date +%s%N); archil checkpoints create /mnt/archil cp-1 > /dev/null; echo $(( ($(date +%s%N) - s) / 1000000 ))"); console.log("checkpoint s", Number(ms) / 1000); t = performance.now(); await (await archil.getDisk(disk.id)).createBranch({ name: "br-1", fromCheckpoint: "cp-1" }); console.log("branch s", seconds(t)); t = performance.now(); await run(c, `mkdir -p /mnt/branch && archil mount ${disk.id} /mnt/branch --region ${ARCHIL_REGION} --branch br-1`, { ARCHIL_MOUNT_TOKEN: token, }); console.log("mount branch s", seconds(t)); console.log(await run(c, "node /tmp/bench.mjs /mnt/branch read && archil unmount /mnt/branch")); } await run(c, "archil unmount /mnt/archil"); await c.stop(); } } finally { for (const sandbox of sandboxes) await sandbox.stop().catch(() => {}); await released(); await drive.delete(); await (await archil.getDisk(disk.id)).delete(); } ``` A Drive takes one read-write mount at a time. The visibility sandbox mounts a read-only snapshot of it, which is fixed when the sandbox starts, so it doesn't see the new file. It mounts the Archil disk with `--shared`. ### Modal Volume Reproduces [archil.com/compare/modal](https://archil.com/compare/modal): a VM sandbox (`runtime="vm"`) with 2 CPUs and 4 GiB in `us-west`, a Modal Volume (v2), and an Archil disk in `aws-us-west-2`. For the fork row, add the fork step from the Vercel recipe. ```bash theme={null} pip install archil modal && modal setup export ARCHIL_API_KEY=key-... ARCHIL_REGION=aws-us-west-2 python modal_bench.py ``` ```python modal_bench.py theme={null} import time import archil import modal ARCHIL_REGION = "aws-us-west-2" NODE = "curl -fsSL https://nodejs.org/dist/v22.20.0/node-v22.20.0-linux-x64.tar.xz | tar xJ -C /usr/local --strip-components=1" app = modal.App.lookup("archil-bench", create_if_missing=True) image = ( modal.Image.debian_slim() .apt_install("curl", "ca-certificates", "xz-utils", "fuse3") .run_commands(NODE, "curl -fsSL https://archil.com/install | sh") .add_local_file("bench.mjs", "/opt/bench.mjs") ) # Provisioning, timed from your machine name = f"bench-{int(time.time())}" t = time.perf_counter() modal.Volume.objects.create(name, version=2) print("create Volume s", round(time.perf_counter() - t, 3)) vol = modal.Volume.from_name(name, version=2) t = time.perf_counter() created = archil.create_disk(name=name) print("create Archil disk s", round(time.perf_counter() - t, 3)) disk = created.disk token = modal.Secret.from_dict({"ARCHIL_MOUNT_TOKEN": created.token}) sandboxes = [] def run(sb, cmd): p = sb.exec("bash", "-c", cmd) p.wait() if p.returncode != 0: raise RuntimeError(p.stderr.read()) return p.stdout.read().strip() def start(flags=""): # The Volume at /vol and the Archil disk at /mnt/archil, side by side sb = modal.Sandbox.create( "sleep", "infinity", app=app, image=image, volumes={"/vol": vol}, secrets=[token], cpu=2, memory=4096, region="us-west", runtime="vm", timeout=3600, ) sandboxes.append(sb) run(sb, f"mkdir -p /mnt/archil && archil mount {disk.id} /mnt/archil --region {ARCHIL_REGION} {flags}") return sb try: # A Modal Volume persists on `sync `, so --commit runs it inside every timed write a = start() print(run(a, "node /opt/bench.mjs /vol --commit 'sync /vol'")) print(run(a, "node /opt/bench.mjs /mnt/archil")) # Visibility: a second sandbox watches while the first writes a file b = start("--shared") for path, commit in (("/vol", "--commit 'sync /vol'"), ("/mnt/archil", "")): mark = f"visible-{int(time.time())}" watch = b.exec("bash", "-c", f"node /opt/bench.mjs {path} watch {mark}") time.sleep(2) run(a, f"node /opt/bench.mjs {path} mark {mark} {commit}") watch.wait() print(watch.stdout.read().strip()) run(b, "archil unmount /mnt/archil") run(a, "archil unmount /mnt/archil") # Cold reads: three new sandboxes each read the 1 GiB file back for i in range(3): c = start() print(run(c, "node /opt/bench.mjs /vol read")) print(run(c, "node /opt/bench.mjs /mnt/archil read")) run(c, "archil unmount /mnt/archil") c.terminate() finally: for sb in sandboxes: sb.terminate() modal.Volume.objects.delete(name) archil.get_disk(disk.id).delete() ``` A running Modal sandbox sees another sandbox's Volume commit only after a reload, so the Volume visibility round is expected to reach its 60 s limit. ### Daytona Volume Reproduces [archil.com/compare/daytona](https://archil.com/compare/daytona): sandboxes from the default snapshot (1 vCPU, 1 GiB) in Daytona's `us` region, each with a Daytona Volume at `/vol`. To add an Archil disk, mount it as in the [Daytona guide](/guides/platforms/daytona) and run the script against `/mnt/archil` too. ```bash theme={null} pip install daytona export DAYTONA_API_KEY=dtn_... python daytona_bench.py ``` ```python daytona_bench.py theme={null} import shlex import time from daytona import CreateSandboxFromSnapshotParams, Daytona, VolumeMount NODE = "curl -fsSL https://nodejs.org/dist/v22.20.0/node-v22.20.0-linux-x64.tar.xz | tar xJ -C /usr/local --strip-components=1" daytona = Daytona() script = open("bench.mjs", "rb").read() # Provisioning, timed from your machine until the Volume is ready name = f"bench-{int(time.time())}" t = time.perf_counter() vol = daytona.volume.create(name) while vol.state != "ready": time.sleep(0.25) vol = daytona.volume.get(name) print("create Volume s", round(time.perf_counter() - t, 3)) sandboxes = [] def run(sb, cmd, timeout=1800): r = sb.process.exec(f"sudo bash -c {shlex.quote(cmd)}", timeout=timeout) if r.exit_code != 0: raise RuntimeError(r.result) return r.result.strip() def start(): sb = daytona.create(CreateSandboxFromSnapshotParams(volumes=[VolumeMount(volume_id=vol.id, mount_path="/vol")])) sandboxes.append(sb) sb.fs.upload_file(script, "/tmp/bench.mjs") run(sb, NODE) return sb try: a = start() print(run(a, "node /tmp/bench.mjs /vol", timeout=3600)) # Visibility: a second sandbox watches while the first writes a file b = start() mark = f"visible-{int(time.time())}" run(b, f"nohup node /tmp/bench.mjs /vol watch {mark} > /tmp/watch.json 2> /dev/null &") time.sleep(2) run(a, f"node /tmp/bench.mjs /vol mark {mark}") print(run(b, "until [ -s /tmp/watch.json ]; do sleep 0.2; done; cat /tmp/watch.json", timeout=90)) # Cold reads: three new sandboxes each read the 1 GiB file back for i in range(3): print(run(start(), "node /tmp/bench.mjs /vol read")) finally: for sb in sandboxes: sb.delete() daytona.volume.delete(daytona.volume.get(name)) ``` In our runs, `npm ci` on a Daytona Volume didn't finish within 15 minutes. The script stops it after 15 minutes (`NPM_TIMEOUT_S` changes the limit) and reports `timed_out`. ### Fly.io Sprites Reproduces the Fly.io Sprites results on [archil.com/compare/sandbox-providers](https://archil.com/compare/sandbox-providers): one Sprite measuring its own persistent disk. A Sprite's disk can't attach to another Sprite, so the cold read drops the page cache in the same Sprite, and the visibility row doesn't apply. To add an Archil disk, mount it as in the [Sprites guide](/guides/platforms/sprites). ```bash theme={null} time (sprite create bench-1 --skip-console && sprite exec --no-stdin -s bench-1 -- true) # provisioning sprite exec --no-stdin -s bench-1 --file bench.mjs:/home/sprite/bench.mjs -- mkdir -p /home/sprite/vol sprite exec --no-stdin -s bench-1 -- node /home/sprite/bench.mjs /home/sprite/vol for i in 1 2 3; do sprite exec --no-stdin -s bench-1 -- sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches' sprite exec --no-stdin -s bench-1 -- node /home/sprite/bench.mjs /home/sprite/vol read done time sprite checkpoint create -s bench-1 # checkpoint sprite destroy -s bench-1 --force ``` ## If your results differ Email [support@archil.com](mailto:support@archil.com) with the script output, the provider, and the regions of the sandbox and the disk. # Vercel Drive fsync durability Source: https://docs.archil.com/reference/benchmarks/vercel-drive-durability Whether a Vercel Drive write can be read through a snapshot, a fork, or a new sandbox once fsync has returned, with the script to rerun it. When `fsync` returns, POSIX says the data is on stable storage. This test checks what another reader of a Vercel Drive sees after the writer's `fsync` has returned. Sandbox A mounts a Drive read-write, writes a 4 KiB file and a 64 MiB file with `dd conv=fsync`, runs `sync` on the directory, and records each file's sha256. After a delay, the script reads the Drive without going through sandbox A: * **snapshot:** mount `drive.snapshot()` in a new sandbox. Vercel describes a snapshot as the Drive "as it was at the moment you mounted it". * **snapshot, writer still writing:** the same, while sandbox A keeps writing other data to the Drive. * **fork:** `drive.fork()`, then mount the fork in a new sandbox. * **stop:** call `sandbox.stop()` on sandbox A right after `fsync`, then mount the Drive read-write in a new sandbox. Each file is reported as `ok`, `missing` (not present), or `corrupt` (wrong sha256). ## Results Region `pdx1`, 10 trials per cell. The delay is the time from `fsync` returning to the snapshot or fork request; the reading sandbox takes about another 0.8 s to start. | Read path | 0 s | 0.1 s | 1 s | 5 s | 30 s | 60 s | 120 s | | - | - | - | - | - | - | - | - | | Snapshot | 10/10 missing | 10/10 missing | 10/10 missing | 8/10 missing | 7/10 missing | 10/10 ok | 10/10 ok | | Snapshot, writer still writing | 11/11 missing | 10/10 missing | 10/10 missing | 8/10 missing | 4/10 corrupt | 1/10 missing | 10/10 ok | | Fork | 11/11 missing | 10/10 missing | 8/8 missing | 9/10 missing | 9/10 missing | 10/10 ok | 10/10 ok | A Drive `fsync` is therefore not durable: when it returns, the write hasn't reached the storage that snapshots and forks are made from. * **Corrupt copies:** the 64 MiB file was full size, but its last 12 to 28 MiB read as zeros. Reading it again 60 s later, with the reader's page cache dropped, returned the same zeros. * **Stop, then mount read-write:** the file was intact in 10 of 10 trials, and the Drive could be mounted again about 3 s after `fsync` returned. Stopping a sandbox lets Vercel flush the host before handing the Drive to the next sandbox, so this doesn't show what happens if the host fails, which a user can't test. * **Archil:** an Archil disk mounted in a new sandbox right after `fsync` returned had both files intact in 20 of 20 trials. An Archil `fsync` returns once the write is stored redundantly across Availability Zones; see [Data durability](/details/architecture#data-durability). ## Run it The script uses only the Vercel Python SDK and standard tools in the sandbox. It creates its own Drive and sandboxes and deletes them when it finishes; use a dedicated project. ```bash theme={null} pip install vercel export VERCEL_TOKEN=... VERCEL_TEAM_ID=team_... VERCEL_PROJECT_ID=prj_... python vercel-fsync-durability.py --region pdx1 --trials 10 --delays 0,0.1,1,5,30,60,120 \ --variants snapshot,snapshot-busy,fork,stop --out results.jsonl ``` > [accordion] vercel-fsync-durability.py > > ```python vercel-fsync-durability.py theme={null} > #!/usr/bin/env python3 > """Does a Vercel Drive hold a file once fsync has returned? > > Sandbox A mounts a Drive read-write, writes two fresh files (4 KiB and 64 MiB), fsyncs each file and the directory, > and reports each file's sha256. After a chosen delay, the script reads the Drive through a path that doesn't go > through sandbox A, and checks both files: > > snapshot mount drive.snapshot() in a new sandbox B (A stays mounted and idle) > snapshot-busy the same, while A keeps writing other data to the Drive > fork drive.fork(), then mount the fork read-write in a new sandbox B > stop sandbox.stop() on A right after fsync, then mount the Drive read-write in a new sandbox B > (no delay sweep; records how long until the Drive can be mounted again) > > Each check is "ok", "missing" or "corrupt". A file that is missing or corrupt after its fsync returned means that > path did not see data the writer had already fsynced. > > Each trial runs in its own child process with a hard timeout, so a stuck SDK call costs one trial, not the run. > > Requires: pip install vercel (the Vercel Python SDK); VERCEL_TOKEN, VERCEL_TEAM_ID and VERCEL_PROJECT_ID set. > usage: python vercel-fsync-durability.py [--region pdx1] [--trials 10] [--delays 0,0.1,1,5,30] > [--variants snapshot,snapshot-busy,fork,stop] [--out results.jsonl] > Everything the script creates (sandboxes, the drive, forks) is named fsync--* and deleted at the end. > """ > import argparse > import json > import secrets > import subprocess > import sys > import time > from collections import Counter > > from vercel.sandbox import DriveQueryByName, SandboxQueryByName, SandboxResources, sync > > MNT = "/data" > BIG_BYTES = 64 << 20 > LIMIT_S = 4 * 3600 > REREAD_AFTER_S = 60 > > # Payloads are a deterministic AES-CTR keystream per file id, so a reader can regenerate the expected bytes and say > # exactly which 1 MiB chunks differ when a file is corrupt. > GEN = "openssl enc -aes-128-ctr -nosalt -pbkdf2 -pass pass:$1-{k} < /dev/zero 2>/dev/null | head -c {n}" > > WRITE = ( > GEN.format(k="small", n=4096) + " > /tmp/$1.small\n" + GEN.format(k="big", n=BIG_BYTES) + " > /tmp/$1.big\n" > + r"""set -e > s=$(sha256sum < /tmp/$1.small | cut -c1-64) > b=$(sha256sum < /tmp/$1.big | cut -c1-64) > dd if=/tmp/$1.small of=/data/$1.small bs=4M conv=fsync,excl status=none > dd if=/tmp/$1.big of=/data/$1.big bs=4M conv=fsync,excl status=none > sync /data > rm -f /tmp/$1.small /tmp/$1.big > echo "FSYNCED $s $b" > """ > ) > > CHECK = r""" > for k in small big; do > f=$2/$1.$k > if [ -e "$f" ]; then echo "$k $(stat -c %s "$f") $(sha256sum < "$f" | cut -c1-64)"; else echo "$k MISSING"; fi > done > """ > > # For a corrupt file: the exact byte ranges that differ from the expected bytes, and whether each range is all zeros. > DIFF = GEN.format(k="big", n=BIG_BYTES) + r""" > /tmp/want > python3 - "$2/$1.big" <<'PY' > import sys, json > want = open('/tmp/want', 'rb').read(); got = open(sys.argv[1], 'rb').read() > ranges, i, n = [], 0, min(len(want), len(got)) > B = 4096 > while i < n: > if want[i:i+B] == got[i:i+B]: > i += B > continue > j = i > while j < n and want[j:j+B] != got[j:j+B]: > j += B > # Narrow both ends to the byte. > a = i > while want[a] == got[a]: > a += 1 > b = min(j, n) > while want[b-1] == got[b-1]: > b -= 1 > ranges.append({"start": a, "end": b, "all_zero": got[a:b].count(0) == b - a}) > i = j > print("DIFF " + json.dumps({"size": len(got), "ranges": ranges[:20], "range_count": len(ranges), > "bytes_wrong": sum(r["end"] - r["start"] for r in ranges)})) > PY > """ > > BUSY = "nohup setsid bash -c 'while :; do dd if=/dev/urandom of=/data/noise bs=1M count=16 conv=fsync status=none; done' >/dev/null 2>&1 &" > > > def sh(sb, script, *args, timeout=600): > r = sb.run_process("bash", ["-c", script, "x", *args], sudo=True, capture_output=True, kill_after=timeout) > return r.returncode, (r.stdout or "") + (r.stderr or "") > > > def new_sandbox(region, name, mounts, retry_s=60): > t, last = time.monotonic(), None > while True: > try: > sb = sync.create_sandbox(name=name, region=region, mounts=mounts, resources=SandboxResources(vcpus=2), > execution_time_limit=LIMIT_S, persistent=False) > return sb, time.monotonic() - t, last > except Exception as e: # noqa: BLE001 > last = str(e)[:200] > if time.monotonic() - t > retry_s: > raise > time.sleep(1) > > > def drop(sb): > for step in (sb.stop, sb.destroy): > try: > step() > except Exception: # noqa: BLE001 > pass > > > def check(sb, file_id, want, directory): > _, out = sh(sb, CHECK, file_id, directory) > result = {} > for k in ("small", "big"): > line = next((l for l in out.splitlines() if l.startswith(k + " ")), f"{k} MISSING") > parts = line.split() > result[k] = "missing" if parts[1] == "MISSING" else ("ok" if parts[2] == want[k] else "corrupt") > if result[k] == "corrupt": > result[f"{k}_size"] = int(parts[1]) > if result["big"] == "corrupt": > diff = sh(sb, DIFF, file_id, directory)[1] > line = next((l for l in diff.splitlines() if l.startswith("DIFF ")), None) > result["big_diff"] = json.loads(line[5:]) if line else diff[-300:] > # Does the same reader see the right bytes later? Drop its page cache so the re-read goes to the device. > time.sleep(REREAD_AFTER_S) > _, again = sh(sb, "sync; echo 3 > /proc/sys/vm/drop_caches; sha256sum < \"$2/$1.big\" | cut -c1-64", file_id, directory) > result["big_reread_after_s"] = REREAD_AFTER_S > result["big_reread"] = "ok" if want["big"] in again else "still corrupt" > return result > > > def child(job): > """One trial. Prints one RESULT line; for `stop` it also names the sandbox that now holds the Drive.""" > region, run, variant, delay, trial, writer, seq = (job[k] for k in ("region", "run", "variant", "delay", "trial", "writer", "seq")) > drive = sync.get_drive(name_or_id=f"fsync-{run}") > a_sb = sync.get_sandbox(name=writer) > file_id = f"{run}-{variant}-{delay}-{trial}" > code, out = sh(a_sb, WRITE, file_id) > line = next((l for l in out.splitlines() if l.startswith("FSYNCED ")), None) > if code != 0 or not line: > raise RuntimeError(f"write failed ({code}): {out[-300:]}") > _, s, b = line.split() > want = {"small": s, "big": b} > t_fsync, w_fsync = time.monotonic(), time.time() > rec = {"variant": variant, "delay_s": delay, "trial": trial, "fsync_returned_utc": iso(w_fsync)} > if variant == "stop": > a_sb.stop() > rec["stop_call_s"] = round(time.monotonic() - t_fsync, 3) > b_sb, wait_s, last = new_sandbox(region, f"fsync-{run}-{seq}", {MNT: drive}, retry_s=1500) > rec.update({"remount_after_stop_s": round(time.monotonic() - t_fsync, 3), "remount_wait_s": round(wait_s, 3), > "last_remount_error": last, "next_writer": b_sb.name}) > rec.update(check(b_sb, file_id, want, MNT)) > else: > time.sleep(delay) > rec["gap_fsync_to_request_s"] = round(time.monotonic() - t_fsync, 4) > rec["request_utc"] = iso(time.time()) > rec["wall_gap_s"] = round(time.time() - w_fsync, 4) > # Monotonic time stops while the orchestrating machine sleeps; a wall/monotonic mismatch flags the trial. > rec["orchestrator_slept"] = rec["wall_gap_s"] - rec["gap_fsync_to_request_s"] > 1 > t_call = time.monotonic() > if variant == "fork": > # A fork call can hit the SDK's HTTP read timeout; retry under a new name (cleanup removes any orphan). > for attempt in range(3): > try: > mount = drive.fork(name=f"fsync-{run}-f{seq}-{attempt}") > break > except Exception as e: # noqa: BLE001 > rec[f"fork_error_{attempt}"] = str(e)[:120] > if attempt == 2: > raise > else: > mount = drive.snapshot() > rec["fork_or_snapshot_call_s"] = round(time.monotonic() - t_call, 3) > b_sb, create_s, _ = new_sandbox(region, f"fsync-{run}-{seq}", {"/check": mount}) > rec["read_sandbox_create_s"] = round(create_s, 3) > rec.update(check(b_sb, file_id, want, "/check")) > drop(b_sb) > print("RESULT " + json.dumps(rec), flush=True) > > > def iso(t): > return time.strftime("%Y-%m-%dT%H:%M:%S", time.gmtime(t)) + f".{int(t % 1 * 1000):03d}Z" > > > def cleanup(run): > for sb in sync.query_sandboxes(query=SandboxQueryByName(name_prefix=f"fsync-{run}")): > drop(sb) > for _ in range(60): > drives = list(sync.query_drives(query=DriveQueryByName(name_prefix=f"fsync-{run}"))) > if not drives: > return > for d in drives: > try: > sync.get_drive(name_or_id=d.name).delete() > except Exception: # noqa: BLE001 > pass > time.sleep(5) > > > def main(): > ap = argparse.ArgumentParser() > ap.add_argument("--region", default="pdx1") > ap.add_argument("--trials", type=int, default=10) > ap.add_argument("--delays", default="0,0.1,1,5,30") > ap.add_argument("--variants", default="snapshot,snapshot-busy,fork,stop") > ap.add_argument("--out", default="vercel-fsync-durability.jsonl") > ap.add_argument("--child", help=argparse.SUPPRESS) > a = ap.parse_args() > if a.child: > return child(json.loads(a.child)) > > run = secrets.token_hex(3) > delays = [float(d) for d in a.delays.split(",")] > out = open(a.out, "a") > tally = Counter() > seq = 0 > try: > drive, _ = sync.get_or_create_drive(name=f"fsync-{run}", region=a.region) > writer, _, _ = new_sandbox(a.region, f"fsync-{run}-a", {MNT: drive}) > writer_name = writer.name > for variant in a.variants.split(","): > if variant == "snapshot-busy": > sh(writer, BUSY) > for delay in ([None] if variant == "stop" else delays): > for trial in range(a.trials): > seq += 1 > job = {"region": a.region, "run": run, "variant": variant, "delay": delay, "trial": trial, > "writer": writer_name, "seq": seq} > try: > p = subprocess.run([sys.executable, __file__, "--child", json.dumps(job)], capture_output=True, > text=True, timeout=(delay or 0) + REREAD_AFTER_S + (1800 if variant == "stop" else 600)) > line = next((l for l in p.stdout.splitlines() if l.startswith("RESULT ")), None) > rec = json.loads(line[7:]) if line else {"variant": variant, "delay_s": delay, "trial": trial, > "error": (p.stderr or p.stdout)[-300:]} > except subprocess.TimeoutExpired: > rec = {"variant": variant, "delay_s": delay, "trial": trial, "error": "trial timed out"} > rec = {"run": run, "t": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), **rec} > out.write(json.dumps(rec) + "\n") > out.flush() > print(json.dumps(rec), flush=True) > for k in ("small", "big"): > tally[(variant, delay, k, rec.get(k, "error"))] += 1 > if rec.get("next_writer"): > writer_name = rec["next_writer"] > writer = sync.get_sandbox(name=writer_name) > elif variant == "stop" and "error" in rec: > raise RuntimeError("stop trial failed; the Drive's writer is unknown") > if variant == "snapshot-busy": > sh(writer, "pkill -f 'of=/data/noise' || true") > finally: > cleanup(run) > print("\nvariant delay_s file result count") > for (variant, delay, k, res), n in sorted(tally.items(), key=lambda x: str(x)): > print(f"{variant:14s} {str(delay):7s} {k:5s} {res:10s} {n}") > > > if __name__ == "__main__": > sys.exit(main()) > ``` # Bring Your Own Cloud (BYOC) Source: https://docs.archil.com/reference/byoc Run the full Archil platform, fully managed by Archil, in your own cloud account or on bare metal. Bring Your Own Cloud (BYOC) runs Archil inside your own environment instead of Archil's public regions. BYOC is available to Enterprise customers. Archil fully manages each deployment, so it works like a managed cloud service that happens to run in your account. ## Supported environments BYOC runs on any cloud, including AWS, Google Cloud, and Azure, as well as on your own bare-metal servers. The only dependency is a Kubernetes cluster with outbound (egress-only) network access. Archil doesn't need inbound access to your network and doesn't depend on any cloud provider's managed services. Both the file system and storage layers run in your environment, so your disk data never leaves it. A BYOC deployment can include the full Archil platform. Choose the parts you need: * **Disks**, including [shared access](/concepts/sharing-disks) from many clients, the [S3-compatible API](/protocols/s3-api), and the [Kubernetes CSI driver](/reference/csi-driver) * **[Persistent sandboxes](/compute/sandboxes/introduction)** * **[Serverless execution](/compute/serverless-execution)** (`exec` and `grep`) BYOC deployments are on the same release pipeline as Archil's public regions and receive new versions on the same schedule. ## Why public regions differ In Archil's public regions, sandboxes and serverless execution run in AWS (see [Region availability](/reference/regions#compute-availability)). That reflects where Archil hosts its own service, not a limit of the software. In a BYOC deployment, every part of the platform runs on whichever cloud or hardware you choose. ## Deploying into your customers' environments If you sell software that runs in your customers' cloud accounts, Archil can be deployed alongside your product in each of those accounts. You set up Archil as part of installing your own software. Your customers don't need a separate agreement with Archil. Because the deployment is egress-only and Archil doesn't process the data stored in it, Archil isn't added as a subprocessor for your customers. ## Setting up a deployment Setup takes about an hour: 1. Create the access role Archil provides, such as an IAM role on AWS or its equivalent on your cloud. 2. Run the provided Helm chart or `kubectl` install in your cluster. Archil's automation does the rest: it provisions the node groups the deployment needs, installs Archil's services, and brings the deployment online. ## Operations and maintenance Archil operates each BYOC deployment the same way it operates its public regions. Archil monitors the deployment, pages its own on-call operators when something goes wrong, and performs upgrades and maintenance through its egress-only deployment tooling. Your team doesn't need to run Archil itself. If you want control over when Archil can make changes, you can require two-person approval: Archil can only perform maintenance on your cluster after someone on your team approves it. ## Telemetry To operate a deployment, Archil collects operational telemetry from it. Telemetry never includes personally identifiable information (PII) or protected health information (PHI). It is censored and scrubbed inside your cluster before it leaves: * **File data is never logged** or sent to Archil. * **File paths and file names are blocked** from leaving the deployment. BYOC requires this telemetry connection. If your environment can't allow any outbound telemetry, [contact us](mailto:sales@archil.com) to discuss options. ## Getting started To try BYOC in your own environment, start with a pilot. [Contact us](mailto:sales@archil.com) to set up a pilot or to choose a deployment model. # Changelog Source: https://docs.archil.com/reference/changelog History of updates to the Archil platform and client Every entry builds on the same storage layer: sandboxes keep their root filesystems on Archil disks, and serverless execution and persistent sandboxes run on the same Firecracker runtime. ## November 2026 * Platform [Serverless execution](/compute/serverless-execution) runs commands in the image you choose: pass `baseImage` with a public OCI image, or `imageId` with an image from the [images API](/compute/sandboxes/images), including private images. See [Choosing an image](/compute/serverless-execution#choosing-an-image). * Platform Serverless commands can ask for 1 to 32 vCPU and 256 MiB to 64 GiB of memory with `vcpuCount` and `memSizeMib`. A command larger than the default 2 vCPU and 4 GiB counts as several tool calls. See [Choosing a size](/compute/serverless-execution#choosing-a-size). * Platform Serverless commands accept a `network` egress policy with the [sandbox firewall](/compute/sandboxes/firewall) rules on paid plans. See [Restricting network access](/compute/serverless-execution#restricting-network-access). * SDK `disk` 1.12.0 and `archil` 0.21.0 for Python add the image, size, and network options to `exec`. * Platform [Prefetch File Ranges](/api-reference/disks/prefetch-disk) warms files, or byte ranges of them, into Archil's storage cache through the API, without a mounted client. See [Warming the storage system cache with the API](/details/caching#warming-the-storage-system-cache-with-the-api). * Platform [Get Sandbox Usage](/api-reference/sandboxes/get-sandbox-usage) returns a sandbox's running vCPU-hours, GiB-hours, and cost at your rates, and stays readable after the sandbox is deleted. ## October 2026 * Platform [Search files](/compute/search-files) runs on ripgrep and accepts `ignoreCase`, `multiline`, `globs`, `types`, `typesNot`, and context lines (`contextBefore` / `contextAfter`), which return matches grouped with their surrounding lines. The `disk grep` CLI gains the matching `-i`, `-U`, `-g`, `-t`, `-T`, `-A`, `-B`, and `-C` flags (`disk` 1.10.0). * Platform The [S3-compatible API](/protocols/s3-api#reading-from-a-checkpoint) reads a file as it was in a [checkpoint](/concepts/branches-and-checkpoints): pass the checkpoint name as `versionId` to `GetObject` or `HeadObject`. `GetObject` and `HeadObject` also honor `If-Match`. * Platform Sandboxes now pause after 30 minutes without an attached process connection, unless you set an [idle timeout](/compute/sandboxes/introduction#setting-a-time-to-live). Sandboxes created earlier keep their setting. Set `idleTtlSeconds` / `idle_ttl_seconds` to `0` to turn idle expiry off. ## September 2026 * Platform [Persistent sandboxes](/compute/sandboxes/introduction) are generally available. * Platform Sandbox [timeouts](/compute/sandboxes/introduction#setting-a-time-to-live) now pause the sandbox instead of shutting it down. The hard timeout defaults to, and can be set up to, 24 hours; an optional idle timeout pauses a sandbox once no process connection is open. Change either on a live sandbox with `setTimeout` / `set_timeout` or the [Set Sandbox Timeouts](/api-reference/sandboxes/set-sandbox-timeout) endpoint. * Platform Sandbox [ports](/compute/sandboxes/introduction#hosting-network-services) can be exposed publicly or reached privately with port tokens. Services started with `archil services create` are private unless the sandbox is created with `enable_service_ingress`. * Platform The [sandbox firewall](/compute/sandboxes/firewall#draining-requests-before-pause) can let in-flight requests to selected hosts finish before a sandbox pauses. * SDK The [`disk` TypeScript SDK](/sdks/typescript#persistent-sandboxes) added sandbox timeouts and ports (**v1.2.0**), port tokens (**v1.3.0**), private certificate authorities (**v1.4.0**), and request draining (**v1.5.0**). **v1.6.0** requires Node.js 22.19 or later. * SDK The [`archil` Python SDK](/sdks/python#persistent-sandboxes) added sandbox timeouts, ports, port tokens, and request draining (**v0.14.0**). * CLI Client **v0.8.40** added [`archil sync`](/reference/archil-cli#sync), [`archil update-token`](/reference/archil-cli#update-token), and adaptive [read-ahead](/details/caching) (`mount --read-ahead`). * Platform The [S3-compatible API](/protocols/s3-api#what-gets-exposed) now lists and deletes symlinks and other special files, and rejects keys longer than the data source allows with `KeyTooLongError`. ## August 2026 * Platform The [S3-compatible API](/protocols/s3-api#copying) supports `CopyObject` within a disk, up to 5 GiB per object. * Platform [Persistent sandboxes](/compute/sandboxes/introduction) entered preview, and became generally available in September 2026. They extend the Firecracker runtime behind serverless execution with long-running VMs, interactive shells, services, and pause and resume, and they fork using the same checkpoints and branches as Archil disks. * SDK Added persistent sandbox support to the [`disk` TypeScript SDK](/sdks/typescript#persistent-sandboxes) (**v0.9.0**). ## June 2026 * Platform The [S3-compatible API](/protocols/s3-api#appending) now supports **appends** — `AppendObject` (`PUT //?append=true`) adds bytes to the end of an existing object (or creates it), up to 1 MiB per call, with the same [read-after-write consistency](/details/consistency) as any other write. This makes log-style and incrementally-grown files possible over the object API without rewriting the whole object or mounting. Exposed in the SDKs as `append_object` ([Python](/sdks/python#reading-and-writing-objects)) / `appendObject` ([TypeScript](/sdks/typescript#reading-and-writing-objects)). * SDK The [Python](/sdks/python#reading-and-writing-objects) and [TypeScript](/sdks/typescript#reading-and-writing-objects) SDKs gained large-object and bulk write operations on the S3-compatible object API. `put_object` / `putObject` now switch to a multipart upload automatically for large bodies — split into parts, uploaded in parallel, and assembled — with tunable `part_size`/`partSize`, `concurrency`, and `multipart_threshold`/`multipartThreshold`. `delete_objects` / `deleteObjects` removes up to 1,000 keys per request (returning per-key results), and the raw multipart lifecycle is available under the opt-in `disk.multipart` namespace. Transient failures (429/5xx/network) are now retried automatically with jittered exponential backoff. * Platform The [S3-compatible API](/protocols/s3-api) now supports **writes** — `PutObject` to create or overwrite a file and `DeleteObject` to remove one, both with the same [read-after-write consistency](/details/consistency) as any other Archil client. Point any S3 client at your disk to write objects without mounting. `CopyObject` followed in [August 2026](#august-2026). Writes are also exposed in the SDKs as `put_object` / `delete_object` ([Python](/sdks/python#reading-and-writing-objects)) and `putObject` / `deleteObject` ([TypeScript](/sdks/typescript#reading-and-writing-objects)). * SDK Launch of the [`archil`](https://pypi.org/project/archil/) Python package — a pure-Python control-plane SDK for creating disks, listing and inspecting them, managing API keys, running serverless commands with `Disk.exec`, and reading and writing disk contents through an S3-compatible object API. Every method works both synchronously and asynchronously via an `.aio` variant. No native dependencies; requires Python 3.10+. See the new [Python SDK reference](/sdks/python). * Platform Launch of [parallel search](/compute/search-files) — `disk.grep` searches every file under a directory for a regular expression, fanning the listing and matching out across many ephemeral microVMs so the search scales across many machines instead of being capped by one. You get back structured matches (file, line, text) plus a `stoppedReason` that says whether the search ran to completion or stopped early on your time or result limit. Available in the [`disk` SDK](/sdks/typescript#searching-files) (**v0.8.12**) and the [control plane API](/api-reference/serverless-execution/grep-disk). * Platform [Branches & checkpoints](/concepts/branches-and-checkpoints) are now generally available in all AWS regions. Capture named, immutable point-in-time snapshots of a disk's filesystem state (checkpoints) and fork independent, writable copies from any checkpoint (branches) without disturbing the original — much like Git commits and branches. Manage them from a mounted disk with the `archil checkpoints` and `archil branches` commands. Available on disks that are not synchronized to a [data source](/concepts/data-sources). ## May 2026 * CLI Added [`archil prefetch`](/reference/archil-cli#prefetch) (client **v0.8.11**) to warm the client cache by reading files and directories into it ahead of time, so later reads are served locally instead of fetching from the server. Paths can also be pre-fetched automatically at mount time with [`mount --pre-fetch`](/reference/archil-cli#mount). ## April 2026 * Platform Launch of the [read-only S3-compatible API](/protocols/s3-api) — point any S3 client (`aws s3 cp`, `boto3`, `s5cmd`) at your Archil disk and read its contents over HTTPS without mounting. Generate per-disk SigV4 credentials in the [Archil console](https://console.archil.com), use the disk ID as the bucket name, and call `HeadBucket`, `ListObjectsV2`, `GetObject` (with HTTP `Range`), or `HeadObject`. The same path also powers signed share URLs for one-off, time-limited file links from the console. * Platform [Serverless execution](/compute/serverless-execution) brought the Firecracker container runtime to the API. `disk.exec(command)` spins up a Firecracker microVM with your file system already mounted, runs an arbitrary shell command to completion, and returns stdout, stderr, exit code, and timing. For current rates, see [Pricing & Billing](/administration/billing). Use it to give an agent a bash tool, fan work out across one disk, or get a result without pulling raw data through your client. See the new [Compute](/compute/serverless-execution) section. * SDK Launch of the [`disk`](https://www.npmjs.com/package/disk) npm package — a pure-JavaScript control-plane SDK and `npx disk` CLI for creating disks, listing and inspecting them, managing API keys, and running serverless commands with `disk exec`. No native dependencies; runs anywhere Node does. See the new [TypeScript SDK reference](/sdks/typescript) and [`disk` CLI reference](/reference/disk-cli). * macOS Launch of [Archil for macOS](/mounting/macos) — a native menu bar app that mounts your Archil disks as macOS volumes via Apple's FSKit framework. Sign in with email, browse personal and organization disks across regions, and mount them straight into Finder. Requires macOS 26 (Tahoe) or later on Apple Silicon. Install with `curl -s https://archil.com/install | sh`. ## February 2026 * SDK Added [disk management support](/sdks/typescript#managing-disks) to the TypeScript SDK — create, list, and delete disks and API tokens programmatically via `@archildata/client` * CLI Added [subdirectory mounting](/reference/archil-cli#subdirectory-mounts) via `disk:/path` syntax — mount any subdirectory of a disk as the root of your local mount. Supports independent delegations, symlink resolution, and mount boundary isolation. * Terraform Launch of the [Archil Terraform Provider](https://registry.terraform.io/providers/archil-data/archil/latest) for managing disks, users, and API tokens with Infrastructure as Code. See the [documentation](/reference/terraform) for details. * Platform Launch of the [Archil Control Plane API](/api-reference/introduction) for programmatic management of disks, mounts, and authentication tokens ## January 2026 * SDK Launch of [`@archildata/just-bash`](https://www.npmjs.com/package/@archildata/just-bash) — just-bash bindings for the Archil filesystem * SDK Launch of the Archil Node.js client [`@archildata/client`](https://www.npmjs.com/package/@archildata/client) ## December 2025 * Platform Launch of private preview regions for early access to new cloud regions * Platform Launch of managed buckets support for Google Cloud Storage * Performance Improved the speed of sequential read operations * CLI Launched client **v0.7.0** including new caching architecture, auth tokens via the `ARCHIL_AUTH_TOKEN` environment variable, and the preview `--writeback-cache` flag for workloads that perform frequent small writes to a small number of files. ## November 2025 * Platform Launch of support for Azure Blob Storage as a data source * Platform Launch of managed bucket support for Amazon S3, enabling users to use Archil without a pre-existing S3 bucket * Platform Active data tracking provides visibility into how much data is actively cached * Platform The Firecracker container runtime, which now also runs serverless execution and persistent sandboxes, added TTY support, environment variables, and file injection in beta ## October 2025 * Performance Launch of scale-out read performance to achieve 8 GiB/s of throughput per client * Performance Improved Archil's ability to handle network congestion under load * CLI Launched client **v0.6.4**: Implemented "silly rename" to support unlinking opened file descriptors, added `--sts-region` flag for IAM authentication outside of AWS, improved authentication error messages * Platform Launch of Archil shells to use bash against Archil disks from the web console ## September 2025 * CLI Launched client **v0.6.1**: Improved a rare issue related to dirty listing * CLI Launched client **v0.6.0**: Added additional tools to debug memory usage and performance * CLI Launched client **v0.5.0**: Reduced memory copies for faster writes, added `--read-only` mount flag, improved error messages * CLI Launched client **v0.4.0**: Disabled extended attributes by default to improve write performance (use `--enable-xattrs`), added `--nconnect` to work around hyperscale single-stream throughput limitations ## August 2025 * Region Launch Archil in Google Cloud us-central1 region * Platform Support the usage of cross-region data sources for Archil disks * CLI Launched client **v0.3.0**: Enables the ability to mount an Archil disks by name, and the ability to mount an Archil disk in a different cloud region ## July 2025 * Platform Launch of the Archil file explorer to browse and manage Archil disks * Kubernetes CSI driver improvements with better mount lifecycle management and ReadWriteMany (RWX) support * CLI Launched client **v0.2.0** in public preview, installable with `curl -s https://archil.com/install | sh`. The Archil client mounts disks over Archil's own file protocol instead of NFS, with faster metadata and higher per-client throughput ## February 2025 * CLI Launched client **v0.1** in private preview, mounting Archil disks over Archil's own file protocol. Packages are available for Debian- and RHEL-based Linux distributions. ## November 2024 * Platform Launched publicly on Hacker News * Platform Added Google Cloud Storage, Cloudflare R2, Backblaze B2, and other S3-compatible services as data sources, connected with an endpoint and access keys * Kubernetes CSI driver for Kubernetes available in private beta * Platform Published the documentation site, with a quickstart and pages on architecture, performance, pricing, and compatibility ## October 2024 * Platform Launch of Archil disks over NFS * Platform Mounts authenticate with the instance's AWS IAM role, and traffic between the client and Archil is encrypted with TLS * Platform The console shows throughput, IOPS, and cache usage graphs for each disk # Client Release Notes Source: https://docs.archil.com/reference/client-release-notes Release history for the Archil client ## v0.10.5 *Released 2026-11-12* ### Fixed * On Linux, opening a file another client just resized could still use the old size and overwrite appended data. Opens now fail with "Stale file handle" while the size keeps changing. ## v0.10.4 *Released 2026-11-11* ### Fixed * `archil mount` no longer hangs before the mount becomes usable on Linux hosts with audit rules loaded, which Daytona sandboxes began using on November 11, 2026. ## v0.10.3 *Released 2026-11-10* ### Fixed * Reads that bypass the page cache, such as with `O_DIRECT`, now stop at the end of the file instead of returning zeros, so programs reading to the end no longer hang. * Reopening a file you wrote no longer returns old data when another client changed it while you had it open. * On Linux, opening a file that another client just resized no longer sees the old size, which could cut reads short or crash writes through a memory mapping with SIGBUS. ## v0.10.2 *Released 2026-11-09* ### Changed * On `--shared` and `--conditional` mounts and in the macOS app, changes from other clients now appear almost immediately instead of after cached data expires. ### Fixed * Disks now mount on IPv6-only hosts that reach IPv4 through NAT64, such as some sandbox providers. Previously file access hung after a successful mount. * Mounts using a one-use disk token no longer fail to reconnect more than about five minutes after mounting. * When clients share a disk, appending to a file another client just appended to no longer overwrites that client's data. * Opening or creating files in a directory another client moved no longer intermittently fails with "Stale file handle". * Writing to a disk that other clients just unmounted fails less often with "Resource temporarily unavailable". * `archil invalidate-cache` now reliably clears stale data, and neither it nor checkouts break `getcwd()` for processes working inside the mount on Linux 6.2 and newer. * `stat` and `git status` no longer slow down after writing many files on an exclusive mount or in a checked-out directory. * Renaming a file to its own name through the Node SDK no longer makes the disk read-only. ### Performance * Lower Linux client CPU usage for directory listings and for opening and closing files, about 30% less on repeated searches over a large source tree. * Better caching: data read again after the cache expires is cached again, and files open in many processes keep their kernel-cached pages. * Lower tail latency when a connection is saturated. * `archil mount` no longer waits on an extra DNS lookup that could add 5 seconds. ## v0.10.1 *Released 2026-10-28* ### Changed * The Linux `.deb` and `.rpm` packages now install the system CA certificates the client needs to verify Archil servers. ### Fixed * The Linux install script now supports `microdnf`-only minimal images (such as Amazon Linux 2023 minimal and UBI minimal) and openSUSE, and installs the `mount` tool and FUSE library when a container image lacks them. * The install script no longer hangs at "Installing package..." during unattended installs, and Ctrl-C now stops it. Set `ARCHIL_AUTO_UPGRADE=true` or `false` to skip its upgrade prompt. * Fixed a rare race where concurrent opens of the same file in separate processes on the same mount could return stale data. * When multiple clients share a disk, moving a directory that another client recently moved no longer fails with "Stale file handle". * Deleting, renaming or creating entries under the reserved `/.archil` directory now fails immediately with "Operation not permitted" instead of reaching the server, where it could turn the mount read-only, so mirror tools like `rclone sync` skip it and carry on. * Files created inside a setgid directory now inherit the directory's group instead of the creating process's group. * Setting an extended attribute with a create-only or replace-only flag no longer makes later extended attribute writes fail for the rest of the mount session. ### Performance * `archil checkout` clears stale cached data from the kernel with less overhead. * Slightly lower CPU overhead for every filesystem request. ## v0.10.0 *Released 2026-10-22* ### Changed * The CLI, macOS app, and Node SDK now verify Archil server certificates against the operating system's trust store, so Linux systems and containers must have CA certificates installed (for example, the `ca-certificates` package) or set `SSL_CERT_FILE`. * `archil checkpoints create` refuses the name `null`, which S3 tools use to mean the current version of a file. ### Fixed * The CLI no longer trusts HTTPS certificates from the eMudhra intermediate certificate authorities whose private keys were disclosed in October 2026. * On disks shared between clients, a file's size and permissions are now up to date immediately after opening it or changing it with chmod or truncate, instead of briefly showing older values. * Linux mounts now support open-file-description locks and report file locks in `/proc/locks` by using kernel-local locking. ### Performance * Prefetching many paths now resolves shared parent directories once per batch instead of once per path, reducing client CPU usage for large prefetch requests over deep directory trees. * Improved performance of the client's caches when many operations run at once. ## v0.9.3 *Released 2026-10-16* ### Fixed * Fixed a bug that could cause a client to report a stale file size when a disk is mounted by multiple clients. * Prevents a rare condition in S3 native-format disks where a file create, hard link, or rename whose full path exceeds S3's key length limit could be accepted locally and then fail to flush, leaving the disk read-only. ### Performance * Sequential writes to large files are faster. * Creating, hard-linking, and renaming files deep in a directory tree is faster in cases where cached attributes have expired. * The Linux client binary and its .deb/.rpm packages are about 80% smaller (roughly 52 MB instead of 267 MB). ## v0.9.2 *Released 2026-10-14* ### Fixed * `archil checkout` of a file or directory created less than about a second earlier no longer fails with "does not exist". * On conditional mounts with a checked-out directory, deleting, renaming or hard-linking a file no longer fails with "Invalid cross-device link" after its other hard links outside the delegation have been removed. * Closed a race in multi-writer scenarios that sometimes caused ENOENT failures when accessing a file immediately after creating it. * Switching a mount to a read-only token with `archil update-token` now makes it read-only instead of failing it. ### macOS * Disks mounted with a read-only token now mount read-only in the macOS app, so writes fail immediately instead of being accepted and rejected later. ## v0.9.1 *Released 2026-10-09* ### Fixed * Fixed the default data cache being sized from the host's total RAM instead of the container's memory limit in privileged containers and on cgroup v1 nodes (such as some GKE node pools). * Fixed renaming or deleting a file in a checked-out directory that also has a hard link outside it turning the whole mount read-only; the operation now fails with an error instead. * Fixed a stall when listing a directory while `archil invalidate-cache` ran on a file in it on `--conditional` mounts. * Fixed `archil set-cache-expiry --readdir-expiry` not applying to names learned from a directory listing, so files another client deleted or renamed now disappear after that interval instead of after five minutes. * Fixed a rare crash or stale file view when a rename raced with other creates, renames, or deletes of the same paths on the same mount. ### Performance * Preserved data in the kernel page cache when re-opening a file that has not changed since it was last opened, instead of reading it from the server again. * Sped up writing large directory trees, such as `npm install`, by no longer asking the server about the contents of newly created directories. ## v0.9.0 *Released 2026-10-06* ### Performance * Improved write throughput in conditional mode by up to 256x when performing small writes. ### Fixed * Fixed kernel tuning reliability for CSI driver mounts on older kernels. * Fixed a rare bug in which unmounting or closing a file emits an error. * Fixed bugs related to stale timestamps and link counts for recently changed file attributes. * Improved consistency of mount time. * Various fixes around setuid/setgid during unprivileged fallocate or truncate when mounted in conditional mode. * Fixed Linux musl (Alpine) client builds. * Fixed a bug where the file data cache was underutilized, improving read performance for large files. # Kubernetes CSI driver Source: https://docs.archil.com/reference/csi-driver Documentation on mounting using the [Kubernetes Container Storage Interface](https://kubernetes-csi.github.io/docs/) (CSI) driver for Archil disks. The Archil CSI Driver provides seamless integration between Kubernetes and Archil's distributed disk technology. It implements the CSI specification to allow Kubernetes workloads to mount and use Archil disks with both ReadWriteOnce (RWO) and ReadWriteMany (RWX) access modes. **Requirements:** * **Helm chart distribution:** The driver is provided as an OCI Helm chart and requires Helm v3.8.0 or newer. * **DNS access:** Pods must have access to external DNS to mount Archil disks. (Note: some providers, such as Fly.io, don’t enable this by default.) ## Dynamic provisioning Dynamic provisioning automatically creates and deletes Archil disks when PersistentVolumeClaims are created and deleted. This is the recommended approach for most workloads. ### 1. Install CSI driver with controller Create a Kubernetes secret with your [API key](/api-reference/introduction): ```bash theme={null} kubectl create secret generic archil-controlplane-api-key \ --from-literal=api-key=YOUR_API_KEY \ --namespace archil-system ``` Install the CSI driver with the controller and StorageClass enabled: > [tab] AWS: IAM Role Authorization > > ```bash theme={null} > helm install archil-csi-driver \ > oci://registry-1.docker.io/archildata/csi-driver-chart \ > --namespace archil-system \ > --create-namespace \ > --set controller.enabled=true \ > --set storageClass.enabled=true \ > --set storageClass.region=aws-us-east-1 \ > --set storageClass.nodeAuthType=awssts \ > --set storageClass.iamRoleArn=arn:aws:iam::123456789012:role/ArchilNodeRole \ > --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::123456789012:role/ArchilNodeRole > ``` > [tab] Disk Token Authorization > > ```bash theme={null} > helm install archil-csi-driver \ > oci://registry-1.docker.io/archildata/csi-driver-chart \ > --namespace archil-system \ > --create-namespace \ > --set controller.enabled=true \ > --set storageClass.enabled=true \ > --set storageClass.region=aws-us-east-1 \ > --set storageClass.nodeAuthType=token > ``` > > With `nodeAuthType=token`, the controller automatically generates a unique [disk token](/concepts/authorization#token-based-disk-authorization) per provisioned disk and stores it in a Kubernetes Secret. No manual token management is required. ### 2. Create a PersistentVolumeClaim ```yaml theme={null} apiVersion: v1 kind: PersistentVolumeClaim metadata: name: my-archil-pvc spec: accessModes: - ReadWriteOnce # Or ReadWriteMany for multi-node access storageClassName: archil resources: requests: storage: 1Ti # Archil disks are elastic; this value is informational ``` ### 3. Use the disk in a pod ```yaml theme={null} apiVersion: v1 kind: Pod metadata: name: archil-test spec: containers: - name: app image: busybox command: ['sh', '-c', 'echo "Hello from Archil" > /data/hello.txt && cat /data/hello.txt'] volumeMounts: - name: archil-storage mountPath: /data volumes: - name: archil-storage persistentVolumeClaim: claimName: my-archil-pvc restartPolicy: Never ``` When the PVC is created, the controller automatically provisions an Archil disk via the controlplane API. When the PVC is deleted (with `reclaimPolicy: Delete`), the disk is cleaned up automatically. *** ## Static provisioning If you need to use a pre-existing Archil disk, you can create PersistentVolumes manually. ### 1. Install CSI driver Install the CSI driver using our official Helm chart from [Docker Hub](https://hub.docker.com/r/archildata/csi-driver-chart). The chart deploys the [archildata/csi-driver](https://hub.docker.com/r/archildata/csi-driver) container image. ```bash theme={null} helm install archil-csi-driver \ oci://registry-1.docker.io/archildata/csi-driver-chart \ --namespace archil-system \ --create-namespace ``` ### 2. Configure disk users > [tab] AWS: IAM Role Authorization > > Archil natively supports using AWS IAM users or roles to authorize access to the disk. Simply add the ARN of the IAM user or role that will be used to connect to the disk. > > * Get your EC2 server's IAM role arn using the following command: > > ```bash theme={null} > archil utils get-iam-role > ``` > > * Copy the resulting IAM role arn. > * Update your helm release: > > ```bash theme={null} > helm upgrade archil-csi-driver \ > oci://registry-1.docker.io/archildata/csi-driver-chart \ > --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::123456789012:role/ArchilCSIRole > ``` > > * Restart the CSI driver daemonset: > > ```bash theme={null} > kubectl rollout restart daemonset/archil-csi-node -n archil-system > ``` > > > [!NOTE] > > > > This IAM role is used for authentication from the CSI driver to *all* disks. The Archil CSI Driver does not currently support specifying IAM roles on a per-disk basis. > [tab] Disk Token Authorization > > Outside of AWS, Archil can issue a [disk token](/concepts/authorization#token-based-disk-authorization) — a per-disk credential used to mount the disk. > > * Navigate to the disk's **Details** page in the Archil console > * Click **Generate Token** to create a new disk token > * Copy the disk token > * Create a Kubernetes secret containing the disk token: > > ```bash theme={null} > # Replace dsk-XXXXXXXXXXXX with your disk ID and YOUR_DISK_TOKEN with the token you just copied. > kubectl create secret generic archil-token-dsk-XXXXXXXXXXXX \ > --from-literal=token=YOUR_DISK_TOKEN \ > --namespace=archil-system > ``` > > > [!NOTE] > > > > This disk token is **not** the same as the control-plane API key used above for dynamic provisioning. The API key manages disks; the disk token grants mount access to one specific disk. ### 3. Create PersistentVolume ```yaml theme={null} apiVersion: v1 kind: PersistentVolume metadata: name: archil-pv spec: capacity: # Can be any size, Archil disks are infinite storage: 1000Gi accessModes: # Or use ReadWriteOnce for single-node access - ReadWriteMany persistentVolumeReclaimPolicy: Retain csi: driver: csi.archil.com # Your disk ID from Archil console volumeHandle: dsk-XXXXXXXXXXXX volumeAttributes: # Region where your disk exists region: aws-us-east-1 # Only include this section if using disk token authentication: # nodePublishSecretRef: # name: archil-token-dsk-XXXXXXXXXXXX # namespace: archil-system ``` ### 4. Create PersistentVolumeClaim ```yaml theme={null} apiVersion: v1 kind: PersistentVolumeClaim metadata: name: archil-pvc namespace: default spec: # Must match PV access mode accessModes: - ReadWriteMany resources: requests: storage: 1000Gi # Must match PV name volumeName: archil-pv ``` ### 5. Create a pod This example creates a simple demo pod to verify connectivity to your Archil disk. You can deploy your own applications using any pod configuration, just mount the PVC as shown below. ```yaml theme={null} apiVersion: v1 kind: Pod metadata: name: archil-test spec: containers: - name: test image: busybox command: ['sh', '-c', 'ls -la /data && echo "---" && echo "Archil disk contents listed above"'] volumeMounts: - name: archil-storage mountPath: /data volumes: - name: archil-storage persistentVolumeClaim: # Must match PVC name claimName: archil-pvc restartPolicy: Never ``` ## Configuration reference ### Storage access modes The `accessMode` that you use when creating your PersistentVolume and PersistentVolumeClaim will determine whether the disk is mounted with support for other clients to access it. When using `ReadWriteOnce`, the disk is mounted into single ownership mode, preventing other clients from accessing the disk. When using `ReadWriteMany`, the disk is mounted in conditional mode (`--conditional`), allowing multiple clients to access the disk simultaneously. Each client can write to any path that another client hasn't checked out, without calling `archil checkout`. If two clients write to the same part of a file, the last write wins. For more details, see [Conditional mode](/concepts/sharing-disks#conditional-mode). ### Helm chart values The following table describes all available configuration options in `values.yaml`: | Parameter | Description | Default | Required | | - | - | - | - | | `image.tag` | Container image tag | `0.1.0` | No | | `image.pullPolicy` | Image pull policy | `IfNotPresent` | No | | `serviceAccount.annotations` | Service account annotations for AWS IRSA | `{}` | No | | `extraEnv` | Additional environment variables for the CSI driver container | `[]` | No | | `resources` | Resource limits and requests for the CSI driver container | `{}` | No | | `nodeSelector` | Node selector for CSI driver pod placement | `{}` | No | | `tolerations` | Pod tolerations | `[]` | No | | `affinity` | Pod affinity rules | `{}` | No | | `controller.enabled` | Enable the controller plugin for dynamic provisioning | `false` | No | | `controller.apiKeySecretName` | Name of the K8s Secret containing the API key | `archil-controlplane-api-key` | No | | `controller.controlplaneURL` | Override controlplane URL (empty = auto-detect from region) | `""` | No | | `controller.serviceAccount.annotations` | Annotations for the controller service account | `{}` | No | | `storageClass.enabled` | Create a StorageClass for dynamic provisioning | `false` | No | | `storageClass.name` | StorageClass name | `archil` | No | | `storageClass.isDefault` | Set as default StorageClass | `false` | No | | `storageClass.reclaimPolicy` | PV reclaim policy (`Delete` or `Retain`) | `Delete` | No | | `storageClass.volumeBindingMode` | When to bind volumes (`Immediate` or `WaitForFirstConsumer`) | `Immediate` | No | | `storageClass.region` | Archil region for provisioned disks | `""` | Yes (if enabled) | | `storageClass.nodeAuthType` | Node authentication type (`awssts` or `token`) | `awssts` | No | | `storageClass.iamRoleArn` | IAM role ARN for node authentication | `""` | Yes (if awssts) | ## Changelog **0.2.0** 2026-03-02 * Dynamic provisioning via controller plugin * Automatic disk creation/deletion when PVCs are created/deleted * Support for both IAM (AWS STS) and disk-token-based node authentication * New StorageClass with configurable region and auth type **0.1.0** 2025-07-28 * Initial preview release # Data ownership and portability Source: https://docs.archil.com/reference/data-ownership Where your data lives, how to get it out, and what happens to it if you stop using Archil. Disks synced to your own bucket store files as ordinary S3 objects that any S3 tool can read without Archil. Picking a storage and sandbox layer means planning for the day you leave it, or the day the vendor changes. This page says exactly what you own, what you depend on Archil for, and how to get your data out. ## Adopt Archil without a one-way door You can run your whole agent stack on Archil, disks and sandboxes, and still be able to replace either half on its own: 1. **Keep workspaces on a disk synced to your own bucket.** Each file is an ordinary object in your S3, GCS, or R2 bucket, readable with any S3 tool without Archil. 2. **Build sandbox images as standard OCI images** in your own registry. The same image runs on other container and microVM platforms. 3. **To move sandboxes elsewhere,** mount the same disk in [E2B](/guides/platforms/e2b), [Daytona](/guides/platforms/daytona), [Modal](/guides/platforms/modal), [Vercel Sandbox](/guides/platforms/vercel-sandbox), or any Linux machine. Your agents keep their files, and nothing needs to be migrated. 4. **To move storage elsewhere,** your bucket already holds the files. Point your new storage layer at it. With this setup, the cost of leaving Archil is changing where your code runs or where you mount the bucket, not migrating data. ## Summary | | Durable copy lives in | Readable without Archil | How to leave | | - | - | - | - | | Disk synced to your bucket ([BYOB](/concepts/data-sources)) | Your S3, GCS, R2, or other S3-compatible bucket | Yes. Each file is an ordinary object at its path. | Stop writing and delete the disk. The objects stay in your bucket. | | Archil-managed disk (the default) | A bucket Archil operates, in Archil's internal format | No | Copy it out over the [S3 API](/protocols/s3-api) or a mount. | | [Sandbox](/compute/sandboxes/introduction) root filesystem | An Archil-managed disk | No | Keep data you need on a mounted disk. Images are standard OCI images. | | [BYOC](/reference/byoc) deployment | Your own cloud account or hardware | Same as above, inside your environment | Same as above. Disk data never leaves your environment. | If you want a copy of your data that never depends on Archil, use a disk synced to your own bucket. Bring your own bucket is included on every paid plan, starting with Developer. ## Disks synced to your own bucket With [bring your own bucket](/concepts/data-sources), Archil writes file contents to your bucket as ordinary objects, at the key that matches the file's path on the disk. You can read them with `aws s3`, boto3, Athena, Spark, or any other S3 tool, without the Archil client and without an Archil account. If you stop using Archil, those objects stay in your bucket, under your control. What to know before relying on the bucket copy: * **Sync is asynchronous.** A write committed to Archil is stored durably on Archil's storage layer, replicated across Availability Zones, and reaches your bucket in the background. How long that takes depends on the workload. Until then, the newest writes exist only in Archil. See [Consistency](/concepts/data-sources#consistency). * **POSIX metadata stays in Archil.** Mode bits, ownership, and extended attributes aren't written to the bucket. Symbolic links are stored as a text file containing the link target, and hard links as full copies. * **Multi-step operations aren't atomic in the bucket.** A directory rename can show partial results in the bucket while it runs. See [Atomicity](/concepts/data-sources#atomicity). * **Deleting the disk doesn't delete your objects.** Archil stops synchronizing and discards its cached copy. Objects already written to your bucket remain. * **You control access.** Archil reaches your bucket through the bucket policy or credentials you grant. Remove them and Archil can no longer read or write the bucket. Before deleting a disk you're moving away from, stop writing to it and give the background sync time to finish. You can compare a listing of the disk over the Archil S3 API with a listing of your bucket to confirm they match. ## Archil-managed disks By default, disks use Archil-managed storage: a bucket Archil operates, holding data in Archil's internal format with multi-AZ durability. You can't point your own tools at that bucket, so you read the data through Archil. Two ways to export it: * **[S3 API](/protocols/s3-api).** Every disk is also an S3-compatible bucket. Add S3 credentials to the disk in the console and copy everything out with any S3 client: ```bash theme={null} aws s3 sync s3://dsk-abc123/ ./dsk-abc123 \ --endpoint-url https://s3.green.us-east-1.aws.prod.archil.com ``` Symbolic links, FIFOs, sockets, and device nodes aren't readable over the S3 API. * **A mount.** [Mount the disk](/mounting/linux) and copy it with `rsync -aH` to keep permissions, ownership, symbolic links, and hard links. To move a managed disk to a bucket you own, create a disk synced to your bucket and copy the data into it. Data transfer out of Archil is billed at the standard [rate](/administration/billing). Deleting a managed disk deletes its data, so export anything you want to keep first. ## Sandboxes [Persistent sandboxes](/compute/sandboxes/introduction) boot from standard OCI images, public or private, so the same image runs on other container and microVM platforms. A sandbox's root filesystem lives on an Archil-managed disk. Write data you want to keep, or to take elsewhere, to a mounted Archil disk, ideally one synced to your own bucket. ## Bring your own cloud [BYOC](/reference/byoc), on Enterprise, runs both the file system and storage layers in your own cloud account or on bare metal, so disk data never leaves your environment. Combined with your own buckets, the durable copy of your data sits entirely in infrastructure you control. Archil operates each BYOC deployment through egress-only access, and you can require two-person approval before Archil makes changes. ## Commitments * **Durability and availability.** Archil is designed for 99.999% durability and 99.99% availability. Per-component uptime history is public at [status.archil.com](https://status.archil.com). * **Security.** Archil undergoes ongoing SOC 2 Type II audits. See [Security & Compliance](/details/security). * **Contracts.** Enterprise customers can sign an availability SLA, a DPA, and a BAA. Questions about data ownership or exit planning: [support@archil.com](mailto:support@archil.com). # `disk` CLI Source: https://docs.archil.com/reference/disk-cli Reference for the Node CLI that creates disks, runs `disk exec`, and manages API keys. The [`disk`](https://www.npmjs.com/package/disk) npm package ships a CLI under the same name. Run it with `npx disk` from any machine that has Node, with no install required and no platform restrictions. Use it to create disks, list and inspect them, manage who can mount them, and run [serverless commands](/compute/serverless-execution) against the data, all from a script, CI job, or your terminal. For a step-by-step walkthrough, see the [Quickstart](/getting-started/quickstart). ```bash theme={null} npx disk [options] ``` `disk` talks to the Archil control plane over HTTPS. It has no native dependencies and works on any platform Node.js does. This page is a reference for every command it accepts; for a walkthrough, see the [Quickstart](/getting-started/quickstart). ## Authentication `disk` reads credentials from environment variables or flags: | | Env var | Flag | | - | - | - | | API key | `ARCHIL_API_KEY` | `-k`, `--api-key` | | Region | `ARCHIL_REGION` | `-r`, `--region` | API keys are account-level credentials. They are **not** the same as disk tokens — see the [disk users concept page](/concepts/authorization#token-based-disk-authorization). ## Disks ### `disk list` List the disks in the current region. ```bash theme={null} npx disk list [--limit ] [-o table|json] ``` Outputs a box-drawing table by default; `-o json` emits an array suitable for piping into `jq`. ### `disk get ` Show details for a disk: status, region, mounts, authorized users, connected clients. ```bash theme={null} npx disk get dsk-abc123 [-o table|json] ``` ### `disk create ` Create a new disk. The response includes a one-time **disk token** for mounting — save it; it cannot be retrieved again. ```bash theme={null} npx disk create my-disk ``` ### `disk delete ` Delete a disk. This does not delete the underlying bucket data. ```bash theme={null} npx disk delete dsk-abc123 ``` ### `disk exec ` Run a bash command inside a Firecracker microVM with the disk mounted at `/mnt/archil`, which is also the working directory — so relative paths address the disk root. Stdout, stderr, and the exit code from the remote command are forwarded back. See [Serverless Execution](/compute/serverless-execution) for the full model. ```bash theme={null} npx disk exec dsk-abc123 "ls -la && cat config.json" ``` `disk` also accepts the sugar form `disk exec ` for ergonomics: ```bash theme={null} npx disk dsk-abc123 exec "du -sh logs" ``` The CLI process exits with the remote command's exit code, so it composes naturally with shell pipelines and CI step outcomes. ### `disk grep [/]` Search a disk for lines matching a regular expression, using [Search files](/compute/search-files). The part of the target after the first `/` is the directory to search; without it, the whole disk is searched. The target accepts a disk ID or name. The search is recursive unless you pass `--no-recursive`. ```bash theme={null} npx disk grep "ERROR|FATAL" dsk-abc123/logs npx disk grep -i -t py "import pandas" my-disk/src npx disk grep panic dsk-abc123/logs -C 3 -g '*.log' -g '!archive/**' ``` | Option | Description | | - | - | | `-i`, `--ignore-case` | Match regardless of case. | | `-U`, `--multiline` | Let matches span lines. | | `-g`, `--glob ` | Search files matching the glob, or skip them with a leading `!`. Repeatable. | | `-t`, `--type ` | Search only files of this ripgrep type, such as `py`. Repeatable. | | `-T`, `--type-not ` | Skip files of this type. Repeatable. | | `-A`, `--after-context ` | Lines of context after each match, up to 10. | | `-B`, `--before-context ` | Lines of context before each match, up to 10. | | `-C`, `--context ` | Lines of context before and after each match, up to 10. | | `--no-recursive` | Search only the given directory, not its subdirectories. | | `--max-results ` | Stop once this many matches are collected. | | `--max-duration ` | Deadline for the whole search, up to 30. | | `--concurrency ` | Maximum parallel workers, up to 100. | | `-o`, `--output ` | `text` (the default) or `json`. | Text output prints matches as `file:line:text` on stdout, and context lines as `file-line-text` with `--` between groups, like `rg`. A summary goes to stderr, with a warning when the search did not complete. The command exits `0` when it finds matches and `1` when it finds none. ## API keys API keys live at the account level. Manage them with the `api-keys` subcommand. ### `disk api-keys list` ```bash theme={null} npx disk api-keys list [--limit ] ``` ### `disk api-keys create ` ```bash theme={null} npx disk api-keys create ci-bot --description "GitHub Actions" ``` The key value is shown once at create time. Save it. ### `disk api-keys delete ` ```bash theme={null} npx disk api-keys delete key-abc123 ``` ## Global options ```bash theme={null} npx disk --help npx disk --version # Override credentials per-invocation rather than using env vars npx disk --api-key key-... --region aws-us-east-1 list # Point at a non-production control plane npx disk --base-url https://staging.archil.com list ``` # Limits Source: https://docs.archil.com/reference/limits Current hard limits for Archil disks, files, sandboxes, and the S3 API, and the highest load observed in production. These are Archil's current limits. Most can be raised. If a limit or the performance below is in your way, tell us at [support@archil.com](mailto:support@archil.com) and we'll prioritize it. Every latency row can be rerun on your own account with the script in [Latency benchmark](/reference/benchmarks/latency). See [Verify Archil yourself](/details/verify) for other checks. The Seen column isn't a cap. It's the highest value Archil production has recorded from July 2025 to October 2026. Rates are averaged over one minute. ## Disks and files | Limit | Current | Seen | | - | - | - | | Max file size | 1 PiB | | | Max files in one directory | Unlimited | 80K+ | | Max connected clients per disk | 5,000 | | | Max connected clients, all disks | Unlimited | 32K+ | | Max data sources per disk | 1 | | | Max disk name length | 100 characters | | | Max branch name length | 100 characters | | Disk and branch names can use letters, digits, `-`, and `_`. ### Disks per plan | Plan | Max disks per region | Seen | | - | - | - | | Free | 5 | | | Developer | 1,000 | | | Team | 25,000 | | | Enterprise | Unlimited | 100K+ | ### Disks synchronized with a data source A disk synchronized with a [data source](/concepts/data-sources) keeps each file as a normal object in your bucket, so these limits apply instead. | Limit | Current | | - | - | | Max file size | 5 TiB | | Max path or S3 key length | 1,024 bytes | | Max entries in one directory listed over S3 | 100,000 | The path length includes the bucket prefix, if the disk uses one. ## Throughput and requests | Limit | Current | Seen | | - | - | - | | Max write throughput per disk | 1 GB/s | 620 MB/s | | Max read throughput per disk | Unlimited | 5.9 GB/s | | Max control-plane API request body | 512 KiB | | ## Latency | Operation | p50 | p95 | | - | - | - | | Create a disk | 9.3 ms | 11 ms | | Checkpoint a disk | 13 ms | 19 ms | | Branch from a checkpoint | 33 ms | 36 ms | | Delete a disk | 8.1 ms | 10 ms | | Mount a disk | 50 ms | 58 ms | | 4 KiB read, client cache miss | 0.50 ms | 0.75 ms | | 4 KiB write, `O_SYNC` | 2.0 ms | 8.2 ms | Checkpoint and branch times don't depend on disk size. A client cache miss is served from the file server's [cache](/details/caching). Data that has to come from the data source takes longer, typically 10 to 30 ms for Amazon S3. An `O_SYNC` write returns once the file server has made it durable. We measured these on October 27, 2026 from a c7gn.2xlarge in AWS `us-east-1`, using the script in [Latency benchmark](/reference/benchmarks/latency). ## Sandboxes | Limit | Current | | - | - | | Max vCPU per sandbox | 32 | | Max memory per sandbox | 64 GiB | | Max session length | 24 hours | | Default idle timeout | 30 minutes | | Max idle timeout | 24 hours | | Default concurrent commands per sandbox | 32 | | Max concurrent commands per sandbox | 256 | A sandbox pauses when its session reaches 24 hours or its idle timeout. On the Free plan, a session lasts at most 30 minutes. See [Running long jobs](/compute/sandboxes/introduction#running-long-jobs). ## Serverless execution | Limit | Current | | - | - | | vCPU per command | 2 | | Memory per command | 4 GiB | | Max command duration | 5 minutes | | Max command length | 100 KiB | | Max output returned per stream | 4 MiB each for stdout and stderr | Output past 4 MiB on a stream is dropped. See [Serverless execution limits](/compute/serverless-execution#limits). ## S3 API | Limit | Current | | - | - | | Max `PutObject` size | 5 GiB | | Max object size with multipart upload | 5 TiB | | Max parts per upload | 10,000 | | Min part size | 5 MiB | | Max part size | 5 GiB | | Max `CopyObject` source size | 5 GiB | | Max keys per `ListObjectsV2` page | 1,000 | | Max keys per `DeleteObjects` request | 1,000 | | Default share URL lifetime | 24 hours | | Max share URL lifetime | 7 days | The last part of a multipart upload can be smaller than 5 MiB. ## Reproduce these numbers The script and our full results are on [Latency](/reference/benchmarks/latency). # Region availability Source: https://docs.archil.com/reference/regions Archil regions and availability Archil disks are available in multiple regions across AWS and Google Cloud Platform (GCP). When creating a disk, you can select the region that best suits your needs. For optimal performance, we recommend choosing a region closest to where your compute resources are located. ## AWS regions Archil disks are available in the following AWS regions: * **US East (N. Virginia)** - `aws-us-east-1` * **US West (Oregon)** - `aws-us-west-2` * **EU West (Ireland)** - `aws-eu-west-1` ## Google Cloud Platform regions Archil disks are available in the following GCP regions: * **US Central (Iowa)** - `gcp-us-central1` > [!NOTE] > > We are continuously expanding our regional presence. If you need Archil in a region not listed here, please [contact us](mailto:support@archil.com?subject=Region%20access%20request). ## Preview regions `aws-us-east-2` (Ohio), `gcp-us-west1` (Oregon), and `gcp-europe-west4` (Netherlands) are available in preview. Other preview regions may also be available; [contact us](mailto:support@archil.com?subject=Region%20access%20request) to confirm access and supported features before choosing one for a workload. ## Compute availability In Archil's public regions, [serverless execution](/compute/serverless-execution) (`disk.exec` and `disk.grep`) and [persistent sandboxes](/compute/sandboxes/introduction) are available in the AWS regions. [BYOC deployments](#private-deployments-byoc) support both on every cloud and on bare metal. | Region | Code | Disk storage | Serverless execution | Persistent sandboxes | | - | - | - | - | - | | US East (N. Virginia) | `aws-us-east-1` | ✅ | ✅ | ✅ | | US West (Oregon) | `aws-us-west-2` | ✅ | ✅ | ✅ | | EU West (Ireland) | `aws-eu-west-1` | ✅ | ✅ | ✅ | | US Central (Iowa) | `gcp-us-central1` | ✅ | — | — | | BYOC (AWS, Google Cloud, Azure, or bare metal) | Per deployment | ✅ | ✅ | ✅ | A disk created in a public region without serverless execution works for storage, but calling `exec` returns **`Exec is not enabled`** (and `grep` returns `Grep is not enabled`). If you plan to run commands on a disk in a public region, create it in one of the AWS regions above. ## Private deployments (BYOC) [Bring Your Own Cloud](/reference/byoc) deployments run on any cloud, including AWS, Google Cloud, and Azure, as well as on bare metal. A BYOC deployment can include the full Archil platform: disks, [persistent sandboxes](/compute/sandboxes/introduction), and [serverless execution](/compute/serverless-execution). The public-region limits above don't apply to BYOC. Archil fully manages each BYOC deployment, and the telemetry it sends to Archil is scrubbed of PII and PHI before it leaves your cluster. There are no public Azure regions, so Azure is available only through BYOC. Archil's public regions run sandboxes and serverless execution in AWS because that's where Archil hosts its own compute, not because of a limit in the software. See [Why public regions differ](/reference/byoc#why-public-regions-differ). # Terraform provider Source: https://docs.archil.com/reference/terraform Manage Archil disks, users, and API tokens with Infrastructure as Code The [Archil Terraform provider](https://registry.terraform.io/providers/archil-data/archil/latest) enables you to manage Archil resources using Infrastructure as Code. It supports creating and managing disks, API tokens, and disk user authorizations. ## Installation Add the provider to your `required_providers` block: ```hcl theme={null} terraform { required_providers { archil = { source = "archil-data/archil" } } } ``` Then run `terraform init` to install the provider. ## Provider configuration ```hcl theme={null} provider "archil" { api_key = var.archil_api_key region = "aws-us-east-1" } ``` ### Arguments | Argument | Description | Required | | - | - | - | | `api_key` | API key for authentication. Can also be set via `ARCHIL_API_KEY` environment variable. Accepts the key with or without the `key-` prefix. | Yes | | `region` | Archil region: `aws-us-east-1`, `aws-us-west-2`, `aws-eu-west-1`, or `gcp-us-central1`. `aws-us-east-2`, `gcp-us-west1`, and `gcp-europe-west4` are available in preview; confirm access before using it. Can also be set via `ARCHIL_REGION`. Default: `aws-us-east-1`. See [Region availability](/reference/regions) for details. | No | | `endpoint` | Override the API endpoint URL. Can also be set via `ARCHIL_ENDPOINT`. | No | > [!WARNING] > > Store your API key securely. Use a `terraform.tfvars` file (and add it to `.gitignore`) or an environment variable rather than hardcoding the key in your configuration. ## Resources ### archil_disk Creates and manages an Archil disk. A [storage mount](/concepts/data-sources) is optional. ```hcl theme={null} resource "archil_disk" "default" { name = "my-disk" } # Synchronize with your own S3 bucket resource "archil_disk" "with_obj_store" { name = "my-disk" mount { type = "s3" bucket_name = "my-s3-bucket" } } ``` #### Arguments | Argument | Description | Required | | - | - | - | | `name` | Disk name (1-100 chars, alphanumeric, dashes, underscores). | Yes | | `mount` | Storage mount configuration block. | No | #### Mount block arguments | Argument | Description | Required | | - | - | - | | `type` | Mount type: `s3`, `gcs`, `r2`, `s3-compatible`, or `azure-blob`. | Yes | | `bucket_name` | Bucket name. | When `type = "s3"`, `"gcs"`, `"r2"`, or `"s3-compatible"` | | `bucket_endpoint` | Storage endpoint URL (auto-populated for S3). | When `type = "r2"` or `"s3-compatible"` | | `bucket_prefix` | Prefix within the bucket. | No | | `bucket_region` | Bucket region (e.g., `us-east-1`). Auto-detected when omitted. | No | | `access_key_id` | Access key ID (sensitive). | When `type = "gcs"`, `"r2"`, or `"s3-compatible"` | | `secret_access_key` | Secret access key (sensitive). | When `type = "gcs"`, `"r2"`, or `"s3-compatible"` | | `session_token` | Session token for temporary credentials (sensitive). | No | | `session_id` | Session identifier for IAM role-based auth. Auto-generated when no access keys are provided; readable after apply as `mount[0].session_id` for use in bucket policies (see [S3 bucket resource policy](#s3-bucket-resource-policy)). | No | | `container_name` | Azure blob container name. | When `type = "azure-blob"` | | `endpoint` | Azure blob endpoint URL. | No | | `storage_account_name` | Azure storage account name. | No | | `tenant_id` | Azure AD tenant ID. | When `type = "azure-blob"` | | `client_id` | Azure AD client ID. | When `type = "azure-blob"` | | `client_secret` | Azure AD client secret (sensitive). | When `type = "azure-blob"` | > [!NOTE] > > All disk attributes require replacement -- changing any attribute will destroy and recreate the disk. #### Attributes | Attribute | Description | | - | - | | `id` | Disk ID (e.g., `dsk-0123456789abcdef`). | | `status` | Disk status (e.g., `available`). | | `organization` | Owning organization. | | `provider_name` | Cloud provider (e.g., `aws`). | | `region` | Disk region (e.g., `aws-us-east-1`). | | `created_at` | Creation timestamp. | | `data_size` | Total data size in bytes. | | `token` | Auto-generated mount token (sensitive, only available at creation). Use it to mount the disk without creating a separate disk user. | | `token_identifier` | Identifier of the auto-generated token user, for managing or removing it. | #### S3 bucket resource policy When an `s3` mount has no access keys, Archil is authorized through a [bucket resource policy](/data-sources/s3-object-storage#bucket-resource-policy). A policy attached to your bucket grants access to Archil's IAM role, and an `archilSessionId` condition scopes that grant to this one disk. ```hcl theme={null} resource "archil_disk" "app" { name = "my-app-disk" mount { type = "s3" bucket_name = aws_s3_bucket.data.id } } resource "aws_s3_bucket_policy" "archil_access" { bucket = aws_s3_bucket.data.id policy = jsonencode({ Version = "2012-10-17" Statement = [{ Sid = "AllowArchilAccess" Effect = "Allow" Principal = { AWS = "arn:aws:iam::904233114299:role/archil-s3.prod.us-east-1" } Action = ["s3:GetBucketLocation", "s3:ListBucket", "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload"] Resource = [aws_s3_bucket.data.arn, "${aws_s3_bucket.data.arn}/*"] Condition = { StringEquals = { "aws:PrincipalTag/archilSessionId" = archil_disk.app.mount[0].session_id } } }] }) } ``` The `Principal` ARN varies by Archil region. See [S3 Object Storage](/data-sources/s3-object-storage#bucket-resource-policy) for the full table and policy details. If your bucket's policy is managed outside this configuration, either export the policy with an `output` and apply it separately or set `session_id` explicitly in the mount block. *** ### archil_api_token Creates and manages an Archil API token for programmatic access. ```hcl theme={null} resource "archil_api_token" "ci" { name = "ci-token" description = "Token for CI/CD pipelines" } ``` #### Arguments | Argument | Description | Required | | - | - | - | | `name` | Token name (1-100 chars). | Yes | | `description` | Token description (max 500 chars). | No | #### Attributes | Attribute | Description | | - | - | | `id` | Token ID (hash). | | `token` | Full token value (sensitive, only available at creation). | | `token_suffix` | Last 4 characters of the token. | | `created_at` | Creation timestamp. | *** ### archil_disk_user Adds an authorized user to an Archil disk. Supports token-based and AWS STS authentication. For token users the server generates a cryptographically secure mount token and returns it in the `token` attribute. ```hcl theme={null} resource "archil_disk_user" "token_user" { disk_id = archil_disk.default.id type = "token" nickname = "ci-pipeline" } resource "archil_disk_user" "iam_user" { disk_id = archil_disk.default.id type = "awssts" principal = "arn:aws:iam::123456789012:role/MyRole" } ``` #### Arguments | Argument | Description | Required | | - | - | - | | `disk_id` | ID of the disk to add the user to. | Yes | | `type` | User type: `token` or `awssts`. | Yes | | `principal` | IAM ARN. | When `type = "awssts"` | | `nickname` | Nickname for the user. | When `type = "token"` | | `ttl` | Time-to-live as a Go duration string (e.g., `"1h"`, `"30m"`, `"24h"`; max `"8760h"`). Mutually exclusive with `expires_at`. | No | | `expires_at` | Absolute expiration as an RFC3339 timestamp (e.g., `"2027-01-01T00:00:00Z"`). Mutually exclusive with `ttl`. | No | | `one_use` | When `true`, the credential works for a single mount authentication only. | No | | `read_only` | When `true`, mounts authenticated with this credential reject writes with `EROFS`. | No | #### Attributes | Attribute | Description | | - | - | | `token` | Server-generated mount token (sensitive, only available at creation). | | `token_suffix` | Last 4 characters of the token. | | `identifier` | Stable identifier for this user, used for removal. | | `expires_at` | Expiration time (computed from `ttl` when set). | | `consumed_at` | When a one-use credential was consumed. | | `status` | Current lifecycle status of the credential. | | `created_at` | Creation timestamp. | ## Data sources ### archil_disk Look up an existing Archil disk by ID or name. ```hcl theme={null} # By name data "archil_disk" "by_name" { name = "my-disk" } # By ID data "archil_disk" "by_id" { id = "dsk-0123456789abcdef" } ``` #### Arguments Exactly one of `id` or `name` must be set. | Argument | Description | | - | - | | `id` | Disk ID to look up. | | `name` | Disk name to look up. | #### Attributes | Attribute | Description | | - | - | | `id` | Disk ID. | | `name` | Disk name. | | `status` | Disk status. | | `organization` | Owning organization. | | `provider_name` | Cloud provider. | | `region` | Disk region. | | `created_at` | Creation timestamp. | | `data_size` | Total data size in bytes. | ## Full example This example creates a disk with an S3 mount authorized by static credentials, a disk user, and an API token. As with the provider's `api_key`, pass real credentials in through variables rather than hardcoding them: ```hcl theme={null} terraform { required_providers { archil = { source = "archil-data/archil" } } } provider "archil" { api_key = var.archil_api_key region = "aws-us-east-1" } variable "archil_api_key" { type = string sensitive = true } resource "archil_disk" "app" { name = "my-app-disk" mount { type = "s3" bucket_name = "my-app-bucket" access_key_id = "AKIAIOSFODNN7EXAMPLE" secret_access_key = "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" } } resource "archil_disk_user" "app" { disk_id = archil_disk.app.id type = "token" nickname = "app-servers" } resource "archil_api_token" "ci" { name = "ci-token" description = "Token for CI/CD pipelines" } data "archil_disk" "app" { name = archil_disk.app.name depends_on = [archil_disk.app] } output "disk_id" { value = archil_disk.app.id } output "mount_token" { value = archil_disk_user.app.token sensitive = true } ``` Mount the disk with the generated token: ```bash theme={null} export ARCHIL_MOUNT_TOKEN=$(terraform output -raw mount_token) sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount my-app-disk /mnt/archil --region aws-us-east-1 ``` ## Import Import a disk using its ID: ```bash theme={null} terraform import archil_disk.default dsk-0123456789abcdef ``` Import an API token using the token value: ```bash theme={null} terraform import archil_api_token.ci key-YOUR-TOKEN-VALUE ``` # Troubleshooting Source: https://docs.archil.com/reference/troubleshooting Solutions to common issues with Archil disks ## Problem: Mount fails with "No such file or directory" ``` ✗ Unable to mount Archil disk to '/mnt/archil': No such file or directory (os error 2) ``` The mount point must already exist as an empty directory before running `archil mount`. Create it first: ```bash theme={null} sudo mkdir -p /mnt/archil sudo archil mount /mnt/archil --region aws-us-east-1 ``` If the directory already exists and you still see this error, confirm you are passing the path exactly as it appears on disk (no typos, no trailing characters) and that a parent directory in the path is not itself a stale Archil mount. ## Problem: Mount fails with permission denied The Archil client cannot authenticate against the disk. This usually means the [disk token](/concepts/authorization#token-based-disk-authorization) is wrong or has been regenerated, or the IAM role is not listed as an authorized disk user. ```bash theme={null} # Verify your IAM identity archil utils get-iam-role ``` Compare the output ARN to the **Authorized Users** list on the disk's Details page in the [Archil console](https://console.archil.com). If you are using disk token authentication, regenerate the disk token from the console — Archil does not store disk tokens in plaintext, so a lost token cannot be recovered. > [!WARNING] > > When passing a disk token through sudo, use `sudo --preserve-env=ARCHIL_MOUNT_TOKEN` so the variable is available to the Archil process. Using `sudo` alone drops the environment. ## Problem: Mount hangs or times out The client cannot reach the Archil control plane. Common causes are a wrong region, a network connectivity issue, or DNS resolution failure. ```bash theme={null} # Confirm the region matches the disk's region shown in the console sudo archil mount /mnt/archil --region aws-us-east-1 # Test connectivity to the Archil control server curl -v https://mount.green.us-east-1.aws.prod.archil.com:8100 2>&1 | head -20 ``` Ensure your security groups or firewall rules allow outbound traffic on ports 8100 and 5000-5002. If you are running inside a VPC without public internet access, verify that DNS resolution is working and that a NAT gateway or VPC endpoint is configured. ## Problem: Mount fails because the client is no longer supported If the mount fails with `this version of the Archil client is no longer supported by the server`, your client is older than the minimum version the server, your organization, or the disk requires. Upgrade the client with `archil upgrade` or rerun the [installer](/mounting/linux), then retry the mount. ## Problem: Files not appearing in S3 This is expected behavior. Archil provides [eventual synchronization consistency](/details/consistency#between-the-disk-and-its-backing-bucket) between the disk and the backing data source. Writes are durable on the disk as soon as they are committed, and Archil copies them to S3 in the background, usually tens of seconds after a file's last write. A file that keeps changing is copied within about 10 minutes. You can't force the copy. If your S3 clients need to see writes immediately, use the [Archil S3 API](/protocols/s3-api), which reads the disk directly. ```bash theme={null} # Verify the disk is healthy and accepting writes archil status /mnt/archil ``` ## Problem: Another client doesn't see my changes Another client reads a change once it has reached the disk and the other client isn't answering from an older cache entry. Check both sides: * **On the writer,** run `fsync` on the file, or `archil sync /mnt/archil` for the whole mount. On exclusive and shared mounts, `close` doesn't wait for writes to reach the disk. * **On the reader,** open the file again; opening a file checks it with the disk. A new file can still be missing from a directory listing the reader cached, for up to five minutes by default. Run `archil invalidate-cache /mnt/archil` to drop the reader's cache, or lower the listing lifetime with `archil set-cache-expiry`. ```bash theme={null} # Writer sudo archil sync /mnt/archil # Reader sudo archil invalidate-cache /mnt/archil ``` See [Consistency](/details/consistency#between-archil-clients) for the full rules. ## Problem: Another client cannot write to files Only one Archil client can have write ownership (a "delegation") over any given file or directory at a time. When a client mounts without `--shared` or `--conditional`, it checks out the entire root, blocking all other clients from writing. A second mount without `--shared`, `--conditional`, or `--read-only` will fail outright, since the first client holds the root delegation — pass `--shared` or `--conditional` on every mount to allow concurrent writes, or `--read-only` on the other clients if only one client needs to write. **Solutions:** ```bash theme={null} # Option 1: Mount every client in conditional mode, so any client can write without a checkout sudo archil mount /mnt/archil --region aws-us-east-1 --conditional # Option 2: Mount both clients in shared mode and partition by directory sudo archil mount /mnt/archil --region aws-us-east-1 --shared archil checkout /mnt/archil/my-dir/ # Option 3: Use subdirectory mounts so each client owns a different subtree sudo archil mount :/models /mnt/models --region aws-us-east-1 # Client A sudo archil mount :/data /mnt/archil --region aws-us-east-1 # Client B # Option 4: Keep one client as the only writer and mount the others read-only sudo archil mount /mnt/archil --region aws-us-east-1 # Writer sudo archil mount /mnt/archil --region aws-us-east-1 --read-only # Each reader # Option 5: Force-claim ownership (use with caution) archil checkout /mnt/archil/contested-dir/ --force ``` > [!WARNING] > > `--force` immediately revokes another client's delegation. Any unflushed writes from that client will be lost and subsequent `fsync` calls will return `EIO`. ## Problem: Writes fail on a conditional mount On a [conditional mount](/concepts/sharing-disks#conditional-mode), the `archil` client buffers recent writes and sends them to the disk in the background, so a write can succeed and then fail when it's sent. There are two common causes: * **Another client deleted the file.** Writes to the file that hadn't reached the disk are lost. Reads, writes, `fsync`, and `close` on file descriptors still open on that file return `ESTALE`. The rest of the mount keeps working. * **Another client checked out the file or a directory above it** with `archil checkout`, or mounted the disk without `--shared` or `--conditional`. The mount becomes read-only: writes that hadn't reached the disk are lost, further writes return `EROFS`, and `fsync` and `close` return `EIO`. Reads keep working. Creating, renaming, or deleting a file under a path another client has checked out fails immediately with `EROFS`, without affecting the rest of the mount. To recover a mount that has become read-only, unmount it and mount the disk again: ```bash theme={null} sudo archil unmount /mnt/archil sudo archil mount /mnt/archil --region aws-us-east-1 --conditional ``` Writes to the path the other client checked out will keep failing until that client releases it with [`archil checkin`](/reference/archil-cli#checkin) or unmounts. To avoid this, don't check out paths that clients in conditional mode are still writing to. ## Problem: Permission denied when accessing files This is expected. The Archil FUSE client mounts as root, so all files start with root ownership. ```bash theme={null} # Change ownership to your user sudo chown -R $USER:$USER /mnt/archil # Or run commands with sudo sudo ls /mnt/archil ``` ## Problem: There's a `.archil` directory at the root of my mount Every Archil disk has a hidden `.archil/` directory at its root. Archil uses it to hold data that still needs to exist on the disk but is no longer reachable through the normal filesystem — most commonly, files that have been unlinked while a client still has them open. POSIX requires those files to keep working until the last handle is closed, so Archil parks them under `.archil/` until they can be reclaimed. The directory and its contents synchronize back to your backing S3 bucket along with the rest of the disk, so you'll see a `.archil/` prefix there as well. The exact layout is an implementation detail and may change between releases. Treat it as opaque: don't read from, write to, or delete anything inside `.archil/`, either on the mount or in S3. ## Problem: Data not appearing after writing to S3 directly Objects created directly in S3 (or another backing store) are subject to [eventual synchronization consistency](/details/consistency#from-the-bucket-to-the-disk). Archil lists a directory in the bucket again in the background when the directory is read and its listing is more than five minutes old, so it can take a few minutes for new objects to appear on the Archil disk. A mounted client also caches directory listings. To stop it from adding its own delay on top, reduce the directory cache expiry: ```bash theme={null} # List the directory from the disk on every read archil set-cache-expiry /mnt/archil/incoming --readdir-expiry 0 # Or use a short TTL for a performance/freshness balance archil set-cache-expiry /mnt/archil/incoming --readdir-expiry 5 ``` > [!NOTE] > > Lower cache expiry values improve freshness but increase the number of requests to the Archil server, which may affect read performance. ## Problem: SDK connection fails The TypeScript SDK cannot connect to the disk. Check the following: ```typescript theme={null} const client = await ArchilClient.connect({ region: "aws-us-east-1", // Must match the disk's region diskName: "myorg/mydisk", // Format: org/disk-name or disk ID authToken: process.env.ARCHIL_DISK_TOKEN, // Disk token; required outside AWS }); ``` 1. **Credentials** — Ensure a valid [disk token](/concepts/authorization#token-based-disk-authorization) is set in `ARCHIL_DISK_TOKEN`, or that IAM credentials are available (e.g., running on EC2 with an attached role). Note that a disk token is different from a Control Plane API key. 2. **Disk name** -- Use the owner-qualified format `org/disk-name`, not just the disk name alone. 3. **Region** -- The region string must match exactly (e.g., `aws-us-east-1`, not `us-east-1`). 4. **Node.js version** -- The native module requires Node.js 18 or later. # Uninstall tools Source: https://docs.archil.com/reference/uninstall Remove the Archil client, the disk CLI, and the CSI driver. This page covers removing each Archil tool: * **The Archil client**, which mounts disks: [Linux](#linux) and [macOS](#macos) * **The [`disk` npm package](#disk-npm-package)**, which provides the `disk` and `sandbox` CLIs * **The [CSI driver](#kubernetes)** for Kubernetes Removing the client works the same way on Linux and macOS: 1. **Unmount every Archil disk.** A clean unmount flushes recently written data to Archil and releases any files you have checked out. If you remove the client while a disk is still mounted, writes that haven't been flushed yet are lost. 2. **Remove the software.** 3. **Optionally remove local files.** These include saved credentials, so remove them if the machine is being decommissioned or handed to someone else. > [!NOTE] > > Uninstalling the client doesn't change your disks or the data on them. Disks stay in your account until you delete them > from the [Archil console](https://console.archil.com). ## Linux The Linux client is distributed as a package named `archil`. It installs two files: | Path | Purpose | | - | - | | `/usr/bin/archil` | The `archil` client | | `/sbin/mount.archil` | The helper that lets `mount` and `/etc/fstab` mount Archil disks | **1. Unmount every disk** List the Archil disks currently mounted on the server: ```bash theme={null} grep archil /proc/mounts ``` Unmount each one with `archil unmount`, which flushes pending writes before it disconnects: ```bash theme={null} sudo archil unmount /mnt/archil ``` Then confirm that no client processes are still running: ```bash theme={null} pgrep -a archil ``` If a process is still listed, it's still flushing data. Wait for it to exit, and don't `kill -9` it. **2. Remove automatic mounts** If you [mount disks automatically on startup](/mounting/linux#mount-automatically-on-startup), remove the Archil entries from `/etc/fstab`. Otherwise, the server tries to mount them at the next boot and fails. ```bash theme={null} grep -nw archil /etc/fstab # find the entries sudo nano /etc/fstab # delete them sudo systemctl daemon-reload # on systemd hosts, drop the generated mount units ``` Also remove any startup scripts, systemd units, or configuration management that runs `archil mount`. **3. Remove the package** > [tab] Debian / Ubuntu > > ```bash theme={null} > sudo apt remove archil > ``` > [tab] RHEL / Fedora / Amazon Linux > > ```bash theme={null} > sudo dnf remove archil > ``` > > On older systems, use `sudo yum remove archil`. On systems without either, use `sudo rpm -e archil`. > [tab] Alpine > > On Alpine, the installer copies a standalone binary instead of installing a package. Remove it directly: > > ```bash theme={null} > sudo rm -f /usr/bin/archil > ``` The FUSE library that the installer pulled in (`libfuse2`, `fuse-libs`, or `fuse`) is left installed, because other software may depend on it. **4. Remove leftover files (optional)** The package manager doesn't remove the files below, even with `apt purge`, because the package didn't create them. | Path | Contents | | - | - | | `/etc/archil/` | Client configuration, including your automatic-upgrade preference | | `~/.cache/archil/` | Upgrade-check state, created separately for each user who ran `archil`, including `root` | | `/var/run/archil/` | Control sockets for running mounts. Empty once every disk is unmounted. | | `/var/log/archil/` | Log files, present only if the client couldn't log to journald | | Any directory you passed to `--log-dir` | Per-mount log files | ```bash theme={null} sudo rm -rf /etc/archil /var/run/archil /var/log/archil /root/.cache/archil rm -rf "${XDG_CACHE_HOME:-$HOME/.cache}/archil" # repeat for each user who ran archil ``` To confirm that the client is gone, run the following command. It should print nothing: ```bash theme={null} command -v archil mount.archil ``` ## macOS The [macOS client](/mounting/macos) is a menu bar app. The first time it launches, it asks for your administrator password and installs a mount helper and a `sudoers` rule, so that mounting and unmounting disks doesn't require a password. Removing the app from `/Applications` doesn't remove these files, so the steps below remove them separately. | Path | Purpose | | - | - | | `/Applications/Archil.app` | The menu bar app and its Finder and file system extensions | | `/usr/local/bin/archil-mount` | Mount helper | | `/etc/sudoers.d/archil` | Lets administrators run the mount helper without a password | | `~/Library/Application Support/Archil/` | Your sign-in session and the tokens for each mounted disk | **1. Unmount every disk and quit Archil** Click the Archil icon in the menu bar and choose **Quit**. Quitting unmounts every disk and waits for pending writes to flush. To check for any disks that are still mounted, run: ```bash theme={null} mount | grep archilfs ``` Unmount anything listed with `sudo umount "/Volumes/"`. **2. Remove the app** Unregister the file system extension, then delete the app: ```bash theme={null} pluginkit -r /Applications/Archil.app/Contents/Extensions/ArchilMountExtension.appex sudo rm -rf /Applications/Archil.app ``` **3. Remove the mount helper and sudoers rule** Remove the `sudoers` rule first, then the helper it points to: ```bash theme={null} sudo rm -f /etc/sudoers.d/archil sudo rm -f /usr/local/bin/archil-mount ``` > [!WARNING] > > Don't skip this step. The `sudoers` rule lets administrators run `/usr/local/bin/archil-mount` as root without a > password. Remove the rule even if the helper file is already gone. **4. Remove your data (optional)** These folders hold your Archil sign-in session, the tokens for disks you've mounted, and the app's settings and caches. Run the following as each macOS user who used Archil: ```bash theme={null} rm -rf ~/Library/Application\ Support/Archil rm -rf ~/Library/Group\ Containers/group.com.archil.Archil rm -rf ~/Library/Containers/com.archil.Archil.Extension rm -rf ~/Library/Containers/com.archil.Archil.FinderSync rm -rf ~/Library/Caches/com.archil.Archil ~/Library/Caches/Archil rm -rf ~/Library/HTTPStorages/com.archil.Archil defaults delete com.archil.Archil ``` If macOS denies access to the `Containers` or `Group Containers` folders, give Terminal **Full Disk Access** in **System Settings → Privacy & Security**, or delete the folders from Finder. > [!NOTE] > > The [`disk` CLI](#disk-npm-package) stores its profiles in `~/Library/Application Support/Archil/disk`, so the > first command above removes those too. To keep them, delete only the files directly inside > `~/Library/Application Support/Archil` and leave the `disk` folder in place. To confirm that the client is gone, open **System Settings → General → Login Items & Extensions → File System Extensions**. Archil should no longer be listed. If it still appears, log out and log back in. > [!NOTE] > > Deleting local files doesn't revoke the credentials on Archil's side. If the machine was lost or is moving to someone > else, also remove its disk tokens from each disk's users in the [Archil console](https://console.archil.com) or with the > [Remove Disk User API](/api-reference/disk-users/remove-user). ## `disk` npm package The [`disk` package](/reference/disk-cli) provides two commands, `disk` and `sandbox`. It doesn't install anything outside npm, so how you remove it depends on how you added it. **1. Remove the package** > [tab] npx > > If you only run `npx disk`, there's no install to remove. npx keeps a copy in npm's cache, which npm manages > on its own. > [tab] Global install > > ```bash theme={null} > npm uninstall -g disk > ``` > [tab] Project dependency > > Run this in your project's directory: > > ```bash theme={null} > npm uninstall disk > ``` > > With other package managers, use `pnpm remove disk`, `yarn remove disk`, or `bun remove disk`. **2. Delete saved profiles** If you saved credentials with `disk profile create`, they stay on disk after you remove the package. List your profiles, then delete each one together with its stored API key: ```bash theme={null} npx disk profile list npx disk profile delete ``` Or delete the whole configuration directory: | Platform | Directory | | - | - | | Linux | `~/.config/archil/disk`, or `$XDG_CONFIG_HOME/archil/disk` if it's set | | macOS | `~/Library/Application Support/Archil/disk` | | Windows | `%APPDATA%\Archil\disk` | If you set `ARCHIL_DISK_CONFIG_DIR`, profiles are stored in that directory instead. > [!NOTE] > > Deleting a profile only removes the local copy of its API key. To revoke the key itself, delete it in the > [Archil console](https://console.archil.com). ## Kubernetes The [CSI driver](/reference/csi-driver) runs its own copy of the client inside your cluster, so the server steps above don't apply. First delete the pods and PersistentVolumeClaims that use Archil volumes. Then uninstall the Helm release: ```bash theme={null} helm uninstall archil-csi-driver --namespace archil-system ``` # Verify Linux downloads Source: https://docs.archil.com/reference/verify-linux-downloads Verify the authenticity and integrity of an Archil Linux release before installing it. Archil publishes a signed SHA-256 checksum manifest for every signed Linux release. Use it to verify a downloaded DEB, RPM, or musl binary **before** installing or running it. This procedure verifies that an artifact matches the release signed by Archil. It does not replace your normal review of the release version. ## Normal install or manual verification? For a normal installation, use the convenience installer: ```bash theme={null} curl -fsSL https://archil.com/install | sh ``` The convenience installer is the standard installation path. If you want to verify a versioned artifact before executing it, follow the manual OpenPGP signature and checksum procedure below. ## What you need * `curl` * `gpg` (GnuPG) * `sha256sum` (supplied by GNU coreutils or BusyBox) The commands below use a temporary GnuPG keyring and do not change your normal GnuPG configuration. ## Trust Archil's release signing key Archil's production Linux release signing-subkey fingerprint is: ```text theme={null} 075082A64956DDB6281A08A827A9B35E790F664A ``` Before trusting a downloaded certificate, confirm that its fingerprint matches the value above. This documentation page is independent from the S3 release bucket. ## Archil Linux release public certificate The following is Archil's public release certificate. Confirm that its fingerprint matches the value above before using it to verify a release. ```text theme={null} -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBGpZOQ8BEADCz/oM30v+wqE0AJBo3zrPG29s5WmAt6iZA+2MC1JMN60EPlfV TE5rBNvwJ6LbR1OaWNPVrwAvcPAHj2n1+bQI0ACuh7iRrcLh5670qLxluod1vJO8 +//8RzzSV4x+SEU6liYX+ncR0OlT1OyeJovQOmtc+yR2iIKf5tnUOww6DrO/JjYY QRG5M27eGHl4n3cZUbzfC46Arup2oFI4hMYXcO21EKwzVxAOX1V9193/1hvqSeCy BMT2F8U/dz7sHDQl3uutkSi71QmVc6y3FrcZGtF8BPXOOKqA/Y1CXCgEymmzIMy9 S8sxX/9Zmw6d1P0sOiz4G6vgGEFXS0d/lf/Qkmv6NewHCDFFbrWm3PnkgBF6weKA irSSrFxcaPR4mrfXJaSw6ZYznfLDsIFQqd8ixRjQO64jhrNgk3QxEQ5ix/P7krE0 0WoIhZWNoUcM3GuWC5cEFDIop/Y76+XQRvvCzm4xBVnLiRAeQ+emvSmFDPmMZumn lOJKvCHNX5UhJM9BgpsRdUhYIYsMSSzlf18UfyPFROIPkgGaZAiz9EgtROtKeKNS xQyrfz9fVVJZNhXQ20GxYQBlv1KiLCoolVCW5+IRxS1kQjd8LvMdB1ANE26KSBo5 O7EozzjbU1CwLerJkX41sBD9tBQWCcmEJfSc9hcPnkBHbxZPK0WYw2SfNwARAQAB tCtBcmNoaWwgTGludXggUmVsZWFzZXMgPHNlY3VyaXR5QGFyY2hpbC5jb20+iQJz BBMBCABdFiEEexRMpLwXJLDlcvF7+t6vSq4R2tAFAmpZOQ8bFIAAAAAABAAObWFu dTIsMi41KzEuMTIsMCwzAhsBBQkB4TOABQsJCAcCAiICBhUKCQgLAgQWAgMBAh4H AheAAAoJEPrer0quEdrQhA0QALdizU5X0v/2/V2m8NdrIW7sI3/J/w6UdRzuTkGT u8Q1rK3oVHcQUmETlnqCw3YFq632cZdBtl1CPIbwIx8sn+Hz0d3nFji/9PjK+nLD FJXzHaD5hiLZ9QPH1U20jU5ogXx2cWUuI5geGGcoQLJOGyt1tjnCRCAaSD+0qb0c vkXvP5dxgpSY3WRyhH/LOEf0ZEsSjwiQkmpf/Bl7fF6gBjpuKRJHqRN32ui6pDBw dO24WXO422jBUQvjWVABnErAhsF9z0IF/4WQmy1j5cluhxLGuwKu/RIl6M+U/c2B BKfDs1lH0vB+mlyg/Wvsl0QCqpldHNT5mMBgXbwDJeSn+WQYTTvur2IXOcxbwv+k hBVCyiFufpkT88FTLLjKCZA/tiu9UWZEiTOG9C+fJV1zo0yqLTw+XvnVm/kMgixs vIKams+HWGH641Fd+M3NbFTvhWm30SmEYXAJ7kQUGSvDPKCxHNCrxzxSAdWVWOFw XiULzcyRPi0+/Hgccr3aFTiN+fRTfmiAVawuyobkSzwhSXsSAFffyegZ70heKd4E dOc5f88X9yr7yM9u92meNtK6Y4jZ0JYMmPaUJzFFxIDwxiqlch48805cfwvOYiA6 Ru5pDoxvzwWBMUjvQryqGESl8vKd4As21H5FUrPLVxE8MGDdDzphe571hxPL5yKO pAHuuQINBGpZOTwBEAC0kVgelpyJGSB7/+QRiZuS8pOgGBKZZw2XWaoxqWADky9S tjTEMQsa4PoGWsmaGyiDoY8T4vvqQbHXYaDctVpWP8cvt1kdg9mVGSougyzr7sbF 1YyMUpINIEpSwFLauhzHz/SatPrM4XPbulkm+AHpUcOsZdm2hy55tYCQkwtDiZB/ 0Oi/qbNYHV9f96fpJePgfU0Jg2wC55r2Mhi70SsF77lJV0hGbwNmIgIqBjs/6eDJ q34VSS2A7TFeIyCRyyXfqCNX7cmMNCuxORgygzAJCCaTWzf2MAgSszU1Vm5mR4We 7l4c0zlZVQ2ajG7R2u1gU9Kc+JqBpnhyxsrihUoNAa4a96sYY+xxmc4zZd1w4GNg fQ7OxP+Mj09Tn3QCcDWJ7QiBvwq8C5VwWqmJjvZm133xb2U7LUaqHbjZ6xibl4rs qaywFfn+eOllbFi2MI3Ik/ysVj8V0j9TOPrhq6PGo3rIpQe0qMovVa5DgM1n2uYU YxwJPMGkSnwFOQ4PkrucMra5Kyl2A5VObRvKljnBtvdUHgpTzTPhP/FNUNAWUZHu s1v3QZRlCElWlXL9YS9wJ6Omy+5mWxAdv/OrGQxzkiWIwpVdU4e/pXEy+F8u52l+ dJ6APlARCnIb2P1ts6q78zsxTAIDvdIwYvUuIGDYVcdjP0ol0CSdgllErMF/ZwAR AQABiQSOBBgBCABCFiEEexRMpLwXJLDlcvF7+t6vSq4R2tAFAmpZOTwbFIAAAAAA BAAObWFudTIsMi41KzEuMTIsMCwzAhsCBQkB4TOAAkAJEPrer0quEdrQwXQgBBkB CAAdFiEEB1CCpklW3bYoGgioJ6mzXnkPZkoFAmpZOTwACgkQJ6mzXnkPZkphfA// Yz2BxmKU0zZp5pWVarHLfPxMbcVL0qFMdEvEPfhKA7B6eBAoygwFKjEuSD5jLHKE E0k0Td6XtvVcaC9NQCskkRFZtDxlSz+eY1fJRrFaCwiwnlxT8fsd89nsvMhwWn0Q bi7DMvf0aAK9cpG2FeNwcd9i4RdyPwRGCAsDj1UA6bEMoDL/eaT2vImloSlGk0BL bGsHpj5PJ1kLj4yGq27z3Gh+uTukWbmJuvy8vxlMolpMpXCdBFgyvIGAlXy9q3Hl GeziPkftVQ8oCW9bmUiZVaYnhLJgoxq8gH0HQI4NA9+JaUg0QV5ylE1EyUk9NDWI sKkF98ia4hpcUHxs1JjEo4vpJHolFiVxcZbARIDCyoO3wIUkWH8FE7xlhSsFRSx4 +IpAYfCRRdG2Za24y9j4CcM2O0LXni+/8LvNt4D56nfB5T4m33Ds4nFV0Sr55ZbU 0RdJFSmxO0OHGjt3qDTfvYabdtv3xyoIL1XhZ4Yk2xLUXONs5UyphdenAvbXipkW LYBRQOnINhFI7K3bBBibjKw5Z651AW8x1e+eAgOTZddF/cQwF2bwEoJfDCvF2ctr 3h0fjJD3Awt1oNG2EWKHTFKH82ZYjdfoiQnUYJO/f8iXj+M3WcuZghhKMz5XJLv1 Eu/oRuW9k25lqm/6Og3tB6mFFQgwYEC1PLc/xFiu8mYieRAAi3nlcDnXpv/TAL1a Pa49wT8KEBWp/rgqzbEjieZq2IZn1XmgcIaR4e4Ei7WmIcZR7Tgzq+0mjodUqCb4 g0VJXrwyG2r9tBhCfn5BYcABfpUC2cLqVQWRBO623njV0GHZk/UXwimk/bTJpKCY 0+JJ+QzQ3HeAxWDZGRCI7TI+CxH/8hGtp+TxtbvGwLkL4RvJBlLXXHTY9IXBiLmT 9Fln2YgwY9AFKLaCD9A6UrPq/QuImgUTLJCPfRbu2oLnjuolP0SiMard/nF20BFz 4W7DVleNo0zm56n+QY6kMvkLLUH9aLPrDi/iIahN1kKztfK5XCYDEzPqTmgYoVlU p772+AA33toxxj7tavf8yACSNVUicXUA5wP2+xoDi/3vOz1ORrgJpZdSqlU9+fzq NVYippYkslYf1bGQdLpajQunOZpx37vyDqVliE1N9/djPn0SvTdYd/tfg0E+WAMH g0L4kPlzkPcswZVrNqxlcA0dsqV61UtFSGXmNfLd5Plg+OxtWoh+iWqnQ3QJ/EsW iymXJb12hpj4RPvIV2vgllRytvfBjct78D7sTYflEOUKTV0pStg6IhiTm2LxJm60 bOXxYVKzhuDRBM8UP3KlShryLTLntPof7hHsTHsB15DlCsEIMfTTTuKIRoHSf3dU YNtC4jeY0l42wAa6kB9lPmUVAHI= =pX2j -----END PGP PUBLIC KEY BLOCK----- ``` ## Choose a release and artifact Choose the exact version and filename you intend to install. Signed Linux artifacts and their verification metadata are published under `https://s3.amazonaws.com/archil-client/pkg/`. To select the version currently used by the production install channel, read its `latest` pointer: ```bash theme={null} curl --fail --location https://s3.amazonaws.com/archil-client/pkg/latest ``` The `latest` pointer is mutable and is **not** authenticated. Use it only to choose a version to download; the signature and checksum steps below verify the artifact. For a reproducible installation, record an explicit version instead of relying on `latest`. For example, if the release includes `archil_0.8.21-123_amd64.deb`, use `0.8.21-123` as `VERSION`: ```bash theme={null} VERSION="0.8.21-123" BASE_URL="https://s3.amazonaws.com/archil-client/pkg" ARTIFACT="archil_${VERSION}_amd64.deb" ``` Use the exact filename listed in the release. The common artifact names are: | Platform | Artifact name | | - | - | | Debian/Ubuntu AMD64 | `archil__amd64.deb` | | Debian/Ubuntu ARM64 | `archil__arm64.deb` | | RPM AMD64 | `archil-.x86_64.rpm` | | RPM ARM64 | `archil-.aarch64.rpm` | | musl AMD64 | `archil-linux-musl-amd64-` | | musl ARM64 | `archil-linux-musl-arm64-` | For RPMs, use the exact filename from the release. Pre-release versions can have a different RPM-safe version string. ## Download the artifact and verification files Create a clean working directory and download the package, checksum manifest, detached signature, and public certificate for the **same version**: ```bash theme={null} mkdir "archil-verify-${VERSION}" cd "archil-verify-${VERSION}" curl --fail-early --fail --location --remote-name-all \ "$BASE_URL/$ARTIFACT" \ "$BASE_URL/archil_${VERSION}_SHA256SUMS" \ "$BASE_URL/archil_${VERSION}_SHA256SUMS.sig" \ "$BASE_URL/archil_${VERSION}_linux_release_signing_key.asc" ``` Each signed release includes these verification files: | File | Purpose | | - | - | | `archil__SHA256SUMS` | SHA-256 checksums for the six Linux artifacts. | | `archil__SHA256SUMS.sig` | ASCII-armored detached OpenPGP signature for the checksum manifest. | | `archil__linux_release_signing_key.asc` | Public certificate for the Linux release signing key. | If any download fails, stop. Do not install an artifact when its checksum manifest, signature, or certificate is unavailable, and do not mix files from different versions or release channels. If the verification files for a release are unavailable, contact [support@archil.com](mailto:support@archil.com). ## Verify the public key and checksum signature Import the downloaded certificate into a temporary keyring. The first check confirms that the certificate contains Archil's expected signing-subkey fingerprint; the second confirms that this exact key signed the checksum manifest. ```bash theme={null} EXPECTED_FINGERPRINT="075082A64956DDB6281A08A827A9B35E790F664A" export GNUPGHOME="$(mktemp -d)" trap 'rm -rf "$GNUPGHOME"' EXIT chmod 700 "$GNUPGHOME" gpg --batch --import "archil_${VERSION}_linux_release_signing_key.asc" if ! gpg --batch --with-colons --fingerprint --list-keys \ | awk -F: -v expected="$EXPECTED_FINGERPRINT" \ '$1 == "fpr" && toupper($10) == toupper(expected) { found = 1 } END { exit !found }'; then echo "The downloaded certificate does not contain Archil's expected signing fingerprint." >&2 exit 1 fi if ! gpg --batch --status-fd 1 --verify \ "archil_${VERSION}_SHA256SUMS.sig" \ "archil_${VERSION}_SHA256SUMS" 2>/dev/null \ | awk -v expected="$EXPECTED_FINGERPRINT" \ '$1 == "[GNUPG:]" && $2 == "VALIDSIG" && toupper($3) == toupper(expected) { found = 1 } END { exit !found }'; then echo "The checksum manifest did not pass Archil's expected signature validation. We recommend not installing the artifact." >&2 exit 1 fi ``` > [!WARNING] > > If either check fails, do not install or run the artifact. Delete the downloaded files and obtain them again only from the canonical release location. ## Verify the downloaded artifact Extract the one expected checksum entry, then verify your downloaded artifact against it: ```bash theme={null} if ! CHECKSUM_LINE="$(awk -v artifact="$ARTIFACT" \ '$2 == artifact { count++; line = $0 } END { if (count != 1) exit 1; print line }' \ "archil_${VERSION}_SHA256SUMS")"; then echo "Expected exactly one checksum entry for $ARTIFACT." >&2 exit 1 fi if ! printf '%s\n' "$CHECKSUM_LINE" | sha256sum -c -; then echo "Checksum verification failed. Do not install or run $ARTIFACT." >&2 exit 1 fi ``` The command must print: ```text theme={null} : OK ``` If it prints `FAILED`, reports that the artifact is missing from the manifest, or produces no `OK` line, do not install or run the file. ## Install after verification Run only the command applicable to your verified artifact: ```bash theme={null} # Debian/Ubuntu sudo apt install ./$ARTIFACT # RPM-based distributions sudo dnf install ./$ARTIFACT # musl binary, for example Alpine sudo install -m 0755 "$ARTIFACT" /usr/local/bin/archil ``` ## Clean up Remove the temporary keyring when you are finished: ```bash theme={null} rm -rf "$GNUPGHOME" unset GNUPGHOME EXPECTED_FINGERPRINT ``` ## Key rotation and security notices Before a planned signing-key rotation, Archil publishes the replacement fingerprint and retains the prior fingerprint during a documented overlap. If you suspect a signing key has been compromised, do not install new artifacts until Archil publishes a security notice and replacement verification instructions through its documentation. Report suspected security issues to [security@archil.com](mailto:security@archil.com). ## Scope This page covers versioned, signed Linux releases only. It does not cover macOS, Windows, package-repository signing, Sigstore or Cosign attestations, or the unversioned `https://archil.com/install` script. # SDKs Source: https://docs.archil.com/sdks/overview Official Archil SDKs for TypeScript and Python, and the agent framework adapters built on them. Archil has official SDKs for **TypeScript** and **Python**. Both talk to the Archil API over HTTPS, have no native dependencies, and cover disks, serverless execution, persistent sandboxes, and file reads and writes. Neither mounts a disk; to mount one as a local filesystem, use the [`archil` CLI](/mounting/linux). | | [TypeScript](/sdks/typescript) | [Python](/sdks/python) | | - | - | - | | Package | [`disk`](https://www.npmjs.com/package/disk) on npm. [`archil`](https://www.npmjs.com/package/archil) on npm is an alias for it. | [`archil`](https://pypi.org/project/archil/) on PyPI | | Install | `npm install disk` | `pip install archil` | | Runtime | Node.js 22.19 or later | Python 3.10 or later | | Create, list, and delete disks, and manage who can mount them | Yes | Yes | | Run commands with [serverless execution](/compute/serverless-execution) and [search files](/compute/search-files) | Yes | Yes | | Create and manage [persistent sandboxes](/compute/sandboxes/introduction) | Yes | Yes | | Read and write files through the [S3-compatible object API](/protocols/s3-api), and create share URLs | Yes | Yes | | Manage API keys | Yes | Yes | | Calling style | Async (Promises) | Sync and async from the same methods | | Command-line tool | [`disk` CLI](/reference/disk-cli) (`npx disk`) | None | The source for both SDKs is public at [github.com/archil-data/archil-sdk](https://github.com/archil-data/archil-sdk). ## Agent framework adapters These packages turn a disk into ready-made file and `bash` tools for an agent framework. | Framework | Language | Package | | - | - | - | | [AI SDK](/integrations/ai-sdk) | TypeScript | [`@archildata/ai-sdk`](https://www.npmjs.com/package/@archildata/ai-sdk) | | [eve](/integrations/eve) | TypeScript | [`@archildata/eve`](https://www.npmjs.com/package/@archildata/eve) | | Mastra | TypeScript | [`@archildata/mastra`](https://www.npmjs.com/package/@archildata/mastra) | | LangChain.js | TypeScript | [`@archildata/langchain`](https://www.npmjs.com/package/@archildata/langchain) | | [just-bash](/integrations/just-bash) | TypeScript | [`@archildata/just-bash`](https://www.npmjs.com/package/@archildata/just-bash) | | OpenAI Agents SDK | Python | `pip install "archil[openai-agents]"`, then `disk.agent_tools().for_openai_agents()` | | LangChain | Python | `pip install "archil[langchain]"`, then `disk.agent_tools().for_langchain()` | For SQLite databases on a disk from TypeScript, see [`@archildata/sqlite`](/compute/sqlite). ## Other languages There is no SDK for other languages. From any language, you can call the [REST API](/api-reference/introduction) directly, use any S3 client with the [S3-compatible API](/protocols/s3-api), or mount the disk with the [`archil` CLI](/mounting/linux) and use ordinary file I/O. # Python SDK Source: https://docs.archil.com/sdks/python Programmatic access to Archil disks from Python scripts, CI, and notebooks The Archil Python SDK is the [`archil`](https://pypi.org/project/archil/) package on PyPI. It's a pure-Python control-plane client: create disks, list and inspect them, manage who can mount them, run commands against them with [`Disk.exec`](/compute/serverless-execution), read and write their contents through an S3-compatible object API, and manage [persistent sandboxes](#persistent-sandboxes). ```bash theme={null} pip install archil ``` `archil` talks to the Archil control plane over HTTPS and has no native dependencies. It requires Python 3.10 or later. Every method works **both synchronously and asynchronously** from a single implementation: `disk.put_object(...)` blocks, while `disk.put_object.aio(...)` returns a coroutine you can `await`. See [Async](#async) below. > [!NOTE] > > Looking for the FUSE mount client? That's the [`archil` CLI](/mounting/linux), which mounts a disk as a real local filesystem. The Python SDK does not mount disks; it talks to the control plane, runs serverless commands, and reads and writes objects over HTTPS. ## Configuration The recommended pattern is a one-time `configure` call, then the module-level helpers: ```python theme={null} import archil archil.configure(api_key="key-...", region="aws-us-east-1") ``` Both options fall back to environment variables (`ARCHIL_API_KEY`, `ARCHIL_REGION`) if omitted, so in most environments you can skip `configure` entirely and let the SDK read the environment. For multi-tenant scripts that need multiple credentials in one process, instantiate `Archil` directly instead of using the module-level `configure`: ```python theme={null} from archil import Archil prod = Archil(api_key=prod_key, region="aws-us-east-1") staging = Archil(api_key=staging_key, region="aws-us-east-1") prod_disks = prod.disks.list() staging_disks = staging.disks.list() ``` > [!NOTE] > > The API key is an account-level credential and is **not** the same thing as a disk token. API keys authenticate calls to the control plane (everything in this page); a disk token grants mount access to a single disk. See the [disk users concept page](/concepts/authorization#token-based-disk-authorization). ### Regions `region` (or `ARCHIL_REGION`) must be one of these Archil region identifiers: | Region | `region` value | | - | - | | AWS US East (N. Virginia) | `aws-us-east-1` | | AWS US West (Oregon) | `aws-us-west-2` | | AWS EU West (Ireland) | `aws-eu-west-1` | | GCP US Central (Iowa) | `gcp-us-central1` | You must use one of these identifiers and not a region for an individual cloud like `us-east-1`. ## Managing disks ```python theme={null} import archil # Create a disk. The returned disk is available for use immediately. `token` # here is the disk token — the one-time credential for mounting. Save it; it # isn't retrievable again. result = archil.create_disk(name="my-disk") disk = result.disk print(f"Created {disk.id}, disk token: {result.token}") # List and look up disks all_disks = archil.list_disks() d = archil.get_disk(disk.id) ``` Per-disk operations are methods on the `Disk` object itself, not top-level functions: ```python theme={null} from archil import TokenUser d = archil.get_disk("dsk-abc123") # Add an additional mount token — save user.token; it's only returned once user = d.add_user(TokenUser(nickname="ci")) # Revoke access d.remove_user("token", user.identifier) # Create keys for the S3 API — keep them server-side; the secret is only returned once s3 = d.create_s3_credentials(nickname="uploads", ttl="24h") d.remove_s3_credentials(s3.s3_access_key_id) # Delete the disk (this does not delete data in your bucket) d.delete() ``` ## Executing commands `Disk.exec(command)` runs a shell command inside a Firecracker microVM with the file system already mounted, and returns stdout, stderr, exit code, and timing. See the [Serverless Execution concept page](/compute/serverless-execution) for the full picture. ```python theme={null} d = archil.get_disk("dsk-abc123") res = d.exec("grep -r ERROR logs") print(res.stdout, res.stderr, res.exit_code) print(f"ran in {res.timing.execute_ms}ms (queued {res.timing.queue_ms}ms)") ``` The disk is the working directory inside the microVM, so commands can reference files using relative paths. The client's default request timeout of 30 seconds also applies to `exec`, so a command that runs longer than that raises a timeout error. Commands can run for up to [5 minutes](/compute/serverless-execution); for longer commands, create the client with a larger timeout, such as `Archil(timeout=300)`. Each invocation counts as one serverless tool call, with execution included. `execute_ms` reports runtime, not billable usage. See [Pricing & Billing](/administration/billing#serverless-tool-calls). The response carries the first 4 MiB of stdout and the first 4 MiB of stderr. Write larger outputs to a file on the disk instead. See [Limits](/compute/serverless-execution#limits). Pass keyword arguments to choose the microVM's image, size, and network policy: `base_image` or `image_id`, `vcpu_count`, `mem_size_mib`, and `network`. These need `archil` 0.21.0 or later. See [Choosing an image](/compute/serverless-execution#choosing-an-image) and [Choosing a size](/compute/serverless-execution#choosing-a-size). ```python theme={null} res = d.exec("python3 train.py", base_image="python:3.13", vcpu_count=4, mem_size_mib=8192) ``` For multi-disk execs (mount several disks at once, optionally pinned to a subdirectory or read-only), call `archil.exec(...)` instead of `Disk.exec`; it takes the same keyword arguments. Each disk is mounted at its own relative path; pass a `Disk`, a disk-id string, or an `ExecMountSpec` for finer control: ```python theme={null} from archil import ExecMountSpec res = archil.exec( disks={ "data": "dsk-abc123", # mount root, read-write "ref": ExecMountSpec("dsk-def456", read_only=True), # read-only "logs": ExecMountSpec("dsk-ghi789", subdirectory="2026/01"), }, command="grep -r ERROR data ref logs", ) ``` See the [quickstart](/getting-started/quickstart) for an end-to-end example wiring `exec` into an AI agent loop. ## Searching files `Disk.grep(...)` searches the files on a disk for lines matching a regular expression. It fans the listing and matching out across many ephemeral microVMs so the search scales across many machines instead of one. Reach for it instead of `exec("grep …")` whenever you just want matching lines. See [Search Files](/compute/search-files) for the full model. ```python theme={null} d = archil.get_disk("dsk-abc123") res = d.grep(directory="logs", pattern="ERROR|FATAL", recursive=True) for m in res.matches: print(f"{m.file}:{m.line}: {m.text}") print(f"{len(res.matches)} matches in {res.duration_ms}ms ({res.stopped_reason})") ``` You control search latency and scope with three knobs: * `max_duration_seconds` — wall-clock deadline (default 30, capped at 30). * `concurrency` — max parallel workers (default 50). More workers can scan a large dataset faster; the request still counts as one call. * `max_results` — short-circuit once this many matches are collected (default 1000). These options follow ripgrep's flags. See [Search files](/compute/search-files#filtering-files) for examples: * `ignore_case` — match regardless of case (`rg -i`). * `multiline` — let a match span lines (`rg -U`); such a match sets `end_line`. * `globs` — include paths matching ripgrep `-g` globs relative to `directory`, or exclude them with a leading `!`. * `types` / `types_not` — search only, or skip, ripgrep file types such as `py` or `ts`. * `context_before` / `context_after` — 0 to 10 lines of context around each match (`rg -B` / `rg -A`); the result then includes `groups`, each with a `file` and `lines` whose `match` is `False` for context lines. Always check `stopped_reason` — it tells you whether the search was exhaustive. When it stops early, the returned `matches` are a sample of whichever workers reported first, **not** the lexicographically first N: | `stopped_reason` | Meaning | | - | - | | `completed` | Every file under the directory was scanned successfully. The matches are exhaustive. | | `max_results` | Stopped after collecting `max_results` matches, or 32 MiB of match and context text, before scanning everything. | | `deadline` | Hit `max_duration_seconds` before scanning everything. | | `incomplete` | The pipeline finished but one or more batches errored (invalid regex, unreadable file) or exceeded a worker's 4 MiB output limit. Results may be partial. | | `list_failed` | Directory listing failed; only partial results, if any, are present. | Each `grep` request counts as one serverless tool call, regardless of worker count or runtime. See [Pricing & Billing](/administration/billing#serverless-tool-calls). ## Persistent sandboxes `archil.create_sandbox()` creates a persistent Linux VM. See [Persistent sandboxes](/compute/sandboxes/introduction) for the lifecycle, timeouts, networking, and limits. The examples below use `archil` 0.16.0 or later. ```python theme={null} import archil sandbox = archil.create_sandbox( name="agent-workspace", base_image="python:3.13", vcpu_count=2, mem_size_mib=4096, max_ttl_seconds=3600, idle_ttl_seconds=300, ) result = sandbox.exec("python --version && pwd") print(result.stdout) sandbox = sandbox.pause() sandbox = sandbox.resume() fork = sandbox.fork(name="agent-task") sandbox.set_timeout(7200) sandbox.set_timeout(idle_ttl_seconds=0) ``` `start`, `stop`, `pause`, `resume`, and `refresh` return a new `Sandbox` with the updated state rather than modifying the one you call them on. They wait for completion by default; pass `wait=False` to return once the operation is accepted. `set_timeout(timeout, idle_ttl_seconds=...)` changes either timeout, and omitted settings stay unchanged. The idle timeout defaults to 30 minutes, and `idle_ttl_seconds=0` disables it. A session cannot exceed 24 hours even after a timeout reset; starting or resuming begins a new session. `fork` pauses a running sandbox while the snapshot is taken and resumes it once the fork is accepted. `run()` starts a process and returns its handle; `exec()` waits for the command result. Pass an absolute `cwd` to either method to choose a working directory. `attach()` reconnects by process ID and output cursor. The older `processes.start()` and `processes.connect()` methods are deprecated compatibility aliases. ```python theme={null} process = sandbox.run("sleep 600", timeout_seconds=600) process_id, cursor = process.id, process.cursor process.disconnect() resumed = sandbox.attach(process_id, offset=cursor) resumed.kill() ``` Expose ports, create port tokens, change the network policy, and transfer files with the same methods as the TypeScript SDK, in snake case: ```python theme={null} hostname = sandbox.expose_port(8080) access = sandbox.create_port_token(3000, ttl="1h") # send access.token in X-Archil-Token sandbox.update_network( archil.SandboxNetwork(egress=archil.SandboxEgressPolicy(default="deny", allow=["api.github.com"])) ) sandbox.files.upload_file("./input.tar.gz", "/workspace/input.tar.gz") sandbox.files.download_file("/workspace/result.json", "./result.json") ``` ### Mounting existing disks ```python theme={null} workspace = archil.create_sandbox( mounts=[archil.SandboxMountSpec(disk="dsk-0123456789abcdef", path="/mnt/archil")] ) print(workspace.exec("ls /mnt/archil").stdout) workspace = workspace.stop() workspace = workspace.start(mounts=[]) ``` `SandboxMountSpec` accepts `disk` (a disk ID or `Disk` object), `path`, `subdirectory`, `read_only`, `conditional`, and `queue_ms`. With a single mount, `path` is optional and defaults to `/mnt/archil`. Omitting `mounts` on `start()` keeps saved mounts; `[]` clears them and a list replaces them. See [Persistent sandboxes](/compute/sandboxes/introduction#attaching-an-archil-disk) for sharing and forking behavior. ### Private images ```python theme={null} import os with archil.Archil() as client: image = client.images.build( source="ghcr.io/acme/agent-runtime:v2", registry_auth=archil.RegistryAuth(username="bot", password=os.environ["REGISTRY_TOKEN"]), ) agent = client.sandboxes.create(image_id=image.image_id) ``` `images.build()` waits until the image is ready, retrying interrupted builds up to three times. The default timeout is 30 minutes; pass `timeout=600` to change it. Failed builds raise `ImageBuildError`; timeouts raise `TimeoutError`. Use `images.get(image_id)` to read the current build state. Pass `image_id` instead of `base_image` when creating a sandbox from a built image. See [Sandbox images](/compute/sandboxes/images) for reusing builds and updating tags. ### Waiting for capacity Sandbox create, start, resume, and fork retry capacity responses (`503` with `no_capacity` or `runtime_retryable`, and `429`) for up to 10 seconds by default, honoring `Retry-After`. Configure `Archil(max_throttle_wait=seconds)` or `ARCHIL_MAX_THROTTLE_WAIT` to change that budget; `0` fails immediately and `float("inf")` waits indefinitely. Forking a running sandbox does not wait for capacity, so its source can be resumed promptly. ## Reading and writing objects A `Disk` doubles as an S3-compatible bucket: read, write, delete, and list its files by key without mounting it. These methods talk to Archil's S3 endpoint using your same API key — no separate S3 credentials or SigV4 signing on your part. ```python theme={null} import json d = archil.get_disk("dsk-abc123") report = {"generated": "2026-01", "rows": 1234} # Write — accepts str or bytes. content_type is optional. Returns the etag. result = d.put_object("reports/2026-01/data.json", json.dumps(report), "application/json") # Read — returns bytes. data = d.get_object("reports/2026-01/data.json") text = data.decode("utf-8") # Metadata / existence without downloading the body meta = d.head_object("reports/2026-01/data.json") # None if absent if d.object_exists("reports/2026-01/data.json"): ... # Delete (idempotent — deleting a missing key succeeds) d.delete_object("reports/2026-01/data.json") ``` `put_object` handles any size with one call. Small bodies go through a single request, and larger bodies are uploaded as a multipart upload automatically: split into parts, uploaded with bounded concurrency, and assembled. If any part fails, the upload is aborted so nothing is left half-staged. Tune the switch point and parallelism with keyword options: ```python theme={null} result = d.put_object( "backups/2026-01.tar", big_bytes, content_type="application/x-tar", multipart_threshold=5 * 1024 * 1024, # switch to multipart above 5 MiB; default = part_size part_size=32 * 1024 * 1024, # >= 5 MiB; default 16 MiB concurrency=8, # parts in flight at once; default 4 ) ``` For very large objects the part size grows automatically so the upload never exceeds S3's 10,000-part limit. `list_objects` auto-paginates by default, returning every matching key. The first argument is a key prefix; a non-recursive listing (the default) returns the immediate level as `objects` plus subdirectory `common_prefixes`: ```python theme={null} result = d.list_objects("reports/") # one level all_keys = d.list_objects("reports/", recursive=True) # whole subtree first_100 = d.list_objects("reports/", limit=100) # cap the total # Stream pages instead of buffering everything (large listings): for page in d.list_objects_pages("reports/"): for obj in page.objects: print(obj.key, obj.size, obj.last_modified) # Or drive pagination yourself: page = d.list_objects("reports/", single_page=True) if page.is_truncated: nxt = d.list_objects("reports/", single_page=True, continuation_token=page.next_continuation_token) ``` `delete_objects` removes many keys in one round trip (auto-batched at S3's 1,000-key limit). Unlike `delete_object`, per-key failures are returned rather than raised: ```python theme={null} result = d.delete_objects(["a.txt", "logs/b.txt", "c.txt"]) for e in result.errors: print(f"{e.key}: {e.code} {e.message}") ``` `append_object` appends bytes to an existing object (creating it if absent) — handy for log-style writes. Each call may append at most 1 MiB; append in chunks to grow past that: ```python theme={null} d.append_object("logs/app.log", b"first line\n") d.append_object("logs/app.log", b"second line\n") # concatenated ``` For manual control over the multipart lifecycle (e.g. uploading parts from separate processes), the raw S3 primitives live in the opt-in `d.multipart` namespace — `create`, `upload_part`, `complete`, `abort`, `list_parts`, `list_uploads`. Most code never needs these; prefer `put_object`, which runs the lifecycle for you. ```python theme={null} upload = d.multipart.create("big.bin") p1 = d.multipart.upload_part("big.bin", upload.upload_id, 1, first_chunk) p2 = d.multipart.upload_part("big.bin", upload.upload_id, 2, second_chunk) d.multipart.complete("big.bin", upload.upload_id, [p1, p2]) ``` ## Share URLs `share` generates a signed, time-limited URL that lets anyone download a file from the disk without credentials: ```python theme={null} share = d.share("reports/2026-08/summary.csv") print(share.url) # https://control.…/api/shared/ print(share.expires_in) # 86400 (seconds) # Custom expiry — any value from 1 to 604800 (7 days) share = d.share("reports/2026-08/summary.csv", expires_in=3600) ``` The returned URL supports HTTP `Range` requests and streams with a sensible `Content-Type`. After expiry, requests to the URL return `401`. ## Async Every method on `Archil`, `Disks`, `Disk`, `Tokens`, `Sandboxes`, `Sandbox`, and sandbox processes and files has an `.aio` variant that returns a coroutine. The module-level helpers (`configure`, `create_disk`, `get_disk`, etc.) are synchronous convenience wrappers, so from async code, construct `Archil(...)` directly and use `.aio`: ```python theme={null} import asyncio from archil import Archil async def main(): async with Archil(api_key="key-...", region="aws-us-east-1") as client: d = await client.disks.get.aio("dsk-abc123") await d.put_object.aio("a/b.txt", b"hello") data = await d.get_object.aio("a/b.txt") async for page in d.list_objects_pages.aio("a/"): for obj in page.objects: print(obj.key) asyncio.run(main()) ``` ## Managing API keys API keys are account-level, so these helpers live at the top level rather than on a `Disk`: ```python theme={null} archil.list_api_keys() k = archil.create_api_key(name="ci-bot", description="GitHub Actions") # save k.token — it's only returned once archil.delete_api_key("key-abc123") ``` ## Error handling All SDK errors extend `ArchilError`, so `except ArchilError` handles control-plane and S3 failures uniformly. Object-API failures raise `ArchilS3Error` with `status` (HTTP status), `code` (the S3 error code, e.g. `"NoSuchKey"`), `request_id`, and the raw body on `raw`: ```python theme={null} from archil import ArchilError, ArchilS3Error try: data = d.get_object("reports/missing.json") except ArchilS3Error as e: print(e.status, e.code, e.request_id) ``` `get_object` on a missing key raises a 404 — use `head_object` / `object_exists` to probe without catching. Transient failures (HTTP 429 and 5xx, plus network errors) are retried automatically with jittered exponential backoff before surfacing; other 4xx are caller errors and aren't retried. The two non-idempotent operations (`complete` for multipart and `append_object`) are not auto-retried, since a retry after a succeeded-but-unacknowledged call would return a spurious `NoSuchUpload` or duplicate the appended bytes. # TypeScript SDK Source: https://docs.archil.com/sdks/typescript Programmatic access to Archil disks from TypeScript and Node.js applications The Archil TypeScript SDK is the [`disk`](https://www.npmjs.com/package/disk) package on npm. It's a pure-JavaScript control-plane client and CLI: create disks, list and inspect them, manage who can mount them, and run commands against them with [`disk.exec`](/compute/serverless-execution). ```bash theme={null} npm install disk ``` > [!NOTE] > > The npm package is named `disk`. The `archil` package on npm is an alias that installs and re-exports `disk`. `disk` has no native dependencies and requires Node.js 22.19 or later. For the rare case where you need raw protocol access from Node (inodes, delegations, byte-level reads), there's a separate [`@archildata/native`](#low-level-protocol-access) package — most users will not need it. | Package | When to use | | - | - | | [`disk`](https://www.npmjs.com/package/disk) | Manage disks, run commands with `disk.exec`, manage API keys. The default. | | [`@archildata/native`](https://www.npmjs.com/package/@archildata/native) | Speak the Archil filesystem protocol directly from Node. Linux/macOS only. Not recommended unless you're building a custom client runtime. | > [!NOTE] > > Looking for the FUSE mount client? That's the [`archil` CLI](/mounting/linux), which mounts a disk as a real local filesystem. The TypeScript SDK does not mount disks; it talks to the control plane and runs serverless commands. ## Configuration The recommended pattern is a module-namespace import with a one-time `configure` call: ```typescript theme={null} import * as archil from "disk"; archil.configure({ apiKey: process.env.ARCHIL_API_KEY, region: "aws-us-east-1", }); ``` Both options fall back to environment variables (`ARCHIL_API_KEY`, `ARCHIL_REGION`) if omitted, so in most environments `configure({})` — or skipping `configure` entirely — is enough. For multi-tenant scripts that need multiple credentials in one process, instantiate `Archil` directly instead: ```typescript theme={null} import { Archil } from "disk"; const prod = new Archil({ apiKey: prodKey, region: "aws-us-east-1" }); const staging = new Archil({ apiKey: stagingKey, region: "aws-us-east-1" }); ``` For endpoints signed by a private certificate authority, pass `tls.ca` (a PEM string, `Buffer`, or array of either) to `new Archil({ ..., tls: { ca } })`. Like Node's TLS `ca` option, it replaces the default trust list, so include `getCACertificates()` from `node:tls` if you still need the public CAs. > [!NOTE] > > The API key is an account-level credential and is **not** the same thing as a disk token. API keys authenticate calls to the control plane (everything in this page); a disk token grants mount access to a single disk. See the [disk users concept page](/concepts/authorization#token-based-disk-authorization). ### Regions `region` (or `ARCHIL_REGION`) must be one of these Archil region identifiers: | Region | `region` value | | - | - | | AWS US East (N. Virginia) | `aws-us-east-1` | | AWS US West (Oregon) | `aws-us-west-2` | | AWS EU West (Ireland) | `aws-eu-west-1` | | GCP US Central (Iowa) | `gcp-us-central1` | You must use one of these identifiers and not a region for an individual cloud like `us-east-1`. ## Managing disks ```typescript theme={null} import * as archil from "disk"; // Create a disk. It uses Archil-managed storage by default. The returned token // is the disk token — save it; it isn't retrievable again. const { disk, token } = await archil.createDisk({ name: "my-disk" }); // Optional: sync the disk with your own bucket (BYOB) instead const { disk: byobDisk } = await archil.createDisk({ name: "my-byob-disk", mounts: [{ type: "s3", bucketName: "my-bucket", accessKeyId: "AKIA...", secretAccessKey: "...", }], }); // List and look up disks const all = await archil.listDisks(); const d = await archil.getDisk(disk.id); ``` Per-disk operations are methods on the `Disk` object itself, not top-level functions: ```typescript theme={null} const d = await archil.getDisk("dsk-abc123"); // Add an additional mount token const user = await d.createToken("ci"); // save user.token — it's only returned once // Revoke access await d.removeUser("token", user.identifier); // Create keys for the S3 API — keep them server-side; the secret is only returned once const s3 = await d.createS3Credentials({ nickname: "uploads", ttl: "24h" }); await d.removeS3Credentials(s3.s3AccessKeyId); // Delete the disk (this does not delete data in your bucket) await d.delete(); ``` ## Executing commands `Disk.exec(command)` runs a bash command inside a Firecracker microVM with the file system already mounted, and returns stdout, stderr, exit code, and timing. See the [Serverless Execution concept page](/compute/serverless-execution) for the full picture. ```typescript theme={null} const d = await archil.getDisk("dsk-abc123"); const { stdout, stderr, exitCode, timing } = await d.exec( "du -sh logs", ); console.log(`ran in ${timing.executeMs}ms (queued ${timing.queueMs}ms)`); ``` The disk is the working directory inside the microVM, so commands can reference files using relative paths. Pass options as the second argument to choose the microVM's image, size, and network policy: `baseImage` or `imageId`, `vcpuCount`, `memSizeMib`, and `network`. These need `disk` 1.12.0 or later. See [Choosing an image](/compute/serverless-execution#choosing-an-image) and [Choosing a size](/compute/serverless-execution#choosing-a-size). ```typescript theme={null} await d.exec("python3 train.py", { baseImage: "python:3.13", vcpuCount: 4, memSizeMib: 8192, }); ``` For multi-disk execs (mount several disks at once, optionally pinned to a subdirectory or read-only), call `archil.exec({ disks, command })` instead of `Disk.exec`; it takes the same options. See [Mounting multiple disks in one command](/compute/serverless-execution#composing-multiple-disks-into-a-single-command). ### `ExecResult` ```typescript theme={null} interface ExecResult { exitCode: number; stdout: string; stderr: string; timing: ExecTiming; } interface ExecTiming { /** End-to-end wall-clock measured on the server. */ totalMs: number; /** Time spent queueing, scheduling, booting a microVM, and mounting the filesystem. */ queueMs: number; /** Time the user's command itself ran, measured by the runtime. */ executeMs: number; } ``` Each invocation counts as one serverless tool call, with execution included. `executeMs` reports runtime, not billable usage. See [Pricing & Billing](/administration/billing#serverless-tool-calls). The HTTP response returns after 5 minutes, and the response carries the first 4 MiB of stdout and the first 4 MiB of stderr. Write larger outputs to a file on the disk instead. See [Limits](/compute/serverless-execution#limits). ### Fan-out Because each `exec` runs in its own microVM, `Promise.all` is a map-reduce: ```typescript theme={null} const files = (await d.exec("ls logs")).stdout.trim().split("\n"); const results = await Promise.all( files.map((f) => d.exec(`wc -l logs/${f}`)), ); ``` For the common case of *searching* files for a pattern, don't hand-roll this — use [`Disk.grep`](#searching-files), which fans the same work out for you and returns structured matches. See the [quickstart](/getting-started/quickstart) for an end-to-end example wiring `disk.exec` into an AI agent loop. ## Searching files `Disk.grep(opts)` searches the files on a disk for lines matching a regular expression. It fans the listing and matching out across many ephemeral microVMs so the search scales across many machines instead of one. It's the productized version of the fan-out above. Reach for it instead of `Promise.all` over `exec("grep …")` whenever you just want matching lines. See [Search Files](/compute/search-files) for the full model. ```typescript theme={null} const d = await archil.getDisk("dsk-abc123"); const { matches, stoppedReason, durationMs } = await d.grep({ directory: "logs", pattern: "ERROR|FATAL", recursive: true, }); for (const m of matches) { console.log(`${m.file}:${m.line}: ${m.text}`); } console.log(`${matches.length} matches in ${durationMs}ms (${stoppedReason})`); ``` You control search latency and scope with three knobs: * `maxDurationSeconds` — wall-clock deadline (default 30, capped at 30). * `concurrency` — max parallel workers (default 50). More workers can scan a large dataset faster; the request still counts as one call. * `maxResults` — short-circuit once this many matches are collected (default 1000). These options follow ripgrep's flags (`disk` 1.10.0 or later). See [Search files](/compute/search-files#filtering-files) for examples: * `ignoreCase` — match regardless of case (`rg -i`). * `multiline` — let a match span lines (`rg -U`); such a match sets `endLine`. * `globs` — include paths matching ripgrep `-g` globs relative to `directory`, or exclude them with a leading `!`. * `types` / `typesNot` — search only, or skip, ripgrep file types such as `py` or `ts`. * `contextBefore` / `contextAfter` — 0 to 10 lines of context around each match (`rg -B` / `rg -A`); the result then includes `groups`. Always check `stoppedReason` — it tells you whether the search was exhaustive. When it stops early, the returned `matches` are a sample of whichever workers reported first, **not** the lexicographically first N: | `stoppedReason` | Meaning | | - | - | | `completed` | Every file under the directory was scanned successfully. The matches are exhaustive. | | `max_results` | Stopped after collecting `maxResults` matches, or 32 MiB of match and context text, before scanning everything. | | `deadline` | Hit `maxDurationSeconds` before scanning everything. | | `incomplete` | The pipeline finished but one or more batches errored (invalid regex, unreadable file) or exceeded a worker's 4 MiB output limit. Results may be partial. | | `list_failed` | Directory listing failed; only partial results, if any, are present. | ### `GrepResult` ```typescript theme={null} interface GrepResult { matches: GrepMatch[]; groups?: GrepGroup[]; // only with contextBefore or contextAfter stoppedReason: GrepStoppedReason; // "completed" | "incomplete" | "max_results" | "deadline" | "list_failed" filesScanned: number; containersDispatched: number; computeSecondsUsed: number; // summed worker runtime; not a billing unit durationMs: number; // end-to-end wall clock listingMs: number; // overlaps grepMs, so the two can sum past durationMs grepMs: number; } interface GrepMatch { file: string; // relative to the disk root line: number; // 1-based; a multiline match's first line endLine?: number; // a multiline match's last line text: string; // the matching line, or every line a multiline match covers } interface GrepGroup { file: string; lines: { line: number; text: string; match: boolean }[]; // match: false for context lines } ``` Each `grep` request counts as one serverless tool call, regardless of worker count or runtime. See [Pricing & Billing](/administration/billing#serverless-tool-calls). ## Persistent sandboxes `Archil.sandboxes` manages persistent Linux VMs. Each sandbox has a dedicated disk, accepts multiple commands over its lifetime, and can preserve CPU and memory state across pause and resume. See [Persistent sandboxes](/compute/sandboxes/introduction) for the lifecycle, interactive shells, services, forks, and limits. ```typescript theme={null} import { Archil } from "disk"; const client = new Archil({ apiKey: process.env.ARCHIL_API_KEY, region: "aws-us-east-1", }); const sandbox = await client.sandboxes.create({ name: "prepared-environment", baseImage: "node:24-bookworm", vcpuCount: 4, memSizeMiB: 8192, env: { NODE_ENV: "development" }, maxTtlSeconds: 14_400, maxConcurrentExecs: 32, }); const execution = await sandbox.run("node --version"); const result = await execution.wait(); console.log(result.stdout); await sandbox.pause(); await sandbox.resume(); await sandbox.stop(); await sandbox.start(); const fork = await sandbox.fork({ name: "agent-task" }); await fork.stop(); await fork.delete(); ``` `maxTtlSeconds` sets the hard timeout from 60 seconds to 24 hours (the default). `idleTtlSeconds` sets the idle timeout from 0 to 24 hours. It defaults to 30 minutes, and 0 disables it. Both pause the VM, preserving memory and processes for `resume()`. Use `sandbox.setTimeout()` to update either setting; a session cannot exceed 24 hours even after a timeout reset. See [Setting a time to live](/compute/sandboxes/introduction#setting-a-time-to-live) for examples and reset behavior. `create`, `pause`, `resume`, `stop`, `start`, and `fork` wait for completion by default. The server handles the initial wait; if its request budget expires first, the SDK continues polling. Pass `{ wait: false }` to return as soon as the operation is accepted. A waited `pause()` throws `SandboxPauseError` if the sandbox does not reach `paused`. `fork()` pauses a running sandbox while the snapshot is taken and resumes it once the fork is accepted. ```typescript theme={null} const running = await sandbox.run("sleep 600", { timeoutSeconds: 600, }); const processId = running.id; const cursor = running.cursor; await running.disconnect(); const resumed = await sandbox.attach(processId, { offset: cursor }); await resumed.kill(); ``` Use `client.sandboxes.list()` and `.get(id)` to retrieve sandboxes. A `Sandbox` provides `refresh`, `fork`, `delete`, `pause`, `resume`, `stop`, `start`, and `setTimeout`; `exposePort`, `listPorts`, `unexposePort`, and port-token methods for [network services](/compute/sandboxes/introduction#hosting-network-services); `getNetwork` and `updateNetwork` for the [firewall](/compute/sandboxes/firewall); plus `run` and `attach` for starting and reattaching to sandbox processes. `exec` runs a command and waits for its result. Pass an absolute `cwd` to `run` or `exec` to choose a working directory. The older `processes.start` and `processes.connect` methods are deprecated compatibility aliases. For an interactive command, provide terminal dimensions. The returned `SandboxProcess` accepts input and terminal resizes while streaming output to `onOutput`: ```typescript theme={null} const shell = await sandbox.run("/bin/sh", { terminal: { cols: 120, rows: 40 }, onOutput: ({ data }) => process.stdout.write(data), }); await shell.sendInput("pwd\n"); await shell.resize({ cols: 160, rows: 50 }); const { exitCode } = await shell.wait(); ``` ### Mounting existing disks Pass `mounts` when creating a sandbox to attach existing disks in addition to its internal root disk: ```typescript theme={null} const workspace = await client.sandboxes.create({ mounts: [{ disk: "dsk-0123456789abcdef", path: "/mnt/archil" }], }); console.log((await workspace.exec("ls /mnt/archil")).stdout); await workspace.stop(); await workspace.start({ mounts: [] }); ``` Each mount accepts `disk` (a disk ID or `Disk` object), `path`, `subdirectory`, `readOnly`, `conditional`, and `queueMs`. With a single mount, `path` is optional and defaults to `/mnt/archil`. Omitting `mounts` on `start()` keeps saved mounts; `[]` clears them and a list replaces them. See [Persistent sandboxes](/compute/sandboxes/introduction#attaching-an-archil-disk) for sharing and forking behavior. ### Private images ```typescript theme={null} const image = await client.images.build({ source: "ghcr.io/acme/agent-runtime:v2", registryAuth: { username: "bot", password: process.env.REGISTRY_TOKEN! }, }); const agent = await client.sandboxes.create({ imageId: image.imageId }); ``` `images.build()` waits until the image is ready, retrying interrupted builds up to three times. The default timeout is 30 minutes; set it with `images.build(request, { timeoutSeconds: 600 })`. Failed builds raise `ImageBuildError`; timeouts raise `ArchilError` with code `IMAGE_BUILD_TIMEOUT`. Use `images.get(imageId)` to read the current build state. Pass `imageId` instead of `baseImage` when creating a sandbox from a built image. See [Sandbox images](/compute/sandboxes/images) for reusing builds and updating tags. ## Reading and writing objects A `Disk` doubles as an S3-compatible bucket: read, write, delete, and list its files by key without mounting it. These methods talk to Archil's S3 endpoint using your same API key, with no separate S3 credentials or SigV4 signing on your part. ```typescript theme={null} const d = await archil.getDisk("dsk-abc123"); const report = { generated: "2026-01", rows: 1234 }; // Write — body is a string, Uint8Array/Buffer, or ArrayBuffer. contentType is // optional (defaults to application/octet-stream). Returns the etag. const { etag } = await d.putObject( "reports/2026-01/data.json", JSON.stringify(report), "application/json", ); // Read — returns a Uint8Array. const bytes = await d.getObject("reports/2026-01/data.json"); const text = new TextDecoder().decode(bytes); // Metadata / existence without downloading the body const meta = await d.headObject("reports/2026-01/data.json"); // null if absent if (await d.objectExists("reports/2026-01/data.json")) { // ... } // Delete (idempotent — deleting a missing key resolves successfully) await d.deleteObject("reports/2026-01/data.json"); ``` `putObject` handles any size with one call. Small bodies go through a single request, and larger bodies are uploaded as a multipart upload automatically: split into parts, uploaded with bounded concurrency, and assembled. If any part fails, the upload is aborted so nothing is left half-staged. You don't pick a different method for big files. Tune the switch point and parallelism with options: ```typescript theme={null} const { etag } = await d.putObject("backups/2026-01.tar", bigBytes, { contentType: "application/x-tar", multipartThreshold: 5 * 1024 * 1024, // switch to multipart above 5 MiB; default = partSize partSize: 32 * 1024 * 1024, // ≥ 5 MiB; default 16 MiB concurrency: 8, // parts in flight at once; default 4 }); ``` For very large objects the part size grows automatically so the upload never exceeds S3's 10,000-part limit. `listObjects` auto-paginates by default, returning every matching key. The first argument is a key prefix; a non-recursive listing (the default) returns the immediate level as `objects` plus subdirectory `commonPrefixes`: ```typescript theme={null} const oneLevel = await d.listObjects("reports/"); // one level const subtree = await d.listObjects("reports/", { recursive: true }); // whole subtree const first100 = await d.listObjects("reports/", { limit: 100 }); // cap the total // Stream pages instead of buffering everything (large listings): for await (const page of d.listObjectsPages("reports/")) { for (const obj of page.objects) { console.log(obj.key, obj.size, obj.lastModified); } } // Or drive pagination yourself: const page = await d.listObjects("reports/", { singlePage: true }); if (page.isTruncated) { const next = await d.listObjects("reports/", { singlePage: true, continuationToken: page.nextContinuationToken, }); } ``` `deleteObjects` removes many keys in one round trip (auto-batched at S3's 1,000-key limit). Unlike `deleteObject`, per-key failures are returned rather than thrown: ```typescript theme={null} const { deleted, errors } = await d.deleteObjects(["a.txt", "logs/b.txt", "c.txt"]); for (const e of errors) console.warn(`${e.key}: ${e.code} ${e.message}`); ``` `appendObject` appends bytes to an existing object (creating it if absent) — handy for log-style writes. Each call may append at most 1 MiB; append in chunks to grow past that: ```typescript theme={null} await d.appendObject("logs/app.log", "first line\n"); await d.appendObject("logs/app.log", "second line\n"); // concatenated ``` For manual control over the multipart lifecycle (e.g. uploading parts from separate processes), the raw S3 primitives live in the opt-in `d.multipart` namespace — `create`, `uploadPart`, `complete`, `abort`, `listParts`, `listUploads`. Most code never needs these; prefer `putObject`, which runs the lifecycle for you. ```typescript theme={null} const { uploadId } = await d.multipart.create("big.bin"); const p1 = await d.multipart.uploadPart("big.bin", uploadId, 1, firstChunk); const p2 = await d.multipart.uploadPart("big.bin", uploadId, 2, secondChunk); await d.multipart.complete("big.bin", uploadId, [p1, p2]); ``` Transient failures (HTTP 429 and 5xx, plus network errors) are retried automatically with jittered exponential backoff before surfacing; other 4xx are caller errors and aren't retried. The two non-idempotent operations (`completeMultipartUpload` and `appendObject`) are not auto-retried, since a retry after a succeeded-but-unacknowledged call would return a spurious `NoSuchUpload` or duplicate the appended bytes. Object-API failures throw `ArchilS3Error` (a subclass of `ArchilError`) with `status` (HTTP status), `code` (the S3 error code, e.g. `"NoSuchKey"`), `requestId`, and the raw XML body on `raw`. `getObject` on a missing key throws a 404 — use `headObject` / `objectExists` to probe without catching. ```typescript theme={null} import { ArchilS3Error } from "disk"; try { await d.getObject("reports/missing.json"); } catch (e) { if (e instanceof ArchilS3Error) { console.log(e.status, e.code, e.requestId); } } ``` ## Share URLs `share` generates a signed, time-limited URL that lets anyone download a file from the disk without credentials: ```typescript theme={null} const share = await d.share("reports/2026-08/summary.csv"); console.log(share.url); // https://control.…/api/shared/ console.log(share.expiresIn); // 86400 (seconds) // Custom expiry — any value from 1 to 604800 (7 days) const hourShare = await d.share("reports/2026-08/summary.csv", { expiresIn: 3600 }); ``` The returned URL supports HTTP `Range` requests and streams with a sensible `Content-Type`. After expiry, requests to the URL return `401`. ## Managing API keys API keys are account-level, so these helpers live at the top level rather than on a `Disk`: ```typescript theme={null} await archil.listApiKeys(); const k = await archil.createApiKey({ name: "ci-bot", description: "GitHub Actions" }); // save k.token — it's only returned once await archil.deleteApiKey("key-abc123"); ``` ## CLI `disk` ships a CLI under the same name. See the [`disk` CLI reference](/reference/disk-cli) for the full command list. ```bash theme={null} npx disk create my-disk npx disk list npx disk dsk-abc123 exec "ls -la" ``` ## Low-level protocol access For the small number of integrations that need to speak Archil's filesystem protocol directly from Node — inodes, delegations, byte-level reads, paginated directory enumeration — install [`@archildata/native`](https://www.npmjs.com/package/@archildata/native) alongside `disk`: ```bash theme={null} npm install disk @archildata/native ``` Then call `Disk.mount()`, which lazy-loads the native client: ```typescript theme={null} import { getDisk } from "disk"; const d = await getDisk("dsk-abc123"); const client = await d.mount({ authToken: process.env.ARCHIL_DISK_TOKEN }); // `client` is an ArchilClient from @archildata/native — see that package's // README on npm for the full inode-level API. await client.close(); ``` `@archildata/native` ships prebuilt binaries for **Linux** (x64, arm64, glibc) and **macOS** (arm64). On other platforms, `mount()` throws; the rest of `disk` still works. > [!NOTE] > > If you're tempted to reach for `@archildata/native` to "run a bash command on a disk," use [`disk.exec`](#executing-commands) instead. It gives you a real shell with the filesystem mounted in a Firecracker microVM, with no native dependencies on the caller side. ## Need a bash executor inside Node? The legacy [`@archildata/just-bash`](/mounting/just-bash) package implements a JavaScript bash interpreter that talks to a disk through `@archildata/native`. It's still published, but for almost every use case `disk.exec` is the better answer. It runs your command in a real Firecracker microVM with the filesystem mounted, returns stdout/stderr/exit code, and has no native dependencies on the caller.