Skip to main content
Temporal is a good fit for agent pipelines: it retries failed steps, tracks long-running work, and fans out to as many tasks as you need. The hard part is where the activities run. An agent that works for an hour holds a worker for the hour, so worker pools have to be sized and autoscaled for the longest task, and every deploy has to wait for in-flight work to drain. This guide runs each activity on a Temporal worker inside its own Archil sandbox instead. For each piece of work, the workflow starts a sandbox with the work’s disk mounted. A Temporal worker in that sandbox polls a task queue of its own, and the workflow sends the activity there, so it runs next to its data, with your image’s tools installed. When the activity is done, the workflow deletes the sandbox. The disk is kept, along with everything the work needs to resume. The complete example is on GitHub at archil-data/temporal-archil-agents. Its workflow gives each of a list of URLs to a Claude Agent SDK agent, which downloads the data and normalizes it into Parquet. A final step merges every agent’s output into one file and returns a signed URL that a warehouse such as ClickHouse can load directly.

Prerequisites

  • Python 3.10 or later, and a Temporal server the sandboxes can reach over the internet, such as Temporal Cloud.
  • An Archil API key and a region where sandboxes are available (aws-us-east-1, aws-us-west-2, or aws-eu-west-1).
  • An Anthropic API key.
  • Docker and a container registry Archil can pull from, for the sandbox image.

Start a sandbox for each piece of work

Starting a sandbox is a short activity, start_sandbox, that runs on a control worker. It finds or creates the sandbox for the work, with the work’s disk mounted, and starts a Temporal worker in it as an Archil service. The service restarts the worker if it ever exits. The sandbox and its task queue are named after the work, so a retried start_sandbox finds the same sandbox instead of creating a second one:
archil_temporal/control.py
ensure_service runs archil services create in the sandbox, unless the service already exists. _wait_for_worker returns once the sandbox’s worker shows up as a poller on its queue, so the activity sent there next starts right away. The worker inside the sandbox is an ordinary Temporal worker. It runs your activities, and polls only its sandbox’s queue:
archil_temporal/sandbox_worker.py

Send the work to its sandbox

In the workflow, every piece of work goes through in_sandbox: start the sandbox, run the activity on the sandbox’s queue, and delete the sandbox when the activity has finished, failed, or been cancelled:
archil_temporal/workflows.py
This is simplified: the version in the repository also starts a new sandbox on the same disk if nothing picks the activity up within schedule_to_start_timeout, which means the sandbox was lost. It doesn’t use finally, because a finally block also runs when a control worker shuts down and Python closes the workflow’s coroutines. The workflow isn’t over then: another control worker carries on with it, and the sandbox must stay. Each agent task, and the merge at the end, is one call:
archil_temporal/workflows.py
The control worker only runs the workflow and activities that take seconds. You can deploy it, scale it, or restart it at any time, without draining it, because no agent runs on it. A sandbox keeps the image it was started with until its work is done, while new work gets the new image, so rolling out a new image never interrupts work in flight.

Resume from the disk

Everything the agent needs to resume lives on its disk, mounted at /mnt/archil: its downloads, its output, and Claude Code’s session transcript (CLAUDE_CONFIG_DIR=/mnt/archil/.claude). The session ID is derived from the workflow ID and the task key, so any attempt finds the conversation an earlier one started, and continues it:
archil_temporal/agent.py
The workflow ends with a cleanup step that deletes any sandbox still there. Each sandbox is also created with a max TTL, which pauses it if nothing ever deletes it, such as when the workflow is terminated; a paused sandbox uses no CPU.

Keep the API key out of the sandbox

The agent runs with every permission in its sandbox, and it downloads data from URLs it’s given. A network policy limits what it can reach: Temporal, the Anthropic API, the host of its URL, and nothing else. Archil’s egress proxy adds the Anthropic API key to the agent’s requests on the way out, using credential brokering, so the sandbox only ever sees a placeholder:
archil_temporal/sandboxes.py
Nothing the agent runs or downloads can read the real key or send it anywhere. Network policies need a paid plan; on the Free plan, set RESTRICT_EGRESS=false in .env to pass the key into the sandbox instead.

Merge next to the data

When the agents are done, the merge runs the same way, in one more sandbox. That sandbox mounts every task’s disk read-only, plus one writable output disk. A read-only mount doesn’t take exclusive access to a disk, so this works even while other clients have the task disks open:
archil_temporal/workflows.py
Inside, DuckDB streams every agent’s Parquet files into one, spilling to the Archil disk, so the result can be larger than the sandbox’s memory. Then a control activity creates a signed URL for the file, which anyone can download without Archil credentials:
archil_temporal/control.py
You can load that URL into ClickHouse directly:

Run it

Build the sandbox image, which installs the example’s package with the Claude Agent SDK, pandas, pyarrow, and DuckDB, and register it with Archil once:
Start a control worker, then start a workflow. With no arguments, the starter uses three public CSV datasets:
The repository’s end-to-end test runs real agents this way. While they work, it replaces the control worker, as a deploy would, and restarts the Temporal worker inside one agent’s sandbox. One run printed:

Clean up

Each task’s disk is kept after the workflow finishes, so you can read an agent’s files and transcript afterwards. To delete a workflow’s disks, and any sandboxes left by a workflow that was terminated rather than cancelled:

Next steps

Claude Code

Run Claude Code headlessly in a sandbox, with its sessions on a disk.

Parallel agents

Fork one prepared sandbox per agent and keep the best result.

Sandbox firewall

Egress policies, credential brokering, and forwarding.

Persistent sandboxes

Lifecycle, services, time to live, and forking in detail.