Firecracker

Firecracker is a lightweight virtualization technology from Amazon that allows you to run Linux-based applications in a secure and isolated environment. Archil can be run inside of a Firecracker microVM, giving these user-created containers an infinite disk that’s synchronized to an S3 bucket. By default, the Linux kernel used by Firecracker is configured without support for FUSE, which is required for Archil. Therefore, you must recompile the kernel with FUSE support.

In AWS, Firecracker can only be run on bare-metal instances. The remainder of this guide assumes that you’re running on one of these instances, such as c6g.metal. This guide assumes that you’ve already installed or compiled the Firecracker binary.

Create an Archil disk

First, follow the Archil Getting Started Guide to create an Archil disk that you wish to attach to your Firecracker microVM.

Compile the kernel

First, download the Kernel source code from kernel.org. The most recent version of the Kernel to be included in the Firecracker CI is 6.1, so use that version. Also clone the Firecracker repository to get the most up to date kernel configuration.

# Download the kernel source code
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.1.tar.xz
tar -xvf linux-6.1.tar.xz

# Download the Firecracker source code for the most recent kernel configuration
git clone https://github.com/firecracker-microvm/firecracker.git

aarch64
x86_64

Move the Kernel configuration you plan to use into the Kernel source code directory.

cp firecracker/resources/guest_configs/microvm-kernel-ci-aarch64-6.1.config linux-6.1/

Next, edit the Kernel configuration to enable FUSE support. Open the configuration file in an editor, identify the line that says CONFIG_FUSE_FS is not set, and change it to CONFIG_FUSE_FS=y. The following command should return CONFIG_FUSE_FS=y.

grep CONFIG_FUSE_FS linux-6.1/microvm-kernel-ci-aarch64-6.1.config

Next, set this configuration as the default for the kernel build, and build the kernel.

cd linux-6.1
cp microvm-kernel-ci-aarch64-6.1.config .config
make -j$(nproc)
cp arch/arm64/boot/Image ../vmlinux-6.1-fuse

Move the Kernel configuration you plan to use into the Kernel source code directory.

cp firecracker/resources/guest_configs/microvm-kernel-ci-x86_64-6.1.config linux-6.1/

grep CONFIG_FUSE_FS linux-6.1/microvm-kernel-ci-x86_64-6.1.config

Next, set this configuration as the default for the kernel build, and build the kernel.

cd linux-6.1
cp microvm-kernel-ci-x86_64-6.1.config .config
make bzImage -j$(nproc)
cp arch/x86/boot/bzImage ../vmlinux-6.1-fuse

Starting the microVM

The remainder of this guide follows the Firecracker Quick Start Guide. Create an SSH key, and download the latest rootfs.

ARCH="$(uname -m)"
release_url="https://github.com/firecracker-microvm/firecracker/releases"
latest_version=$(basename $(curl -fsSLI -o /dev/null -w  %{url_effective} ${release_url}/latest))
CI_VERSION=${latest_version%.*}

latest_ubuntu_key=$(curl "http://spec.ccfc.min.s3.amazonaws.com/?prefix=firecracker-ci/$CI_VERSION/$ARCH/ubuntu-&list-type=2" \
    | grep -oP "(?<=<Key>)(firecracker-ci/$CI_VERSION/$ARCH/ubuntu-[0-9]+\.[0-9]+\.squashfs)(?=</Key>)" \
    | sort -V | tail -1)
ubuntu_version=$(basename $latest_ubuntu_key .squashfs | grep -oE '[0-9]+\.[0-9]+')

# Download a rootfs
wget -O ubuntu-$ubuntu_version.squashfs.upstream "https://s3.amazonaws.com/spec.ccfc.min/$latest_ubuntu_key"

# Create an ssh key for the rootfs
unsquashfs ubuntu-$ubuntu_version.squashfs.upstream
ssh-keygen -f id_rsa -N ""
cp -v id_rsa.pub squashfs-root/root/.ssh/authorized_keys
mv -v id_rsa ./ubuntu-$ubuntu_version.id_rsa
# create ext4 filesystem image
sudo chown -R root:root squashfs-root
truncate -s 400M ubuntu-$ubuntu_version.ext4
sudo mkfs.ext4 -d squashfs-root -F ubuntu-$ubuntu_version.ext4

# Verify everything was correctly set up and print versions
echo "Kernel: $(ls vmlinux-* | tail -1)"
echo "Rootfs: $(ls *.ext4 | tail -1)"
echo "SSH Key: $(ls *.id_rsa | tail -1)"

The Kernel: line should be the kernel you just compiled, vmlinux-6.1-fuse. To install the Archil client, you need additional disk space for the binaries themselves along with SSL root certificates. Expand the size of the rootfs image:

sudo truncate -s 1G ubuntu-$ubuntu_version.ext4
sudo resize2fs ubuntu-$ubuntu_version.ext4

Next, start the Firecracker server with the following commands:

API_SOCKET="/tmp/firecracker.socket"

# Remove API unix socket
sudo rm -f $API_SOCKET

# Run firecracker
sudo ./firecracker --api-sock "${API_SOCKET}"

In a new terminal (keeping the terminal with the server running open), start the microVM with the following commands:

TAP_DEV="tap0"
TAP_IP="172.16.0.1"
MASK_SHORT="/30"

# Setup network interface
sudo ip link del "$TAP_DEV" 2> /dev/null || true
sudo ip tuntap add dev "$TAP_DEV" mode tap
sudo ip addr add "${TAP_IP}${MASK_SHORT}" dev "$TAP_DEV"
sudo ip link set dev "$TAP_DEV" up

# Enable ip forwarding
sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
sudo iptables -P FORWARD ACCEPT

# This tries to determine the name of the host network interface to forward
# VM's outbound network traffic through. If outbound traffic doesn't work,
# double check this returns the correct interface!
HOST_IFACE=$(ip -j route list default |jq -r '.[0].dev')

# Set up microVM internet access
sudo iptables -t nat -D POSTROUTING -o "$HOST_IFACE" -j MASQUERADE || true
sudo iptables -t nat -A POSTROUTING -o "$HOST_IFACE" -j MASQUERADE

API_SOCKET="/tmp/firecracker.socket"
LOGFILE="./firecracker.log"

# Create log file
touch $LOGFILE

# Set log file
sudo curl -X PUT --unix-socket "${API_SOCKET}" \
    --data "{
        \"log_path\": \"${LOGFILE}\",
        \"level\": \"Debug\",
        \"show_level\": true,
        \"show_log_origin\": true
    }" \
    "http://localhost/logger"

KERNEL="./$(ls vmlinux* | tail -1)"
KERNEL_BOOT_ARGS="console=ttyS0 reboot=k panic=1 pci=off"

ARCH=$(uname -m)

if [ ${ARCH} = "aarch64" ]; then
    KERNEL_BOOT_ARGS="keep_bootcon ${KERNEL_BOOT_ARGS}"
fi

# Set boot source
sudo curl -X PUT --unix-socket "${API_SOCKET}" \
    --data "{
        \"kernel_image_path\": \"${KERNEL}\",
        \"boot_args\": \"${KERNEL_BOOT_ARGS}\"
    }" \
    "http://localhost/boot-source"

ROOTFS="./$(ls *.ext4 | tail -1)"

# Set rootfs
sudo curl -X PUT --unix-socket "${API_SOCKET}" \
    --data "{
        \"drive_id\": \"rootfs\",
        \"path_on_host\": \"${ROOTFS}\",
        \"is_root_device\": true,
        \"is_read_only\": false
    }" \
    "http://localhost/drives/rootfs"

# The IP address of a guest is derived from its MAC address with
# `fcnet-setup.sh`, this has been pre-configured in the guest rootfs. It is
# important that `TAP_IP` and `FC_MAC` match this.
FC_MAC="06:00:AC:10:00:02"

# Set network interface
sudo curl -X PUT --unix-socket "${API_SOCKET}" \
    --data "{
        \"iface_id\": \"net1\",
        \"guest_mac\": \"$FC_MAC\",
        \"host_dev_name\": \"$TAP_DEV\"
    }" \
    "http://localhost/network-interfaces/net1"

# Increase machine memory to allow for running apt and Archil
sudo curl -X PUT --unix-socket "${API_SOCKET}" \
    --data "{
        \"vcpu_count\": 1,
        \"mem_size_mib\": 512
    }" \
    "http://localhost/machine-config"

# API requests are handled asynchronously, it is important the configuration is
# set, before `InstanceStart`.
sleep 0.015s

# Start microVM
sudo curl -X PUT --unix-socket "${API_SOCKET}" \
    --data "{
        \"action_type\": \"InstanceStart\"
    }" \
    "http://localhost/actions"

# API requests are handled asynchronously, it is important the microVM has been
# started before we attempt to SSH into it.
sleep 2s

KEY_NAME=./$(ls *.id_rsa | tail -1)

# Setup internet access in the guest
ssh -i $KEY_NAME root@172.16.0.2  "ip route add default via 172.16.0.1 dev eth0"

# Setup DNS resolution in the guest
ssh -i $KEY_NAME root@172.16.0.2  "echo 'nameserver 8.8.8.8' > /etc/resolv.conf"

# SSH into the microVM
ssh -i $KEY_NAME root@172.16.0.2

# Use `root` for both the login and password.
# Run `reboot` to exit.

Installing and running Archil

You should now be SSH’d into the new microVM. You can now install the Archil client inside the microVM. Run the remainder of this guide from the microVM itself. Start by installing the root certificates to allow for SSL connections for downloading the Archil client and connecting to the Archil disk itself.

apt update
apt install -y ca-certificates
update-ca-certificates

Next, install the Archil client.

curl https://archil.com/install | sh

Finally, mount the Archil disk.

mkdir -p /mnt/archil
export ARCHIL_MOUNT_TOKEN="<token>"
archil mount <disk-name> /mnt/archil --region aws-us-east-1

Your microVM now has access to an unlimited disk that’s synchronized to an S3 bucket. Your microVM can instantly access data from the S3 bucket without having to first download it.

Compute

AI

Data Science

Databases

Observability

Virtualization

Create an Archil disk

Compile the kernel

Starting the microVM

Installing and running Archil

​Create an Archil disk

​Compile the kernel

​Starting the microVM

​Installing and running Archil

Create an Archil disk

Compile the kernel

Starting the microVM

Installing and running Archil