# Flatcar installation

[Flatcar](https://www.flatcar.org/) allows you to create Kubernetes clusters with ease based on immutable Linux nodes.

With i3D.net FlexMetal you can boot Flatcar nodes using the [Custom iPXE](/compute/flexmetal/custom-ipxe-booting.md) feature. On this page we will describe the process.

## Create your custom iPXE script

To boot the Flatcar OS, you must use the Custom iPXE feature when requesting a FlexMetal server. Below you will find how to perform an API request to do that.

Inside the iPXE script, you must point to the version of Flatcar you want to boot, while also providing the URL to your ignition configuration file. This URL should point to the [Metadata API userdata endpoint](/compute/flexmetal/metadata-api.md#userdata), so that the ignition configuration is fetched from your server's userdata during boot.

## Example iPXE script

This is an example iPXE script to boot the latest LTS version of the Flatcar OS, using [Custom iPXE network variables](/compute/flexmetal/custom-ipxe-booting.md#ipxe-network-variables) to configure the network for the initial boot phase. Note the `ignition.config.url` kernel parameter that points to the [userdata endpoint](/compute/flexmetal/metadata-api.md#userdata), which will serve the ignition configuration you uploaded as userdata when creating the server. See how to set userdata to your configuration file in the [example request](#example-flexmetal-api-request) section below.

{% code overflow="wrap" %}

```
#!ipxe

set bootimage_url https://lts.release.flatcar-linux.net/amd64-usr/current
set os_parameters initrd=flatcar_production_pxe_image.cpio.gz flatcar.first_boot=1 ignition.config.url=https://metadata.i3d.net/v1/userdata flatcar.autologin net.ifnames=0 hostname=${IPXE_BOOT_HOSTNAME} ${IPXE_BOOT_LINUX_IP_CONFIG}

kernel ${bootimage_url}/flatcar_production_pxe.vmlinuz ${os_parameters}
initrd ${bootimage_url}/flatcar_production_pxe_image.cpio.gz

boot
```

{% endcode %}

> \[!NOTE] You can also set the ignition config url to download the config file from your own server if you prefer. For example: `ignition.config.url=https://my-server.com/ignition-configuration.ign`

More details can be found on [Flatcar's iPXE boot documentation](https://www.flatcar.org/docs/latest/installing/bare-metal/booting-with-ipxe/).

## Example FlexMetal API request

To [request a FlexMetal server](/compute/flexmetal/api.md#creating-a-server-post) via our API, you select the `custom-ipxe` OS with the `ipxeScriptUrl` parameter pointing to your custom iPXE script and provide your ignition configuration as [userdata](/compute/flexmetal/metadata-api.md#userdata):

`POST https://api.i3d.net/v3/flexMetal/servers`

Request body:

```json
{
  "name": "server.example.org",
  "location": "EU: Rotterdam",
  "instanceType": "bm7.std.8",
  "os": {
    "slug": "custom-ipxe",
    "ipxeScriptUrl": "https://example.org/custom_ipxe.cfg"
  },
  "userData": {
    "data": "{\"ignition\":{\"version\":\"3.3.0\"},\"storage\":{\"files\":[{\"path\":\"/opt/get-metadata.sh\",\"contents\":{\"source\":\"https://example.org/metadata-script.sh\"},\"mode\":448}]}}"
  },
  "sshKey": [
    "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHo8IaPkQ6UnDZvi4F4RBSouRa6Gtysdg2EF+SIXheVF2SGBQ2uH7RfDjXRfvq4VpHJrKYs4kWfNoHQg8ZG6PH4= ecdsa-key-20240131"
  ],
  "tags": [
    "a-tag"
  ]
}
```

The SSH keys and tags you provide can be fetched from the [Metadata API](/compute/flexmetal/metadata-api.md) during configuration of Flatcar using ignition.

> When requesting a FlexMetal server with the `custom-ipxe` OS, the `delivered` state will be set as soon as the iPXE boot script is downloaded. This means that the OS images must still be downloaded and the OS needs to start up and come online.

## Ignition configuration

Your ignition configuration file determines how to setup Flatcar as it is booting. What you configure here is up to you. You can find the full ignition configuration in [Flatcar's ignition documentation](https://www.flatcar.org/docs/latest/provisioning/ignition/).

## Flatcar configuration using Metadata

Flatcar uses Butane/Ignition to configure the OS during first boot. When booting Flatcar via iPXE, you can pass a URL pointing to the Ignition configuration file where you can include a directive to use the i3D.net Metadata API for \[network] configuration. This requires two configurations: one to download a script to perform the configuration and one to run the script.

The functionality of this process is based on <https://www.flatcar.org/docs/latest/provisioning/ignition/dynamic-data/>

> i3D.net does not have direct integration with CoreOS or Flatcar, so you cannot use the `provider` kernel directive at this time to perform automatic configuration.

### Ignition configuration file additions

Download the [Metadata configuration script](#metadata-configuration-script) `metadata-script.sh` and store it in `/opt/get-metadata.sh` . An example script is provided in the next chapter below. You can host this on your own web server, or you can provide it to Ignition as a base64 encoded file instead.

```json
{
  "ignition": {
    "version": "3.3.0"
  },
  "storage": {
    "files": [
      {
        "path": "/opt/get-metadata.sh",
        "contents": {
          "source": "https://example.org/metadata-script.sh"
        },
        "mode": 448
      }
    ]
  }
}
```

Run the downloaded Metadata configuration script:

```json
{
  "ignition": {
    "version": "3.3.0"
  },
  "systemd": {
    "units": [
      {
        "name": "coreos-metadata.service",
        "contents": "[Unit]\nDescription=Metadata agent\nAfter=nss-lookup.target\nAfter=network-online.target\nWants=network-online.target\n[Service]\nType=oneshot\nRestart=on-failure\nRemainAfterExit=yes\nExecStart=/opt/get-metadata.sh\n"
      }
    ]
  }
}
```

### Metadata configuration script

The following bash script can be used for the Metadata configuration script as referenced in the Ignition configuration above. This will download a server's metadata, install the SSH keys for the `core` user, and configure the network. The network is only (re)configured when the server has NIC bonding, which FlexMetal servers are always delivered with. By default Flatcar automatically configures a single uplink according to the `ip` kernel parameter.

> \[!NOTE] Flatcar does not provision SSH keys out of the box. Unless you declare keys under `passwd.users[].sshAuthorizedKeys` in your ignition configuration, you must install them from the Metadata API as shown below — otherwise the server will be unreachable via SSH after boot.

You can customize this script if needed.

You must host this script at a location from which your server can download it.

`metadata-script.sh`

```bash
#!/bin/bash

# Get metadata
curl -4 https://metadata.i3d.net/v1/metadata > /opt/metadata.json

# If no metadata json file was written or the file is empty, quit
if [ ! -s "/opt/metadata.json" ]; then
    echo "Metadata JSON file doesn't exist or is empty - exiting"
    exit 1
fi

# Reject malformed JSON (e.g. an error page returned by curl). Every
# jq query below relies on this, so validate once and fail loudly here.
if ! jq -e . /opt/metadata.json > /dev/null 2>&1; then
    echo "Metadata response is not valid JSON - exiting"
    exit 1
fi

# Install SSH keys from metadata for the 'core' user.
# Flatcar merges fragments from /home/core/.ssh/authorized_keys.d/ into
# /home/core/.ssh/authorized_keys via update-ssh-keys. Do not write
# authorized_keys directly -- it is regenerated on the next run.
if jq -e '.ssh_keys | type == "array" and length > 0' /opt/metadata.json > /dev/null 2>&1; then
    install -d -o core -g core -m 700 /home/core/.ssh
    install -d -o core -g core -m 700 /home/core/.ssh/authorized_keys.d

    fragment=/home/core/.ssh/authorized_keys.d/i3d-metadata
    jq -r '.ssh_keys[]' /opt/metadata.json > "$fragment"
    chown core:core "$fragment"
    chmod 600 "$fragment"

    runuser -u core -- update-ssh-keys -a i3d-metadata "$fragment"
else
    echo "No ssh_keys in metadata - skipping SSH key install"
fi

# If there's no bond, we don't have to do anything, quit
if ! jq -e '.network_interfaces[0].is_bond' /opt/metadata.json > /dev/null 2>&1; then
    echo "No bond to configure - exiting"
    exit 0
fi

# Remove auto-generated configuration
rm /run/systemd/network/*.network

# Write new network configuration
cat > /run/systemd/network/00-bond0.netdev <<EOL
[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=802.3ad
LACPTransmitRate=fast
MIIMonitorSec=100ms
UpDelaySec=200ms
DownDelaySec=200ms
EOL

childInterfaces=""
while read -r childInterface; do
    macAddress=$(echo "$childInterface" | jq -r '.mac_address')
    childInterfaces+="${macAddress} "
done < <(jq -c '.network_interfaces[0].child_interfaces[]' /opt/metadata.json)

cat > /run/systemd/network/10-bond0-dev.network <<EOL
[Match]
MACAddress=$childInterfaces

[Network]
Bond=bond0
EOL

nl="
"
dnsIpv4=""
dnsIpv6=""
ifNetworkDetails=""
while read -r network; do
    ip=$(echo "$network" | jq -r '.ip')
    prefix=$(echo "$network" | jq -r '.prefix')
    gateway=$(echo "$network" | jq -r '.gateway')

    ifNetworkDetails+="Address=${ip}/${prefix}${nl}"
    ifNetworkDetails+="Gateway=${gateway}${nl}"

    if [[ $ip == *":"* ]]; then
        dnsIpv6="${dnsIpv6:-DNS=2001:4860:4860::8888${nl}}"
    else
        dnsIpv4="${dnsIpv4:-DNS=5.200.5.200${nl}DNS=5.200.5.5${nl}DNS=8.8.8.8${nl}}"
    fi
done < <(jq -c '.network_interfaces[0].networks[]' /opt/metadata.json)
ifNetworkDetails=$(printf '%s' "${ifNetworkDetails}")

cat > /run/systemd/network/20-bond0.network <<EOL
[Match]
Name=bond0

[Network]
$ifNetworkDetails
$dnsIpv4$dnsIpv6
EOL

# Apply the new network configuration
systemctl restart systemd-networkd
```

### Running a userdata script

The [`userData`](/compute/flexmetal/metadata-api.md#userdata) object carries a single blob in `userData.data` (optionally with `isBase64`), so you need to pick one of the following patterns — that single `userData.data` payload cannot be both an ignition config and a post-boot script at the same time:

* **`userData.data` holds the ignition config** (the pattern shown in the [example iPXE script](#example-ipxe-script) earlier in this guide, via `ignition.config.url=https://metadata.i3d.net/v1/userdata`). If you also need a post-boot script, host it at a URL your server can reach, or embed it in the ignition config as a `storage.files` entry.
* **`userData.data` holds a post-boot script.** Host your ignition config at a URL of your own (e.g. `https://example.org/ignition.cfg`) and point the iPXE script at that URL via `ignition.config.url=`. Then fetch the userdata payload from within `metadata-script.sh` and execute it, as shown below.

The snippet below covers the second pattern.

> \[!IMPORTANT] The userdata fetch must be placed **before** the bond reconfiguration (i.e., before the `If there's no bond` block). The bond reconfiguration restarts `systemd-networkd`, during which LACP renegotiates and connectivity is briefly unavailable — any network call made at that point will fail or return a partial response, which can leave a truncated or empty file on disk. Running the fetch earlier keeps it on the pre-bond network that is already up from iPXE boot.

Insert the following block in `metadata-script.sh` **after** the JSON validation block and the SSH-key provisioning block, but still **before** the `If there's no bond` block:

```bash
# Fetch and run the userdata script (pre-bond network, so this must run
# before systemd-networkd is restarted by the bond reconfiguration).
# Download to a temp file on the same filesystem as /opt/userdata.sh and
# only move it into place on success, so a partial or failed download
# cannot leave a broken /opt/userdata.sh behind. The mktemp path is in
# /opt/ specifically so the final mv is a same-filesystem atomic rename
# (mktemp defaults to /tmp, which is tmpfs on Flatcar).
tmp=$(mktemp /opt/.userdata.sh.XXXXXX)
if curl -fSL --retry 10 --retry-delay 2 --retry-connrefused --max-time 30 \
        -4 https://metadata.i3d.net/v1/userdata -o "$tmp" \
    && [ -s "$tmp" ]; then
    mv "$tmp" /opt/userdata.sh
    chmod u+x /opt/userdata.sh
    bash /opt/userdata.sh
else
    rm -f "$tmp"
    echo "Userdata download failed or was empty - skipping"
fi
```

The curl flags are worth calling out:

* `-f` — fail on HTTP errors instead of writing an error body to the file
* `-S` / `-L` — surface errors and follow redirects
* `--retry 10 --retry-delay 2 --retry-connrefused` — survive transient connectivity blips
* `--max-time 30` — bounded overall wait so the service cannot hang indefinitely

Invoking via `bash /opt/userdata.sh` instead of relying on the shebang means execution does not silently depend on the first line of the downloaded content being a valid interpreter directive.

The `isBase64` flag on the create-server request is **submission-side only**, and exists because a JSON request body can only carry string values — so if your payload is binary (a zip, a compiled binary, an image, etc.) you must base64-encode it and set `isBase64: true` to get it through the API at all. When i3D receives a payload marked with `isBase64: true` it decodes it on submission and stores the original raw bytes.

The [userdata endpoint](/compute/flexmetal/metadata-api.md#userdata) always returns those stored raw bytes. Your Flatcar server does **not** need to `base64 -d` the response — whether you submitted the payload as plain text or as base64 with `isBase64: true`, the bytes returned by `curl https://metadata.i3d.net/v1/userdata` are the original, decoded content. The snippet above writes that response straight to `/opt/userdata.sh` and runs it, which is correct in both cases.

If your userdata is not a shell script (for example a JSON config or a binary file), adjust the output path and drop the `chmod u+x` / `bash` execution lines accordingly.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.i3d.net/compute/flexmetal/flatcar-installation.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
