# Flatcar installation

[Flatcar](https://www.flatcar.org/) allows you to create Kubernetes clusters with ease based on immutable Linux nodes.

With i3D.net FlexMetal you can boot Flatcar nodes using the [Custom iPXE](https://docs.i3d.net/compute/flexmetal/custom-ipxe-booting) feature. On this page we will describe the process.

## Create your custom iPXE script

To boot the Flatcar OS, you must use the Custom iPXE feature when requesting a FlexMetal server. Below you will find how to perform an API request to do that.

Inside the iPXE script, you must point to the version of Flatcar you want to boot, while also providing the URL to your ignition configuration file. This URL should point to the [Metadata API userdata endpoint](https://docs.i3d.net/compute/metadata-api#userdata), so that the ignition configuration is fetched from your server's userdata during boot.

## Example iPXE script

This is an example iPXE script to boot the latest LTS version of the Flatcar OS, using [Custom iPXE network variables](https://docs.i3d.net/compute/custom-ipxe-booting#ipxe-network-variables) to configure the network for the initial boot phase. Note the `ignition.config.url` kernel parameter that points to the [userdata endpoint](https://docs.i3d.net/compute/metadata-api#userdata), which will serve the ignition configuration you uploaded as userdata when creating the server. See how to set userdata to your configuration file in the [example request](#example-flexmetal-api-request) section below.

{% code overflow="wrap" %}

```
#!ipxe

set bootimage_url https://lts.release.flatcar-linux.net/amd64-usr/current
set os_parameters initrd=flatcar_production_pxe_image.cpio.gz flatcar.first_boot=1 ignition.config.url=https://metadata.i3d.net/v1/userdata flatcar.autologin net.ifnames=0 hostname=${IPXE_BOOT_HOSTNAME} ${IPXE_BOOT_LINUX_IP_CONFIG}

kernel ${bootimage_url}/flatcar_production_pxe.vmlinuz ${os_parameters}
initrd ${bootimage_url}/flatcar_production_pxe_image.cpio.gz

boot
```

{% endcode %}

> \[!NOTE] You can also set the ignition config url to download the config file from your own server if you prefer. For example: `ignition.config.url=https://my-server.com/ignition-configuration.ign`

More details can be found on [Flatcar's iPXE boot documentation](https://www.flatcar.org/docs/latest/installing/bare-metal/booting-with-ipxe/).

## Example FlexMetal API request

To [request a FlexMetal server](https://docs.i3d.net/compute/api#creating-a-server-post) via our API, you select the `custom-ipxe` OS with the `ipxeScriptUrl` parameter pointing to your custom iPXE script and provide your ignition configuration as [userdata](https://docs.i3d.net/compute/metadata-api#userdata):

`POST https://api.i3d.net/v3/flexMetal/servers`

Request body:

```json
{
  "name": "server.example.org",
  "location": "EU: Rotterdam",
  "instanceType": "bm7.std.8",
  "os": {
    "slug": "custom-ipxe",
    "ipxeScriptUrl": "https://example.org/custom_ipxe.cfg"
  },
  "userData": {
    "data": "{\"ignition\":{\"version\":\"3.3.0\"},\"storage\":{\"files\":[{\"path\":\"/opt/get-metadata.sh\",\"contents\":{\"source\":\"https://example.org/metadata-script.sh\"},\"mode\":700}]}}"
  },
  "postInstallScript": "",
  "sshKey": [
    "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHo8IaPkQ6UnDZvi4F4RBSouRa6Gtysdg2EF+SIXheVF2SGBQ2uH7RfDjXRfvq4VpHJrKYs4kWfNoHQg8ZG6PH4= ecdsa-key-20240131"
  ],
  "tags": [
    "a-tag"
  ]
}
```

The SSH keys and tags you provide can be fetched from the [Metadata API](https://docs.i3d.net/compute/flexmetal/metadata-api) during configuration of Flatcar using ignition.

> When requesting a FlexMetal server with the `custom-ipxe` OS, the `delivered` state will be set as soon as the iPXE boot script is downloaded. This means that the OS images must still be downloaded and the OS needs to start up and come online.

## Ignition configuration

Your ignition configuration file determines how to setup Flatcar as it is booting. What you configure here is up to you. You can find the full ignition configuration in [Flatcar's ignition documentation](https://www.flatcar.org/docs/latest/provisioning/ignition/).

## Flatcar configuration using Metadata

Flatcar uses Butane/Ignition to configure the OS during first boot. When booting Flatcar via iPXE, you can pass a URL pointing to the Ignition configuration file where you can include a directive to use the i3D.net Metadata API for \[network] configuration. This requires two configurations: one to download a script to perform the configuration and one to run the script.

The functionality of this process is based on <https://www.flatcar.org/docs/latest/provisioning/ignition/dynamic-data/>

> i3D.net does not have direct integration with CoreOS or Flatcar, so you cannot use the `provider` kernel directive at this time to perform automatic configuration.

### Ignition configuration file additions

Download the [Metadata configuration script](#metadata-configuration-script) `metadata-script.sh` and store it in `/opt/get-metadata.sh` . An example script is provided in the next chapter below. You can host this on your own web server, or you can provide it to Ignition as a base64 encoded file instead.

```json
{
  "ignition": {
    "version": "3.3.0"
  },
  "storage": {
    "files": [
      {
        "path": "/opt/get-metadata.sh",
        "contents": {
          "source": "https://example.org/metadata-script.sh"
        },
        "mode": 700
      }
    ]
  }
}
```

Run the downloaded Metadata configuration script:

```json
{
  "ignition": {
    "version": "3.3.0"
  },
  "systemd": {
    "units": [
      {
        "name": "coreos-metadata.service",
        "contents": "[Unit]\nDescription=Metadata agent\nAfter=nss-lookup.target\nAfter=network-online.target\nWants=network-online.target\n[Service]\nType=oneshot\nRestart=on-failure\nRemainAfterExit=yes\nExecStart=/opt/get-metadata.sh\n"
      }
    ]
  }
}
```

### Metadata configuration script

The following bash script can be used for the Metadata configuration script as referenced in the Ignition configuration above. This will download a server's metadata, configure the hostname and network, but only if that contains a bond (FlexMetal servers are always delivered with NIC bonding). By default Flatcar automatically configures a single uplink according to the `ip` kernel parameter.

You can customize this script if needed.

You must host this script at a location from which your server can download it.

`metadata-script.sh`

```bash
#!/bin/bash

# Get metadata
curl -4 https://metadata.i3d.net/v1/metadata > /opt/metadata.json

# If no metadata json file was written or the file is empty, quit
if [ ! -s "/opt/metadata.json" ]; then
    echo "Metadata JSON file doesn't exist or is empty - exiting"
    exit 1
fi

# If there's no bond, we don't have to do anything, quit
if ! jq -e '.network_interfaces[0].is_bond' /opt/metadata.json > /dev/null 2>&1; then
    echo "No bond to configure - exiting"
    exit 0
fi

# Remove auto-generated configuration
rm /run/systemd/network/*.network

# Write new network configuration
cat > /run/systemd/network/00-bond0.netdev <<EOL
[NetDev]
Name=bond0
Kind=bond

[Bond]
Mode=802.3ad
LACPTransmitRate=fast
MIIMonitorSec=100ms
UpDelaySec=200ms
DownDelaySec=200ms
EOL

childInterfaces=""
while read -r childInterface; do
    macAddress=$(echo "$childInterface" | jq -r '.mac_address')
    childInterfaces+="${macAddress} "
done < <(jq -c '.network_interfaces[0].child_interfaces[]' /opt/metadata.json)

cat > /run/systemd/network/10-bond0-dev.network <<EOL
[Match]
MACAddress=$childInterfaces

[Network]
Bond=bond0
EOL

nl="
"
dnsIpv4=""
dnsIpv6=""
ifNetworkDetails=""
while read -r network; do
    ip=$(echo "$network" | jq -r '.ip')
    prefix=$(echo "$network" | jq -r '.prefix')
    gateway=$(echo "$network" | jq -r '.gateway')

    ifNetworkDetails+="Address=${ip}/${prefix}${nl}"
    ifNetworkDetails+="Gateway=${gateway}${nl}"

    if [[ $ip == *":"* ]]; then
        dnsIpv6="${dnsIpv6:-DNS=2001:4860:4860::8888${nl}}"
    else
        dnsIpv4="${dnsIpv4:-DNS=5.200.5.200${nl}DNS=5.200.5.5${nl}DNS=8.8.8.8${nl}}"
    fi
done < <(jq -c '.network_interfaces[0].networks[]' /opt/metadata.json)
ifNetworkDetails=$(printf '%s' "${ifNetworkDetails}")

cat > /run/systemd/network/20-bond0.network <<EOL
[Match]
Name=bond0

[Network]
$ifNetworkDetails
$dnsIpv4$dnsIpv6
EOL

# Apply the new network configuration
systemctl restart systemd-networkd
```