How to Automate Installation of Fleet

Fleet is part of the Elastic Stack, and allows you to manage all your Elastic Agents. Installing it in a fully automated manner is difficult because it requires configuration in Kibana. Here’s how to do it.

Configuring Kibana

Kibana’s REST API can be used to set up all the necessary configuration options for the Fleet server.

A note about SSL: This tutorial includes SSL configuration and supports using certificates signed by your own Certificate Authority (CA), as recommended by the Elasticsearch installation guide. If you are not using SSL then you can remove the --cacert options and change https to http.

First, we’ll set up some variables. Modify these values to suit your environment:

# The location of the Certificate Authority file.
# This is generated by Elasticsearch in the setup container and mounted into the container.
CA=/certs/ca/ca.crt

# The host and port of the Elasticsearch server.
ELASTIC_HOST=es01:9200

# The username this script uses to connect to Elasticsearch.
ELASTIC_USER=elastic

# The password this script uses to connect to Elasticsearch.
ELASTIC_PASS=elastic

# The URL to the Kibana server, without a trailing slash.
KIBANA_URL=http://kibana:5601

Service Account Token

The Service Account Token is used by Fleet to authorize itself with Elasticsearch. We will create a token with a random name so that it does not collide with any existing tokens.

createServiceAccountToken()
{
    echo -n 'Creating service account token... '

    # Create a random name for the token so it does not collied with any existing tokens.
    tokenName=`cat /proc/sys/kernel/random/uuid`

    # Create the token.
    response=`curl \
        -u $ELASTIC_USER:$ELASTIC_PASS \
        --silent \
        --cacert $CA \
        -X POST "https://$ELASTIC_HOST/_security/service/elastic/fleet-server/credential/token/$tokenName?pretty"`

    # Check that the response was successful.
    if ! echo "$response" | grep --silent '"created" : true'; then
        echo 'failed'
        echo "$response"
        exit 1
    fi

    # Parse the token out of the response.
    token=`echo "$response" | awk '/value/ { print $3 }' | sed 's/"//g'`

    echo "success: $token"
}

This will store the token in the variable $token for use later.

Agent Policy

The Agent Policy configures which integrations (plugins) are associated with the agent. This will create an empty policy, to which we will later add the Fleet Server integration.

createAgentPolicy()
{
    echo -n 'Creating Fleet server policy... '

    response=`curl \
        -u $ELASTIC_USER:$ELASTIC_PASS \
        --silent \
        --request POST \
        --url $KIBANA_URL/api/fleet/agent_policies?sys_monitoring=true \
        --header 'content-type: application/json' \
        --header 'kbn-xsrf: true' \
        --data '{"id": "docker-fleet-server-policy","name":"Docker Fleet Server Policy","namespace":"default","monitoring_enabled":["logs","metrics"],"is_default_fleet_server": true}'`

    if echo "$response" | grep --silent 'document already exists'; then
        green 'already exists'
        return 0
    fi

    green 'success'
}

Server Integration

The Agent Policy needs the “Fleet Server” integration added to it:

addServerIntegration()
{
    echo -n 'Adding Fleet server integration to policy... '

    response=`curl \
        -u $ELASTIC_USER:$ELASTIC_PASS \
        --silent \
        --request POST \
        --url $KIBANA_URL/api/fleet/package_policies \
        --header 'content-type: application/json' \
        --header 'kbn-xsrf: true' \
        --data '
        {
            "policy_id": "fleet-server-policy",
            "package": {
                "name": "fleet_server",
                "version": "1.2.0"
            },
            "name": "fleet_server",
            "description": "",
            "namespace": "default",
            "inputs": {
                "fleet_server-fleet-server": {
                "enabled": true,
                "vars": {
                    "host": "fleet",
                    "port": [
                    8220
                    ],
                    "custom": ""
                },
                "streams": {}
                }
            }
        }'`

    if echo $response | grep --silent 'already exists'; then
        echo 'already exists';
        return 0
    fi

    if echo $response | grep --silent 'error'; then
        echo 'failed'
        echo "$response"
        exit 1
    fi

    echo 'success'
}

Server Host

The Fleet Server Host setting tells the Elastic Agents the hostname and port on which they can contact the Fleet server. Make sure to update the URL to match your environment. The default port for a Fleet server is 8220.

Note: There is a bug in Kibana that will clear the is_default option if you try to overwrite an ID that already exists. If there is a chance that the setting may already exist, you must check first.

addServerHost()
{
    echo -n 'Checking if Fleet server host exists... '

    response=`curl \
        -u elastic:elastic \
        --silent \
        --request GET \
        --cacert /usr/share/kibana/config/certs/ca/ca.crt \
        --url http://kibana:5601/api/fleet/fleet_server_hosts/fleet-server \
        --header 'content-type: application/json' \
        --header 'kbn-xsrf: true'`

    if ! echo $response | grep --silent 'Not Found'; then
        echo 'already exists'
        return 0
    else
        echo 'does not exist'
    fi

    echo -n 'Adding Fleet server host... '

    response=`curl \
        -u $ELASTIC_USER:$ELASTIC_PASS \
        --silent \
        --request POST \
        --url $KIBANA_URL/api/fleet/fleet_server_hosts \
        --header 'content-type: application/json' \
        --header 'kbn-xsrf: true' \
        --data '
        {
            "id": "fleet-server",
            "name": "Fleet Server",
            "is_default": true,
            "host_urls": [
                "https://fleet:8220"
            ]
        }'`

    if echo $response | grep --silent 'error'; then
        echo 'failed'
        echo "$response"
        exit 1
    fi

    echo 'success'

Output

The Output setting defines where Fleet should send its data to. In this case, it’s being sent to Elasticsearch.

setOutput()
{
    echo -n 'Setting agent output location... '

    response=`curl \
        -u $ELASTIC_USER:$ELASTIC_PASS \
        --silent \
        --request PUT \
        --url $KIBANA_URL/api/fleet/outputs/fleet-default-output \
        --header 'content-type: application/json' \
        --header 'kbn-xsrf: true' \
        --data "
        {
            \"name\": \"default\",
            \"type\": \"elasticsearch\",
            \"is_default\": true,
            \"is_default_monitoring\": true,
            \"hosts\": [
                \"https://$ELASTIC_HOST\"
            ],
            \"config_yaml\": \"ssl.certificate_authorities: ['/certs/ca/ca.crt']\"
        }"`

    if echo "$response" | grep --silent 'document already exists'; then
        echo 'already exists'
        return 0
    fi

    if echo $response | grep --silent 'error'; then
        echo 'failed'
        echo "$response"
        exit 1
    fi

    echo 'success'
}

Installing Fleet

Now that Kibana has been configured, Fleet can be installed. Be sure to edit the --url= parameter to the hostname of your fleet server.

installFleet()
{
    apt update
    apt install -y curl
    curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.6.0-linux-x86_64.tar.gz
    tar xzvf elastic-agent-8.6.0-linux-x86_64.tar.gz
    cd elastic-agent-8.6.0-linux-x86_64

    ./elastic-agent install \
        --non-interactive \
        --fleet-server-es=https://$ELASTIC_HOST \
        --fleet-server-service-token=$token  \
        --fleet-server-policy=docker-fleet-server-policy \
        --fleet-server-es-ca=/certs/ca/ca.crt \
        --fleet-server-cert=/certs/fleet/fleet.crt \
        --fleet-server-cert-key=/certs/fleet/fleet.key \
        --certificate-authorities=/certs/ca/ca.crt \
        --url=https://fleet:8220
}