HA - Active Passive Deployment

In this document we describe how to install and upgrade two SkyFormation servers in an active-passive deployment by having a shared mounted directory.
Active-passive is a deployment in which we have only a single server running at every given moment (the active server) and another server in a standby (the passive server).
In the event that the active server crashes, a procedure is executed to switch between the two and change the active to passive and vice versa.
We start by having two linux machines which will be referred to as sk4-ha-active and sk4-ha-passive (please see the prerequisites for these two machines)

Preliminary Steps

1. Create a shared folder of size 512GB and mount it in both sk4-ha-active and sk4-ha-passive
   Without loss of generality we assume the shared folder is mapped to /active/sk4-shared and /passive/sk4-shared in sk4-ha-active and sk4-ha-passive respectively
2. In both sk4-ha-active and sk4-ha-passive create the SkyFormation’s root directory /opt/exabeam/data/sk4

mkdir -p /opt/exabeam/data/sk4

3. The following directories should be created in the shared folder:

• conf
• kafka_conf
• kafka_data
• pg_conf
• pg_data
• zoo_data
• zoo_datalog

From sk4-ha-active it can be done by running the following bash commands:

mkdir /active/sk4-shared/conf
mkdir /active/sk4-shared/kafka_conf
mkdir /active/sk4-shared/kafka_data
mkdir /active/sk4-shared/pg_conf
mkdir /active/sk4-shared/pg_data
mkdir /active/sk4-shared/zoo_data
mkdir /active/sk4-shared/zoo_datalog

4. Create soft links in sk4-ha-active to later be used to by the SkyFormation’s docker volumes

ln -s /active/sk4-shared/conf /opt/exabeam/data/sk4/conf
ln -s /active/sk4-shared/kafka_conf /opt/exabeam/data/sk4/kafka_conf
ln -s /active/sk4-shared/kafka_data /opt/exabeam/data/sk4/kafka_data
ln -s /active/sk4-shared/pg_conf /opt/exabeam/data/sk4/pg_conf
ln -s /active/sk4-shared/pg_data /opt/exabeam/data/sk4/pg_data
ln -s /active/sk4-shared/zoo_data /opt/exabeam/data/sk4/zoo_data
ln -s /active/sk4-shared/zoo_datalog /opt/exabeam/data/sk4/zoo_datalog

5. Create soft links in sk4-ha-passive to later be used to by the SkyFormation’s docker volumes

ln -s /passive/sk4-shared/conf /opt/exabeam/data/sk4/conf
ln -s /passive/sk4-shared/kafka_conf /opt/exabeam/data/sk4/kafka_conf
ln -s /passive/sk4-shared/kafka_data /opt/exabeam/data/sk4/kafka_data
ln -s /passive/sk4-shared/pg_conf /opt/exabeam/data/sk4/pg_conf
ln -s /passive/sk4-shared/pg_data /opt/exabeam/data/sk4/pg_data
ln -s /passive/sk4-shared/zoo_data /opt/exabeam/data/sk4/zoo_data
ln -s /passive/sk4-shared/zoo_datalog /opt/exabeam/data/sk4/zoo_datalog

Installation

Active Server

1. Install SkyFormation’s server in sk4-ha-active with the exabeam deployment flag (-d=exabeam)
2. Stop SkyFormation’s server in sk4-ha-active

sudo systemctl stop sk4compose

3. Copy the encryption key generated during the installation. Key can be retrieved by running the following bash command:

grep SKYFORMATION_ENC_KEY /opt/exabeam/data/sk4/docker-compose.yml | head -1 | sed 's/.*SKYFORMATION_ENC_KEY=//'

Passive Server

1. Install SkyFormation’s server in sk4-ha-passive with the exabeam deployment flag (-d=exabeam)
2. Stop SkyFormation’s server in sk4-ha-passive

sudo systemctl stop sk4compose

4. Disable the SkyFormation’s service (to prevent it from running following a machine reboot)

sudo systemctl disable sk4compose

5. Copy the encryption key generated during the installation. Key can be retrieved by running the following bash command:

grep SKYFORMATION_ENC_KEY /opt/exabeam/data/sk4/docker-compose.yml | head -1 | sed 's/.*SKYFORMATION_ENC_KEY=//'

Use Same Encryption Key in Both Active and Passive Servers

Override the ecryption key in sk4-ha-passive with the encryption key coppied from sk4-ha-active by running, in sk4-ha-passive, the following bash command (after replacing SKYFORMATION_ENC_KEY_FROM_PASSIVE and SKYFORMATION_ENC_KEY_FROM_ACTIVE)

sed -i 's/SKYFORMATION_ENC_KEY_FROM_PASSIVE/SKYFORMATION_ENC_KEY_FROM_ACTIVE/g' /opt/exabeam/data/sk4/docker-compose.yml

7. Start SkyFormation’s server in sk4-ha-active

sudo systemctl start sk4compose

Upgrade

Active Server

Upgrade SkyFormation’s server in sk4-ha-active

Passive Server

In sk4-ha-passive run the following bash commands:

sudo su -
curl -L https://download.skyformation.net/v2/download/installer/edge/compose-commons.sh > compose-commons.sh
source compose-commons.sh
docker_login REPLACE_WITH_YOUR_LICENSE_KEY
docker-compose -f /opt/exabeam/data/sk4/docker-compose.yml pull
download_docker_images

IMPORTANT: do not run the upgrade procedure in the passive server

Switch Between Active and Passive

Active Server

Stop and disable the SkyFormation’s service

sudo systemctl stop sk4compose
sudo systemctl disable sk4compose

Passive Server

Start and enable the SkyFormation’s service

sudo systemctl start sk4compose
sudo systemctl enable sk4compose