Updating previous versions of KUMA

To install KUMA version 2.1.3 over version 2.0.x, complete the preliminary steps and then update.

Preliminary steps

1. Creating a backup copy of the KUMA Core.
2. Make sure that all application installation requirements are met.
3. Make sure that MongoDB versions are compatible by running the following sequence of commands on the device where KUMA Core is located:

   cd /opt/kaspersky/kuma/mongodb/bin/

   ./mongo

   use kuma

   db.adminCommand({getParameter: 1, featureCompatibilityVersion: 1})

   If the component version is different from 4.4, set the value to 4.4 using the following command: 

   db.adminCommand({ setFeatureCompatibilityVersion: "4.4" })

4. During installation or update, ensure network accessibility of TCP port 7220 on the KUMA Core for the KUMA storage hosts.
5. If you have a keeper deployed on a separate device in the ClickHouse cluster, install the storage service on the same device before performing the update:
   • Use the existing storage of the cluster to create a storage service for the keeper in the web interface.
   • Install the service on a device with a dedicated ClickHouse keeper.
6. In the inventory file, specify the same hosts that were used when installing KUMA version 2.0.X. Set the following settings to false:

   deploy_to_k8s false

   need_transfer false

   deploy_example_services false

   When the installer uses this inventory file, all KUMA components are upgraded to version 2.1.3. The available services and storage resources are also reconfigured on hosts from the kuma_storage group:

   • ClickHouse systemd services are deleted.
   • Certificates are deleted from the /opt/kaspersky/kuma/clickhouse/certificates directory.
   • The Shard ID, Replica ID, Keeper ID, and ClickHouse configuration override fields are filled in for each node in the storage resource based on values from the inventory and configuration files of the service on the host. Subsequently, you will manage the roles of each node in the KUMA web interface.
   • All existing configuration files from the /opt/kaspersky/kuma/clickhouse/cfg directory are deleted (they will be subsequently generated by the storage service).
   • The value of the LimitNOFILE parameter (Service section) is changed from 64,000 to 500,000 in the kuma-storage systemd services.
7. If you use alert segmentation rules, prepare the data for migrating the existing rules and save. In the next step, you can use this data to re-create the rules. During the update, alert segmentation rules are not migrated automatically.
8. To perform an update, you need a valid password from the admin user. If you forgot the admin user password, contact Technical Support to reset the current password and use the new password to perform the update at the next step.

Updating KUMA

1. Depending on the KUMA deployment scheme being used, do one the following:

• Use the prepared distributed.inventory.yml inventory file and follow the instructions for distributed installation of the application.
• Use the prepared k0s.inventory.yml inventory file and follow the instructions for distributed installation in a high availability configuration.

  If an inventory file is not available for the current version, use the provided inventory file template and fill in the corresponding settings. To view a list of hosts and host roles in the current KUMA system, in the web interface, go to Resources → Active services section.

  The upgrade process completely reproduces the installation process.

  If you want to upgrade from a distributed installation to a distributed installation in a high availability configuration, first upgrade the distributed installation and then migrate the Core to a Kubernetes cluster.

  To migrate KUMA Core to a new Kubernetes cluster:

  1. Prepare the k0s.inventory.yml inventory file.

     The kuma_core, kuma_ collector, kuma_correlator, kuma_storage sections of your k0s.inventory.yml inventory file must contain the same hosts that were used when KUMA was upgraded from version 2.1.3 to version 3.0.2 or when a new installation was performed. In the inventory file, set the deploy_to_k8s, need_transfer and airgap parameters to true. The deploy_example_services parameter must be set to false.

     Sample inventory file with 3 dedicated controllers, 2 worker nodes, and 1 balancer.

  2. Follow the steps for distributed installation using your prepared k0s.inventory.yml inventory file.

  Migrating the KUMA Core to a new Kubernetes cluster

  When the installer is started with the inventory file, it looks for an installed KUMA Core on all hosts where you want to deploy worker nodes of the cluster. The found Core will be moved from the host to within the newly created Kubernetes cluster.

  If the component is not detected on the worker nodes, a clean installation of the KUMA Core is performed in the cluster without migrating resources to it. Existing components must be manually rebuilt with the new Core in the KUMA web interface.

  Certificates for collectors, correlators and storages will be re-issued from the inventory file for communication with the Core within the cluster. This does not change the Core URL for components.

  On the Core host, the installer does the following:

  • Removes the following systemd services from the host: kuma-core, kuma-mongodb, kuma-victoria-metrics, kuma-vmalert, and kuma-grafana.
  • Deletes the internal certificate of the Core.
  • Deletes the certificate files of all other components and deletes their records from MongoDB.
  • Deletes the following directories:
    • /opt/kaspersky/kuma/core/bin
    • /opt/kaspersky/kuma/core/certificates
    • /opt/kaspersky/kuma/core/log
    • /opt/kaspersky/kuma/core/logs
    • /opt/kaspersky/kuma/grafana/bin
    • /opt/kaspersky/kuma/mongodb/bin
    • /opt/kaspersky/kuma/mongodb/log
    • /opt/kaspersky/kuma/victoria-metrics/bin
  • Migrates data from the Core and its dependencies to a network drive within the Kubernetes cluster.
  • On the Core host, it migrates the following directories:
    • /opt/kaspersky/kuma/core → /opt/kaspersky/kuma/core.moved
    • /opt/kaspersky/kuma/grafana → /opt/kaspersky/kuma/grafana.moved
    • /opt/kaspersky/kuma/mongodb → /opt/kaspersky/kuma/mongodb.moved
    • /opt/kaspersky/kuma/victoria-metrics → /opt/kaspersky/kuma/victoria-metrics.moved

  After you have verified that the Core was correctly migrated to the cluster, these directories can be deleted.

  If you encounter problems with the migration, check the logs for records of the 'core-transfer' migration task in the 'kuma' namespace in the cluster (this task is available for 1 hour after the migration).

  If you need to perform migration again, you must convert the names of the /opt/kaspersky/kuma/*.moved directories back to their original format.

  If an /etc/hosts file with lines not related to addresses in the range 127.X.X.X was used on the Core host, the contents of the /etc/hosts file from the Core host is entered into the CoreDNS ConfigMap when the Core is migrated to the Kubernetes cluster. If the Core is not migrated, the contents of the /etc/hosts file from the host where the primary controller is deployed are entered into the ConfigMap.

1. When upgrading on systems that contain large amounts of data and are operating with limited resources, the system may return the 'Wrong admin password' error message after you enter the administrator password. If you specify the correct password, KUMA may still return an error because KUMA could not start the Core service due to a timeout error and resource limit. If you enter the administrator password three times without waiting for the installation to complete, the update may end with a fatal error. Resolve the timeout error to proceed with the update.

The final stage of preparing KUMA for work

1. After updating KUMA, you must clear your browser cache.
2. Re-create the alert segmentation rules.
3. Manually update the KUMA agents.

KUMA update completed successfully.

To install KUMA version 2.1.3 over version 2.1.x, complete the preliminary steps and then update.

Preliminary steps

1. Creating a backup copy of the KUMA Core.
2. Make sure that all application installation requirements are met.
3. During installation or update, ensure network accessibility of TCP port 7220 on the KUMA Core for the KUMA storage hosts.
4. To perform an update, you need a valid password from the admin user. If you forgot the admin user password, contact Technical Support to reset the current password and use the new password to perform the update at the next step.

Updating KUMA

1. Depending on the KUMA deployment scheme being used, do one the following:

• Use the prepared distributed.inventory.yml inventory file and follow the instructions for distributed installation of the application.
• Use the prepared k0s.inventory.yml inventory file and follow the instructions for distributed installation in a high availability configuration.

  If an inventory file is not available for the current version, use the provided inventory file template and fill in the corresponding settings. To view a list of hosts and host roles in the current KUMA system, in the web interface, go to Resources → Active services section.

  The upgrade process completely reproduces the installation process.

  If you want to upgrade from a distributed installation to a distributed installation in a high availability configuration, first upgrade the distributed installation and then migrate the Core to a Kubernetes cluster.

  To migrate KUMA Core to a new Kubernetes cluster:

  1. Prepare the k0s.inventory.yml inventory file.

     The kuma_core, kuma_ collector, kuma_correlator, kuma_storage sections of your k0s.inventory.yml inventory file must contain the same hosts that were used when KUMA was upgraded from version 2.1.3 to version 3.0.2 or when a new installation was performed. In the inventory file, set the deploy_to_k8s, need_transfer and airgap parameters to true. The deploy_example_services parameter must be set to false.

     Sample inventory file with 3 dedicated controllers, 2 worker nodes, and 1 balancer.

  2. Follow the steps for distributed installation using your prepared k0s.inventory.yml inventory file.

  Migrating the KUMA Core to a new Kubernetes cluster

  When the installer is started with the inventory file, it looks for an installed KUMA Core on all hosts where you want to deploy worker nodes of the cluster. The found Core will be moved from the host to within the newly created Kubernetes cluster.

  If the component is not detected on the worker nodes, a clean installation of the KUMA Core is performed in the cluster without migrating resources to it. Existing components must be manually rebuilt with the new Core in the KUMA web interface.

  Certificates for collectors, correlators and storages will be re-issued from the inventory file for communication with the Core within the cluster. This does not change the Core URL for components.

  On the Core host, the installer does the following:

  • Removes the following systemd services from the host: kuma-core, kuma-mongodb, kuma-victoria-metrics, kuma-vmalert, and kuma-grafana.
  • Deletes the internal certificate of the Core.
  • Deletes the certificate files of all other components and deletes their records from MongoDB.
  • Deletes the following directories:
    • /opt/kaspersky/kuma/core/bin
    • /opt/kaspersky/kuma/core/certificates
    • /opt/kaspersky/kuma/core/log
    • /opt/kaspersky/kuma/core/logs
    • /opt/kaspersky/kuma/grafana/bin
    • /opt/kaspersky/kuma/mongodb/bin
    • /opt/kaspersky/kuma/mongodb/log
    • /opt/kaspersky/kuma/victoria-metrics/bin
  • Migrates data from the Core and its dependencies to a network drive within the Kubernetes cluster.
  • On the Core host, it migrates the following directories:
    • /opt/kaspersky/kuma/core → /opt/kaspersky/kuma/core.moved
    • /opt/kaspersky/kuma/grafana → /opt/kaspersky/kuma/grafana.moved
    • /opt/kaspersky/kuma/mongodb → /opt/kaspersky/kuma/mongodb.moved
    • /opt/kaspersky/kuma/victoria-metrics → /opt/kaspersky/kuma/victoria-metrics.moved

  After you have verified that the Core was correctly migrated to the cluster, these directories can be deleted.

  If you encounter problems with the migration, check the logs for records of the 'core-transfer' migration task in the 'kuma' namespace in the cluster (this task is available for 1 hour after the migration).

  If you need to perform migration again, you must convert the names of the /opt/kaspersky/kuma/*.moved directories back to their original format.

  If an /etc/hosts file with lines not related to addresses in the range 127.X.X.X was used on the Core host, the contents of the /etc/hosts file from the Core host is entered into the CoreDNS ConfigMap when the Core is migrated to the Kubernetes cluster. If the Core is not migrated, the contents of the /etc/hosts file from the host where the primary controller is deployed are entered into the ConfigMap.

1. When upgrading on systems that contain large amounts of data and are operating with limited resources, the system may return the 'Wrong admin password' error message after you enter the administrator password. If you specify the correct password, KUMA may still return an error because KUMA could not start the Core service due to a timeout error and resource limit. If you enter the administrator password three times without waiting for the installation to complete, the update may end with a fatal error. Resolve the timeout error to proceed with the update.

The final stage of preparing KUMA for work

1. After updating KUMA, you must clear your browser cache.
2. Manually update the KUMA agents.

KUMA update completed successfully.

To install KUMA version 3.0.2 over version 2.1.3, complete the preliminary steps and then perform the upgrade.

Preliminary steps

1. Creating a backup copy of the KUMA Core. If necessary, you can restore data from backup.
2. Make sure that all application installation requirements are met.
3. During installation or update, ensure network accessibility of TCP port 7220 on the KUMA Core for the KUMA storage hosts.

Updating KUMA

Depending on the KUMA deployment scheme being used, do one the following:

• Use the prepared distributed.inventory.yml inventory file and follow the instructions for distributed installation of the application.
• Use the prepared k0s.inventory.yml inventory file and follow the instructions for distributed installation in a high availability configuration.

  If an inventory file is not available for the current version, use the provided inventory file template and fill in the corresponding settings. To view a list of hosts and host roles in the current KUMA system, in the web interface, go to Resources → Active services section.

The upgrade process completely reproduces the installation process.

If you want to upgrade from a distributed installation to a distributed installation in a high availability configuration, first upgrade the distributed installation and then migrate the Core to a Kubernetes cluster.

The final stage of preparing KUMA for work

1. After updating KUMA, you must clear your browser cache.
2. Manually update the KUMA agents.

KUMA update completed successfully.

Known limitations

1. Hierarchical structure is not supported in 3.0.2, therefore all KUMA hosts become standalone hosts when upgrading from version 2.1.3 to 3.0.2.
2. For existing users, upgrading from 2.1.3 to 3.0.2 does not update the universal dashboard layout.

   Possible solution: restart the Core service (kuma-core.service), and the data refresh interval configured for the layout will be used.