When upgrading from KUMA 3.4 to 4.0.x, you must determine how much disk space the metrics data occupies and evaluate whether it is necessary to override the metrics_disk_request setting in the k0s.inventory.yml inventory file.
To evaluate the amount of disk space that will be taken up by metrics data in KUMA 3.4:
On the target machine, run the get_old_metrics_size.sh script with the k0s.inventory.yml inventory file:
sudo ./get_old_metrics_size_sh k0s.inventory.yml
This script does not make any changes to the existing installation; it connects to the primary controller of the cluster to determine the name of the KUMA Core volume and to the worker nodes of the cluster to determine the directory size inside the mounted volume.
As a result, a search for KUMA Core volumes is performed on the worker nodes of the cluster, and the size of the victoria-metrics directory in them is determined. The output of the script is displayed and written to the log/get_old_metrics_size-YYYYMMDD-HHMMSS.log file. The following message is logged:
TASK [Print metrics folder size] ***********************************************
ok: [kuma-inst-06.avp.ru] => {}
MSG:
Found KUMA Core 3.4.1.53
Found old metrics folder size: 1Gi
Minimum metrics_data_size required to proceed: 3Gi (including 2Gi threshold)
The message contains the following information:
Found KUMA Core is the found version of the KUMA Core. Found old metrics folder size is current size of the victoria-metrics directory in GB.Minimum metrics_data_size required to proceed is the minimum possible size of the metrics volume for migration from version 3.4.x. This size includes a safety margin of 2Gi of free space. If necessary, you can filter the output by the Minimum metrics_data_size required to proceed string.Based on the results, you need to determine the required size of the Metrics service volume in KUMA 4.0.x, depending on your needs. The Metrics service volume size must not be less than the Minimum metrics_data_size required to proceed value.
When upgrading from version 3.4.x to version 4.0.x, if you specify a volume size less than the Minimum metrics_data_size required to proceed value in the inventory file k0s.inventory.yml, or if you specify the metrics_disk_request value that is less than the Minimum metrics_data_size required to proceed value, the installer aborts the upgrade at the installation validation stage with the following message:
TASK [Check metrics volume size requirement] ***********************************
fatal: [kuma-inst-06.avp.ru]: FAILED! => {
"changed": false
}
MSG:
Current metrics folder size: 20Gi
Metrics disk request when low_resources=false: 10Gi
Metrics disk request when low_resources=true: 10Gi
Error: Available metric data exceeds specified volume size.
The available metric data (20Gi + 2Gi threshold) is too large.
Solution: Increase the metrics volume size by updating the metrics_disk_request variable in your inventory file.
In the above example, the user tried to set the volume size to 10Gi through the metrics_disk_request inventory variable, while the current size of metrics data on the disk is 20Gi. The intended volume size is less than the current data size + the 2Gi safety margin, therefore the upgrade cannot be performed. The installation is aborted at the validation step, and no changes are made to the current installation. You can safely choose a suitable volume size for metrics, override it in the inventory file, and retry the upgrade.
Page top