Hardware and software requirements

Recommended hardware requirements

This section lists the hardware requirements for processing an incoming event stream in KUMA at various Events per Second (EPS) rates.

The table below lists the hardware and software requirements for installing the KUMA components, assuming that the ClickHouse cluster only accepts INSERT queries. Hardware requirements for SELECT queries are calculated separately for the particular database usage profile of the customer.

When designing the storage cluster, the EPS, the average event size, and the storage depth are not the only things you must take into account. You must also be mindful of special considerations that are presented in this article for your convenience in the following sections:

Disk subsystem
Network interface
RAM
Usage profile

Disk subsystem

Insert queries (write access) constitute the majority of the load on ClickHouse in KUMA:

Events arrive in the database in an almost constant stream.
Inserts are small and frequent by ClickHouse standards.
ClickHouse has a high write amplification figure because of the constant merging of parts of partitions in the background.
A partition part under 1 GB in size is stored as a single file. When the part reaches 1 GB, wide column format is applied, wherein each column of the table is represented by two files, .dat and .mrk. In this case, overwriting a part to merge it with another part requires writing to 384 files + fsync for each, which is a lot of IOPS.

Of course, ClickHouse simultaneously handles search queries, but there are much fewer of those than write operations. Considering the load profile, we recommend organizing the disk subsystem in the following way:

Use DAS (Direct Attached Storage) with NVME or SAS/SATA interfaces, avoiding SAN/NAS solutions.
If the nodes of the ClickHouse cluster are deployed in a virtual environment, the disk array must be directly mounted to the /opt/kaspersky/kuma directory. Avoid virtual disks.
Ideally, SSD (SATA/NVME) should be used, if possible. Any server-grade SSDs will outperform HDDs (even 15k RPM). For SSDs, use RAID-10 or RAID-0, for RAID-0, make sure that replication in ClickHouse is used.
If SSDs cannot be used, use 15k-RPM SAS HDDs in RAID-10.
A hybrid option is possible: storing hot data (for example, for the last month) on SSDs, and cold data on HDDs. This arrangement makes sure the SSD arrays always handle the write operations.
A software RAID array (madm) will always outperform a hardware RAID controller and offer more flexibility. When using RAID-0 or RAID-10, it is important that the stripe size be 1 MB. In hardware RAID controllers, the default stripe size is often 64 KB, and the maximum supported value is 256–512 KB. For RAID-10, near-layout is optimal for writing, and far-layout is optimal for reading. You must keep this in mind when using a hybrid SSD + HDD configuration.
The recommended file systems are EXT4 or XFS, preferably mounted with the noatime option.

Network interface

On each node of the cluster, a network interface with a bandwidth of at least 10 Gbps must be installed and appropriately switched. Considering write amplification and replication, 25 Gbps is ideal.
If necessary, replication processes can use a separate network interface that does not handle insert and search traffic.

RAM

ClickHouse recommends keeping RAM to stored data ratio at 1:100.
But how much RAM is needed if each node is supposed to store 50 TB, given that the event storage depth is often dictated by regulatory requirements, while search queries are not performed to the full storage depth? Therefore, the recommendation can be interpreted as follows: if the average query depth involves scanning partitions with a total volume of 10 TB, then you need 100 GB of RAM. This is a general recommendation for broad analytical queries.
In the general case, more free RAM on the server improves the performance of search queries for the most recent events because the contents of the partition files will be in operating system's page cache, so file content will be read from RAM, and not from disk.

Usage profile

The hardware requirements are designed for the database to consume a stream of a certain EPS when the database is at rest (no search queries are being executed, only inserts). At the KUMA deployment stage, it is not always possible to accurately answer the following questions:

What search and analytic queries will be performed in the system?
What is the intensity, concurrency, and depth of search queries?
What is the actual performance of the cluster nodes?

Thus, it is perfectly normal for a KUMA ClickHouse cluster to evolve over time. If an increase of EPS or intensity/resource consumption of search queries makes the cluster unable to cope with the load, you can add more shards (horizontal scaling) or improve the disk subsystems/CPU/RAM on the cluster nodes.

The configuration of the equipment must be chosen based on the system load profile. You can use the "All-in-one" configuration for an event stream of under 10,000 EPS and when using graphical panels supplied with the system.

KUMA supports Intel and AMD CPUs with SSE 4.2 and AVX instruction set support.

	Up to 3,000 EPS	Up to 10,000 EPS	Up to 20,000 EPS	Up to 50,000 EPS

Configuration	Installation on a single server One device. Device characteristics: At least 16 threads or vCPUs. At least 32 GB of RAM. At least 500 GB in the /opt directory. Data storage type: SSD*. Data transfer rate: at least 100 Mbps.	Installation on a single server One device. Device characteristics: At least 24 threads or vCPUs. At least 64 GB of RAM. At least 500 GB in the /opt directory. Data storage type: SSD*. Data transfer rate: at least 100 Mbps.	1 server for the Core + 1 server for the Collector + 1 server for the Correlator + 3 dedicated servers with the Keeper role + 2 servers for the Storage* *Recommended configuration. 2 Storage servers are used when ClickHouse is configured with 2 replicas in each shard to ensure fault tolerance and high availability of events collected in the Storage. If fault tolerance requirements do not apply to the Storage, a ClickHouse configuration with 1 replica in each shard may be used and, accordingly, 1 server may be used for the Storage.	1 server for the Core + 2 servers for the Collector + 1 server for the Correlator + 3 dedicated servers with the Keeper role + 4 servers for the Storage* *Recommended configuration. 4 Storage servers are used when ClickHouse is configured with 2 replicas in each shard to ensure fault tolerance and high availability of events collected in the Storage. If fault tolerance requirements do not apply to the Storage, a ClickHouse configuration with 1 replica in each shard may be used and, accordingly, 2 servers may be used for the Storage.
Requirements for the Core component	-	-	One device. Device characteristics: At least 10 threads or vCPUs. At least 24 GB of RAM. At least 500 GB in the /opt directory. Data storage type: SSD. Data transfer rate: at least 100 Mbps.	One device. Device characteristics: At least 10 threads or vCPUs. At least 24 GB of RAM. At least 500 GB in the /opt directory. Data storage type: SSD. Data transfer rate: at least 100 Mbps.
Requirements for the Collector component	-	-	One device. Device characteristics: At least 8 threads or vCPUs. At least 16 GB of RAM. At least 500 GB in the /opt directory. Data storage type: HDD allowed. Data transfer rate: at least 100 Mbps.	Two devices. Characteristics of each device: At least 8 threads or vCPUs. At least 16 GB of RAM. At least 500 GB in the /opt directory. Data storage type: HDD allowed. Data transfer rate: at least 100 Mbps.
Requirements for the Correlator component	-	-	One device. Device characteristics: At least 8 threads or vCPUs. At least 32 GB of RAM. At least 500 GB in the /opt directory. Data storage type: HDD allowed. Data transfer rate: at least 100 Mbps.	One device. Device characteristics: At least 8 threads or vCPUs. At least 32 GB of RAM. At least 500 GB in the /opt directory. Data storage type: HDD allowed. Data transfer rate: at least 100 Mbps.
Requirements for the Keeper component	-	-	Three devices. Characteristics of each device: At least 6 threads or vCPUs. At least 12 GB of RAM. At least 50 GB in the /opt directory. Data storage type: SSD. Data transfer rate: at least 100 Mbps.	Three devices. Characteristics of each device: At least 6 threads or vCPUs. At least 12 GB of RAM. At least 50 GB in the /opt directory. Data storage type: SSD. Data transfer rate: at least 100 Mbps.
Requirements for the Storage component	-	-	Two devices. Characteristics of each device: At least 24 threads or vCPUs. At least 64 GB of RAM. At least 500 GB in the /opt directory. Data storage type: SSD*. The recommended transfer rate between ClickHouse nodes is at least 10 Gbps if the data stream is equal to or exceeds 20,000 EPS.	Four devices. Characteristics of each device: At least 24 threads or vCPUs. At least 64 GB of RAM. At least 500 GB in the /opt directory. Data storage type: SSD*. The recommended transfer rate between ClickHouse nodes is at least 10 Gbps if the data stream is equal to or exceeds 20,000 EPS.
Operating systems	Ubuntu 22.04 LTS Oracle Linux 8.6, 8.7, 9.2, 9.4 Astra Linux Special Edition RUSB.10015-01 (2021-1126SE17 update 1.7.1). Astra Linux Special Edition RUSB.10015-01 (2022-1011SE17MD update 1.7.2.UU.1). Astra Linux Special Edition RUSB.10015-01 (2022-1110SE17 update 1.7.3). Core version 5.15.0.33 or higher is required. Astra Linux Special Edition RUSB.10015-01 (2023-0630SE17MD, update 1.7.4.UU.1). Astra Linux Special Edition RUSB.10015-01 (2024-0212SE17MD, update 1.7.5.UU.1). Astra Linux Special Edition RUSB.10015-01 (2024-0830SE17, update 1.7.6). RED OS 7.3.4, 8
TLS ciphersuites	TLS versions 1.2 and 1.3 are supported. Integration with a server that does not support the TLS versions and ciphersuites that KUMA requires is impossible. Supported TLS 1.2 ciphersuites: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 Supported TLS 1.3 ciphersuites: TLS_AES_128_GCM_SHA256 TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256

Depending on the number and complexity of database queries made by users, reports, and dashboards, a greater amount of resources may be required.

For every 50,000 (above 50,000) assets, you must add 2 extra threads or vCPUs and 4 GB of RAM to the resources of the Core component.

For every 100 (above 100) services managed by the Core component, you must add 2 additional threads or vCPUs to the resources of the Core component.

ClickHouse must be deployed on solid-state drives (SSD). SSDs help improve data access speed.

* If the system usage profile does not involve running aggregation SQL queries with a great storage depth, you can use HDD arrays (15,000-RPM SAS HDDs in RAID-10).

Hard drives can be used to store data using the HDFS technology.

Exported events are written to the drive of the Core component to the /opt/kaspersky/kuma/core/tmp/ temporary folder. The exported data is stored for 10 days and then automatically deleted. If you plan to export a large amount of events, you must allocate additional space.

Working in virtual environments

Installation of KUMA is supported in the following virtual environments:

VMware 6.5 or later
Hyper-V for Windows Server 2012 R2 or later
QEMU-KVM 4.2 or later
Software package of virtualization tools "Brest" RDTSP.10001-02

Resource recommendations for the Collector component

Consider that for event processing efficiency, the CPU core count is more important than the clock rate. For example, eight CPU cores with a medium clock rate can process events more efficiently than four CPU cores with a high clock rate.

Consider also that the amount of RAM utilized by the collector depends on configured enrichment methods (DNS, accounts, assets, enrichment with data from Kaspersky CyberTrace) and whether aggregation is used (RAM consumption is influenced by the data aggregation window setting, the number of fields used for aggregation of data, volume of data in fields being aggregated). The utilization of computation resources by KUMA depends on the type of events being parsed and the efficiency of the normalizer.

For example, with an event stream of 1,000 EPS and event enrichment disabled (event enrichment is disabled, event aggregation is disabled, 5,000 accounts, 5,000 assets per tenant), one collector requires the following resources:

• 1 CPU core or 1 virtual CPU

• 512 MB of RAM

• 1 GB of disk space (not counting event cache)

For example, to support 5 collectors that do not perform event enrichment, you must allocate the following resources: 5 CPU cores, 2.5 GB of RAM, and 5 GB of free disk space.

Kaspersky recommendations for storage servers

To connect a data storage system to storage servers, you must use high-speed protocols, such as Fibre Channel or iSCSI 10G. We do not recommend using application-level protocols such as NFS and SMB to connect data storage systems.

On ClickHouse cluster servers, we recommend using the ext4 file system.

If you are using RAID arrays, it is recommended to use RAID 0 for high performance, or RAID 10 for high performance and high availability.

To ensure high availability and performance of the data storage subsystem, we recommend making sure that ClickHouse nodes are deployed strictly on different disk arrays.

If you are using a virtualized infrastructure to host system components, we recommend deploying ClickHouse cluster nodes on different hypervisors. In this case, it is necessary to prevent two virtual machines with ClickHouse from working on the same hypervisor.

For high-load KUMA installations, we recommend installing ClickHouse on physical servers.

Requirements for devices for installing agents

To have data sent to the KUMA collector, you must install agents on the network infrastructure devices. Device requirements are listed in the following table.

	Windows devices	Linux devices
CPU	Single-core, 1.4 GHz or higher	Single-core, 1.4 GHz or higher
RAM	512 MB	512 MB
Free disk space	1 GB	1 GB
Operating systems	Microsoft® Windows® 2012 Microsoft® Windows® 2012 has reached end of life, therefore this operating system is supported in a limited way. Microsoft Windows Server® 2012 R2 Microsoft Windows Server 2016 Microsoft Windows Server 2019 Microsoft Windows 10 20H2, 21H1	Oracle® Linux 8.6, 8.7, 9.2. Astra Linux Special Edition RUSB.10015-01 (2021-1126SE17 update 1.7.1). Astra Linux Special Edition RUSB.10015-01 (2022-1011SE17MD update 1.7.2.UU.1). Astra Linux Special Edition RUSB.10015-01 (2022-1110SE17 update 1.7.3). Astra Linux Special Edition RUSB.10015-01 (2023-0630SE17MD, update 1.7.4.UU.1).

Requirements for client devices for managing the KUMA web interface

CPU: Intel® Core™ i3 8th generation

RAM: 8 GB

Supported browsers:

Google™ Chrome™ 110 or later.
Mozilla™ Firefox™ 110 or later.

Device requirements for installing KUMA on Kubernetes

The minimum configuration of a Kubernetes cluster for deployment of a high availability KUMA configuration includes the following:

1 load balancer node (not part of the cluster).
3 controller nodes.
2 worker nodes.

The minimum hardware requirements for devices for installing KUMA on Kubernetes are listed in the table below.

	Balancer	Controller	Worker node
CPU	1 core with 2 threads or 2 vCPUs.	1 core with 2 threads or 2 vCPUs.	12 threads or 12 vCPUs.
RAM	At least 2 GB	At least 2 GB	At least 24 GB
Free disk space	At least 30 GB	At least 30 GB	At least 1 TB in the /opt directory. At least 32 GB in the /var/lib directory.
Network bandwidth	10 Gbps	10 Gbps	10 Gbps

Page top