Aggregation rules

Aggregation rules let you combine repetitive events of the same type and replace them with one common event. Aggregation rules support fields of the standard KUMA event schema as well as fields of the extended event schema. In this way, you can reduce the number of similar events sent to the storage and/or the correlator, reduce the workload on services, conserve data storage space and licensing quota (EPS). An aggregation event is created when a time or number of events threshold is reached, whichever occurs first.

For aggregation rules, you can configure a filter and apply it only to events that match the specified conditions.

You can configure aggregation rules under Resources → Aggregation rules, and then select the created aggregation rule from the drop-down list in the collector settings. You can also configure aggregation rules directly in collector settings. Available aggregation rule settings are listed in the table below.

Available aggregation rule settings

Setting		Description

Name	Unique name of the resource. Maximum length of the name: 128 Unicode characters. Required setting.
Tenant	The name of the tenant that owns the resource. Required setting.
Threshold	Threshold on the number of events. After accumulating the specified number of events with identical fields, the collector creates an aggregation event and begins accumulating events for the next aggregated event. The default value is `100`.
Triggered rule lifetime	Threshold on time in seconds. When the specified time expires, the accumulation of base events stops, the collector creates an aggregated event and starts obtaining events for the next aggregated event. The default value is `60`. Required setting.
Description	Description of the resource. Maximum length of the description: 4000 Unicode characters.
Identical fields	Fields of normalized events whose values must match. For example, for network events, `SourceAddress`, `DestinationAddress`, and `DestinationPort` normalized event fields can be used. In the aggregation event, these normalized event fields are populated with the values of the base events. Required setting.
Unique fields	Fields whose range of values must be preserved in in the aggregated event. For example, if the `DestinationPort` field is specified under Unique fields and not Identical fields, the aggregated event combines base connection events for a variety of ports, and the `DestinationPort` field of the aggregated event contains a list of all ports to which connections were made.
Sum fields	Fields whose values are summed up during aggregation and written to the same-name fields of the aggregated event. The following special considerations are relevant to field behavior: The values of fields of the "Number" and "Float" types are summed up. The values of fields of the "String" type are concatenated with commas added as separators. The values of fields with the types "Array of strings", "Array of numbers" and "Array of floats" are appended to the end of the array.
Filter	Conditions for determining which events must be processed by the resource. In the drop-down list, you can select an existing filter Create new to create a new filter. In aggregation rules, do not use filters with the TI operand or the TIDetect, inActiveDirectoryGroup, or hasVulnerability operators. The `Active Directory` fields for which you can use the inActiveDirectoryGroup operator will appear during the enrichment stage (after aggregation rules are executed). Creating a filter in resources To create a filter: In the Filter drop-down list, select Create new. If you want to keep the filter as a separate resource, select the Save filter check box. In this case, you will be able to use the created filter in various services. This check box is cleared by default. If you selected the Save filter check box, enter a name for the created filter resource in the Name field. Maximum length of the name: 128 Unicode characters. In the Conditions settings block, specify the conditions that the events must meet: Click the Add condition button. In the Left operand and Right operand drop-down lists, specify the search parameters. Depending on the data source selected in the Right operand field, you may see fields of additional parameters for identifying the value to be passed to the filter. For example, when you select active list, you must specify the name of the active list, the entry key, and the entry key field. In the operator drop-down list, select an operator. Filter operators =—the left operand equals the right operand. <—the left operand is less than the right operand. <=—the left operand is less than or equal to the right operand. >—the left operand is greater than the right operand. >=—the left operand is greater than or equal to the right operand. inSubnet—the left operand (IP address) is in the subnet of the right operand (subnet). contains—the left operand contains values of the right operand. startsWith—the left operand starts with one of the values of the right operand. endsWith—the left operand ends with one of the values of the right operand. match—the left operand matches the regular expression of the right operand. The RE2 regular expressions are used. hasBit—checks whether the left operand (string or number) contains bits whose positions are listed in the right operand (in a constant or in a list). The value to be checked is converted to binary and processed right to left. Chars are checked whose index is specified as a constant or a list. If the value being checked is a string, then an attempt is made to convert it to integer and process it in the way described above. If the string cannot be converted to a number, the filter returns `False`. hasVulnerability—checks whether the left operand contains an asset with the vulnerability and vulnerability severity specified in the right operand. If you do not specify the ID and severity of the vulnerability, the filter is triggered if the asset in the event being checked has any vulnerability. inActiveList—this operator has only one operand. Its values are selected in the Key fields field and are compared with the entries in the active list selected from the Active List drop-down list. inDictionary—checks whether the specified dictionary contains an entry defined by the key composed with the concatenated values of the selected event fields. inCategory—the asset in the left operand is assigned at least one of the asset categories of the right operand. inActiveDirectoryGroup—the Active Directory account in the left operand belongs to one of the Active Directory groups in the right operand. TIDetect—this operator is used to find events using CyberTrace Threat Intelligence (TI) data. This operator can be used only on events that have completed enrichment with data from CyberTrace Threat Intelligence. In other words, it can only be used in collectors at the destination selection stage and in correlators. inContextTable—presence of the entry in the specified context table. intersect—presence in the left operand of the list items specified in the right operand. If you want the operator to be case-insensitive, select the do not match case check box. The selection of this check box does not apply to the InSubnet, InActiveList, InCategory or InActiveDirectoryGroup operators. This check box is cleared by default. If you want to add a negative condition, select If not from the If drop-down list. You can add multiple conditions or a group of conditions. If you have added multiple conditions or groups of conditions, choose a selection condition (and, or, not) by clicking the AND button. If you want to add existing filters that are selected from the Select filter drop-down list, click the Add filter button. You can view the nested filter settings by clicking the button.

The KUMA distribution kit includes aggregation rules listed in the table below.

Predefined aggregation rules

Aggregation rule name

Description

[OOTB] Netflow 9

The rule is triggered after 100 events or 10 seconds.

Events are aggregated by the following fields:

DestinationAddress
DestinationPort
SourceAddress
TransportProtocol
DeviceVendor
DeviceProduct

The DeviceCustomString1 and BytesIn fields are summed up.

Page top