Step 4. Enrichment

This is an optional step of the Installation Wizard. On the Enrichment tab of the Installation Wizard, you can select or create a enrichment rules and indicate which data from which sources you want to add to correlation events that the correlator creates. There can be more than one enrichment rule. You can add them by clicking the Add button and can remove them by clicking the button.

To add an existing enrichment rule to a set of resources:

1. Click Add.

   This opens the enrichment rule settings block.

2. In the Enrichment rule drop-down list, select the relevant resource.

The enrichment rule is added to the set of resources for the correlator.

To create a new enrichment rule in a set of resources:

1. Click Add.

   This opens the enrichment rule settings block.

2. In the Enrichment rule drop-down list, select Create new.
3. In the Source kind drop-down list, select the source of data for enrichment and define its corresponding settings:
   • constant

     This type of enrichment is used when a constant needs to be added to an event field. Available enrichment type settings are listed in the table below.

     Available enrichment type settings

     Setting	Description
     Constant	The value to be added to the event field. Maximum length of the value: 255 Unicode characters. If you leave this field blank, the existing event field value is removed.
     Target field	The KUMA event field that you want to populate with the data.

     If you are using the event enrichment functions for extended schema fields of "String", "Number", or "Float" type with a constant, the constant is added to the field.

     If you are using the event enrichment functions for extended schema fields of "Array of strings", "Array of numbers", or "Array of floats" type with a constant, the constant is added to the elements of the array.

   • dictionary

     This type of enrichment is used if you need to add a value from the dictionary of the Dictionary type. Available enrichment type settings are listed in the table below.

     Available enrichment type settings

     Setting	Description
     Dictionary name	The dictionary from which the values are to be taken.
     Key fields	Event fields whose values are to be used for selecting a dictionary entry. To add an event field, click Add field. You can add multiple event fields.

     If you are using event enrichment with the dictionary type selected as the Source kind setting, and an array field is specified in the Key enrichment fields setting, when an array is passed as the dictionary key, the array is serialized into a string in accordance with the rules of serializing a single value in the TSV format.

     Example: The Key fields setting of the enrichment uses the SA.StringArrayOne extended schema field. The SA.StringArrayOne extended schema field contains the values "a", "b", "c". The following values are passed to the dictionary as the key: ['a','b','c'].

     If the Key enrichment fields setting uses an array extended schema field and a regular event schema field, the field values are separated by the "|" character when the dictionary is queried.

     Example: The Key enrichment fields setting uses the SA.StringArrayOne extended schema field and the Code string field. The SA.StringArrayOne extended schema field contains the values "a", "b", "c", and the Code string field contains the myCode sequence of characters. The following values are passed to the dictionary as the key: ['a','b','c']|myCode.

   • event

     This type of enrichment is used when you need to write a value from another event field to the current event field. Settings of this type of enrichment:

     • In the Target field drop-down list, select the KUMA event field to which you want to write the data.
     • In the Source field drop-down list, select the event field whose value will be written to the target field.
     • In the Conversion settings block, you can create rules for modifying the original data before it is written to the KUMA event fields. The conversion type can be selected from the drop-down list. You can use the Add conversion and Delete buttons to add or delete a conversion, respectively. The order of conversions is important.

       Available conversions

       Conversions are modifications that are applied to a value before it is written to the event field. You can select one of the following conversion types from the drop-down list:

       • entropy is used for converting the value of the source field using the information entropy calculation function and placing the conversion result in the target field of the float type. The result of the conversion is a number. Calculating the information entropy allows detecting DNS tunnels or compromised passwords, for example, when a user enters the password instead of the login and the password gets logged in plain text.
       • lower—is used to make all characters of the value lowercase
       • upper—is used to make all characters of the value uppercase
       • regexp – used to convert a value using a specified RE2 regular expression. When you select this type of conversion, a field is displayed in which you must specify the RE2 regular expression.
       • substring is used to extract characters in a specified range of positions. When you select this type of conversion, the Start and End fields are displayed, in which you must specify the range of positions.
       • replace—is used to replace specified character sequence with the other character sequence. When you select this type of conversion, the following fields are displayed:
         • Replace chars specifies the sequence of characters to be replaced.
         • With chars is the character sequence to be used instead of the character sequence being replaced.
       • trim removes the specified characters from the beginning and from the end of the event field value. When you select this type of conversion, the Chars field is displayed in which you must specify the characters. For example, if a trim conversion with the Micromon value is applied to Microsoft-Windows-Sysmon, the new value is soft-Windows-Sys.
       • append appends the specified characters to the end of the event field value. When you select this type of conversion, the Constant field is displayed in which you must specify the characters.
       • prepend prepends the specified characters to the beginning of the event field value. When you select this type of conversion, the Constant field is displayed in which you must specify the characters.
       • replace with regexp is used to replace RE2 regular expression results with the specified character sequence. When you select this type of conversion, the following fields are displayed:
         • Expression is the RE2 regular expression whose results you want to replace.
         • With chars is the character sequence to be used instead of the character sequence being replaced.
       • Converting encoded strings to text:
         • decodeHexString—used to convert a HEX string to text.
         • decodeBase64String—used to convert a Base64 string to text.
         • decodeBase64URLString—used to convert a Base64url string to text.

         When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

         During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

         If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, such a string is truncated to fit the size of the event field.

       Conversions when using the extended event schema

       Whether or not a conversion can be used depends on the type of extended event schema field being used:

       • For an additional field of the "String" type, all types of conversions are available.
       • For fields of the "Number" and "Float" types, the following types of conversions are available: regexp, substring, replace, trim, append, prepend, replaceWithRegexp, decodeHexString, decodeBase64String, and decodeBase64URLString.
       • For fields of "Array of strings", "Array of numbers", and "Array of floats" types, the following types of conversions are available: append and prepend.

   • template

     This type of enrichment is used when you need to write the result of processing Go templates into the event field. We recommend matching the value and the size of the field. Available enrichment type settings are listed in the table below.

     Available enrichment type settings

     Setting	Description
     Template	The Go template. Event field names are passed in the {{.EventField}} format, where EventField is the name of the event field from which the value must be passed to the script, for example, {{.DestinationAddress}} attacked from {{.SourceAddress}}.
     Target field	The KUMA event field that you want to populate with the data.

     If you are using enrichment of events that have template selected as the Source kind, and in which the target field has the "String" type, and the source field is an extended event schema field containing an array of strings, you can use one of the following examples for the template:

     • {{.SA.StringArrayOne}}
     • {{- range $index, $element := . SA.StringArrayOne -}}

       {{- if $index}}, {{end}}"{{$element}}"{{- end -}}

     To convert the data in an array field in a template into the TSV format, use the toString function, for example:

     template {{toString.SA.StringArray}}

   • dns

     This type of enrichment is used to send requests to a private network DNS server to convert IP addresses into domain names or vice versa. IP addresses are converted to DNS names only for private addresses: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 100.64.0.0/10.

     Available settings:

     • URL—in this field, you can specify the URL of a DNS server to which you want to send requests. You can use the Add URL button to specify multiple URLs.
     • RPS—maximum number of requests sent to the server per second. The default value is 1,000.
     • Workers—maximum number of requests per one point in time. The default value is 1.
     • Max tasks—maximum number of simultaneously fulfilled requests. By default, this value is equal to the number of vCPUs of the KUMA Core server.
     • Cache TTL—the lifetime of the values stored in the cache. The default value is 60.
     • Cache disabled—you can use this drop-down list to enable or disable caching. Caching is enabled by default.
   • cybertrace

     This type of enrichment is deprecated, we recommend using cybertrace-http instead.

     This type of enrichment is used to add information from CyberTrace data streams to event fields.

     Available settings:

     • URL (required)—in this field, you can specify the URL of a CyberTrace server to which you want to send requests.
     • Number of connections—maximum number of connections to the CyberTrace server that can be simultaneously established by KUMA. By default, this value is equal to the number of vCPUs of the KUMA Core server.
     • RPS—maximum number of requests sent to the server per second. The default value is 1,000.
     • Timeout—amount of time to wait for a response from the CyberTrace server, in seconds. The default value is 30.
     • Maximum number of events in the enrichment queue—maximum number of events stored in the enrichment queue for re-sending. The default value is 1,000,000,000.
     • Mapping (required)—this settings block contains the mapping table for mapping KUMA event fields to CyberTrace indicator types. The KUMA field column shows the names of KUMA event fields, and the CyberTrace indicator column shows the types of CyberTrace indicators.

       Available types of CyberTrace indicators:

       • ip
       • url
       • hash

       In the mapping table, you must provide at least one string. You can use the Add row button to add a string, and can use the button to remove a string.

   • cybertrace-http

     This is a new streaming event enrichment type in CyberTrace that allows you to send a large number of events with a single request to the CyberTrace API. Recommended for systems with a lot of events. Cybertrace-http outperforms the previous 'cybertrace' type, which is still available in KUMA for backward compatibility.

     Limitations:

     • The cybertrace-http enrichment type cannot be used for retroscan in KUMA.
     • If the cybertrace-http enrichment type is being used, detections are not saved in CyberTrace history in the Detections window.

     Available settings:

     • URL (required)—in this field, you can specify the URL of a CyberTrace server to which you want to send requests.
     • Secret (required) is a drop-down list in which you can select the secret which stores the credentials for the connection.
     • Timeout—amount of time to wait for a response from the CyberTrace server, in seconds. The default value is 30.
     • Key fields (required) is the list of event fields used for enriching events with data from CyberTrace.
     • Maximum number of events in the enrichment queue—maximum number of events stored in the enrichment queue for re-sending. The default value is 1,000,000,000. After reaching 1 million events received from the CyberTrace server, events stop being enriched until the number of received events is reduced to less than 500,000.
   • timezone

     This type of enrichment is used in collectors and correlators to assign a specific timezone to an event. Timezone information may be useful when searching for events that occurred at unusual times, such as nighttime.

     When this type of enrichment is selected, the required timezone must be selected from the Timezone drop-down list.

     Make sure that the required time zone is set on the server hosting the enrichment-utilizing service. For example, you can do this by using the timedatectl list-timezones command, which shows all time zones that are set on the server. For more details on setting time zones, please refer to your operating system documentation.

     When an event is enriched, the time offset of the selected timezone relative to Coordinated Universal Time (UTC) is written to the DeviceTimeZone event field in the +-hh:mm format. For example, if you select the Asia/Yekaterinburg timezone, the value +05:00 will be written to the DeviceTimeZone field. If the enriched event already has a value in the DeviceTimeZone field, it will be overwritten.

     By default, if the timezone is not specified in the event being processed and enrichment rules by timezone are not configured, the event is assigned the timezone of the server hosting the service (collector or correlator) that processes the event. If the server time is changed, the service must be restarted.

     Permissible time formats when enriching the DeviceTimeZone field

     When processing incoming raw events in the collector, the following time formats can be automatically converted to the +-hh:mm format:

     Time format in a processed event	Example
     +-hh:mm	-07:00
     +-hhmm	-0700
     +-hh	-07

     If the date format in the DeviceTimeZone field differs from the formats listed above, the collector server timezone is written to the field when an event is enriched with timezone information. You can create custom normalization rules for non-standard time formats.

4. Use the Debug toggle switch to indicate whether or not to enable logging of service operations. Logging is disabled by default.
5. In the Filter section, you can specify conditions to identify events that will be processed using the enrichment rule. You can select an existing filter from the drop-down list or create a new filter.

   Creating a filter in resources

   To create a filter:

   1. In the Filter drop-down list, select Create new.
   2. If you want to keep the filter as a separate resource, select the Save filter check box. In this case, you will be able to use the created filter in various services. This check box is cleared by default.
   3. If you selected the Save filter check box, enter a name for the created filter resource in the Name field. Maximum length of the name: 128 Unicode characters.
   4. In the Conditions settings block, specify the conditions that the events must meet:
      1. Click the Add condition button.
      2. In the Left operand and Right operand drop-down lists, specify the search parameters. Depending on the data source selected in the Right operand field, you may see fields of additional parameters for identifying the value to be passed to the filter. For example, when you select active list, you must specify the name of the active list, the entry key, and the entry key field.
      3. In the operator drop-down list, select an operator.

         Filter operators

         • =—the left operand equals the right operand.
         • <—the left operand is less than the right operand.
         • <=—the left operand is less than or equal to the right operand.
         • >—the left operand is greater than the right operand.
         • >=—the left operand is greater than or equal to the right operand.
         • inSubnet—the left operand (IP address) is in the subnet of the right operand (subnet).
         • contains—the left operand contains values of the right operand.
         • startsWith—the left operand starts with one of the values of the right operand.
         • endsWith—the left operand ends with one of the values of the right operand.
         • match—the left operand matches the regular expression of the right operand. The RE2 regular expressions are used.
         • hasBit—checks whether the left operand (string or number) contains bits whose positions are listed in the right operand (in a constant or in a list).

           The value to be checked is converted to binary and processed right to left. Chars are checked whose index is specified as a constant or a list.

           If the value being checked is a string, then an attempt is made to convert it to integer and process it in the way described above. If the string cannot be converted to a number, the filter returns False.

         • hasVulnerability—checks whether the left operand contains an asset with the vulnerability and vulnerability severity specified in the right operand.

           If you do not specify the ID and severity of the vulnerability, the filter is triggered if the asset in the event being checked has any vulnerability.

         • inActiveList—this operator has only one operand. Its values are selected in the Key fields field and are compared with the entries in the active list selected from the Active List drop-down list.
         • inDictionary—checks whether the specified dictionary contains an entry defined by the key composed with the concatenated values of the selected event fields.
         • inCategory—the asset in the left operand is assigned at least one of the asset categories of the right operand.
         • inActiveDirectoryGroup—the Active Directory account in the left operand belongs to one of the Active Directory groups in the right operand.
         • TIDetect—this operator is used to find events using CyberTrace Threat Intelligence (TI) data. This operator can be used only on events that have completed enrichment with data from CyberTrace Threat Intelligence. In other words, it can only be used in collectors at the destination selection stage and in correlators.
         • inContextTable—presence of the entry in the specified context table.
         • intersect—presence in the left operand of the list items specified in the right operand.
      4. If you want the operator to be case-insensitive, select the do not match case check box. The selection of this check box does not apply to the InSubnet, InActiveList, InCategory or InActiveDirectoryGroup operators. This check box is cleared by default.
      5. If you want to add a negative condition, select If not from the If drop-down list.

      You can add multiple conditions or a group of conditions.

   5. If you have added multiple conditions or groups of conditions, choose a selection condition (and, or, not) by clicking the AND button.
   6. If you want to add existing filters that are selected from the Select filter drop-down list, click the Add filter button. You can view the nested filter settings by clicking the button.

The new enrichment rule was added to the set of resources for the correlator.

Proceed to the next step of the Installation Wizard.

Page top