Enrichment in the normalizer

Expand all | Collapse all

When creating event parsing rules in the normalizer settings window, on the Enrichment tab, you can configure the rules for adding extra data to the fields of the normalized event using enrichment rules. These enrichment rules are stored in the settings of the normalizer where they were created.

Enrichments are created by clicking the Add enrichment button. There can be more than one enrichment rule. You can delete enrichment rules by clicking the button.

Settings available in the enrichment rule settings block:

• Source kind (required)—drop-down list for selecting the type of enrichment. Depending on the selected type, you may see advanced settings that will also need to be completed.

  Available Enrichment rule source types:

  • constant

    This type of enrichment is used when a constant needs to be added to an event field. Available enrichment type settings are listed in the table below.

    Available enrichment type settings

    Setting	Description
    Constant	The value to be added to the event field. Maximum length of the value: 255 Unicode characters. If you leave this field blank, the existing event field value is removed.
    Target field	The KUMA event field that you want to populate with the data.

    If you are using the event enrichment functions for extended schema fields of String, Number, or Float type with a constant, the constant is added to the field.

    If you are using the event enrichment functions for extended schema fields of Array of strings, Array of numbers, or Array of floats type with a constant, the constant is added to the elements of the array.

  • dictionary

    This type of enrichment is used if you need to add a value from the dictionary of the Dictionary type. Available enrichment type settings are listed in the table below.

    Available enrichment type settings

    Setting	Description
    Dictionary name	The dictionary from which the values are to be taken.
    Key fields	Event fields whose values are to be used for selecting a dictionary entry. To add an event field, click Add field. You can add multiple event fields.

    If you are using event enrichment with the dictionary type selected as the Source kind setting, and an array field is specified in the Key enrichment fields setting, when an array is passed as the dictionary key, the array is serialized into a string in accordance with the rules of serializing a single value in the TSV format.

    Example: The Key fields setting of the enrichment uses the SA.StringArrayOne extended schema field. The SA.StringArrayOne extended schema field contains the values "a", "b", "c". The following values are passed to the dictionary as the key: ['a','b','c'].

    If the Key enrichment fields setting uses an array extended schema field and a regular event schema field, the field values are separated by the "|" character when the dictionary is queried.

    Example: The Key enrichment fields setting uses the SA.StringArrayOne extended schema field and the Code string field. The SA.StringArrayOne extended schema field contains the values "a", "b", "c", and the Code string field contains the myCode sequence of characters. The following values are passed to the dictionary as the key: ['a','b','c']|myCode.

  • table

    This type of enrichment is used if you need to add a value from the dictionary of the Table type.

    When this enrichment type is selected in the Dictionary name drop-down list, select the dictionary for providing the values. In the Key fields group of settings, click the Add field button to select the event fields whose values are used for dictionary entry selection.

    In the Mapping table, configure the dictionary fields to provide data and the event fields to receive data:

    • In the Dictionary field column, select the dictionary field. The available fields depend on the selected dictionary resource.
    • In the KUMA field column, select the event field to which the value is written. For some of the selected fields (*custom* and *flex*), in the Label column, you can specify a name for the data written to them.

    New table rows can be added by clicking the Add new element button. Columns can be deleted by clicking the button.

  • event

    This type of enrichment is used when you need to write a value from another event field to the current event field. Available enrichment type settings are listed in the table below.

    Available enrichment type settings

    Setting	Description
    Target field	The KUMA event field that you want to populate with the data.
    Source field	The event field whose value is written to the target field.

    Clicking opens the Conversion window, in which you can click Add conversion to create rules for modifying the source data before writing them to the KUMA event fields. You can reorder and delete created rules. To change the position of a rule, click next to it. To delete a rule, click next to it.

    Available conversions

    Conversions are modifications that are applied to a value before it is written to the event field. You can select one of the following conversion types from the drop-down list:

    • entropy is used for converting the value of the source field using the information entropy calculation function and placing the conversion result in the target field of the float type. The result of the conversion is a number. Calculating the information entropy allows detecting DNS tunnels or compromised passwords, for example, when a user enters the password instead of the login and the password gets logged in plain text.
    • lower—is used to make all characters of the value lowercase
    • upper—is used to make all characters of the value uppercase
    • regexp – used to convert a value using a specified RE2 regular expression. When you select this type of conversion, a field is displayed in which you must specify the RE2 regular expression.
    • substring is used to extract characters in a specified range of positions. When you select this type of conversion, the Start and End fields are displayed, in which you must specify the range of positions.
    • replace—is used to replace specified character sequence with the other character sequence. When you select this type of conversion, the following fields are displayed:
      • Replace chars specifies the sequence of characters to be replaced.
      • With chars is the character sequence to be used instead of the character sequence being replaced.
    • trim removes the specified characters from the beginning and from the end of the event field value. When you select this type of conversion, the Chars field is displayed in which you must specify the characters. For example, if a trim conversion with the Micromon value is applied to Microsoft-Windows-Sysmon, the new value is soft-Windows-Sys.
    • append appends the specified characters to the end of the event field value. When you select this type of conversion, the Constant field is displayed in which you must specify the characters.
    • prepend prepends the specified characters to the beginning of the event field value. When you select this type of conversion, the Constant field is displayed in which you must specify the characters.
    • replace with regexp is used to replace RE2 regular expression results with the specified character sequence. When you select this type of conversion, the following fields are displayed:
      • Expression is the RE2 regular expression whose results you want to replace.
      • With chars is the character sequence to be used instead of the character sequence being replaced.
    • Converting encoded strings to text:
      • decodeHexString—used to convert a HEX string to text.
      • decodeBase64String—used to convert a Base64 string to text.
      • decodeBase64URLString—used to convert a Base64url string to text.

      When converting a corrupted string or if conversion error occur, corrupted data may be written to the event field.

      During event enrichment, if the length of the encoded string exceeds the size of the field of the normalized event, the string is truncated and is not decoded.

      If the length of the decoded string exceeds the size of the event field into which the decoded value is to be written, the string is truncated to fit the size of the event field.

    Conversions when using the extended event schema

    Whether or not a conversion can be used depends on the type of extended event schema field being used:

    • For an additional field of the String type, all types of conversions are available.
    • For fields of the Number and Float types, the following types of conversions are available: regexp, substring, replace, trim, append, prepend, replaceWithRegexp, decodeHexString, decodeBase64String, and decodeBase64URLString.
    • For fields of Array of strings, Array of numbers, and Array of floats types, the following types of conversions are available: append and prepend.

    When using enrichment of events that have event selected as the Source kind and the extended event schema fields are used as arguments, the following special considerations apply:

    • If the source extended event schema field has the Array of strings type, and the target extended event schema field has the String type, the values are written to the target extended event schema field in TSV format.

      Example: The SA.StringArray extended event schema field contains values: "string1", "string2", "string3". An event enrichment operation is performed. The result of the event enrichment operation is written to the DeviceCustomString1 extended event schema field. The DeviceCustomString1 extended event schema field contains values: ["string1", "string2", "string3"].

    • If the source and target extended event schema fields have the Array of strings type, values of the source extended event schema field are added to the values of the target extended event schema field, and the "," character is used as the delimiter character.

      Example: The SA.StringArrayOne field of the extended event scheme contains the ["string1", "string2", "string3"] values, and the SA.StringArrayTwo field of the extended event scheme contains the ["string4", "string5", "string6"] values. An event enrichment operation is performed. The result of the event enrichment operation is written to the SA.StringArrayTwo field of the extended event scheme. The SA.StringArrayTwo extended event schema field contains values: ["string4", "string5", "string6", "string1", "string2", "string3"].

  • template

    This type of enrichment is used when you need to write a value obtained by processing Go templates into the event field. We recommend matching the value and the size of the field. Available enrichment type settings are listed in the table below.

    Available enrichment type settings

    Setting	Description
    Template	The Go template. Event field names are passed in the {{.EventField}} format, where EventField is the name of the event field from which the value must be passed to the script, for example, {{.DestinationAddress}} attacked from {{.SourceAddress}}.
    Target field	The KUMA event field that you want to populate with the data.

    If you are using enrichment of events that have template selected as the Source kind, and in which the target field has the String type, and the source field is an extended event schema field containing an array of strings, you can use one of the following examples for the template:

    • {{.SA.StringArrayOne}}
    • {{- range $index, $element := . SA.StringArrayOne -}}

      {{- if $index}}, {{end}}"{{$element}}"{{- end -}}

    To convert the data in an array field in a template into the TSV format, use the toString function, for example:

    template {{toString .SA.StringArray}}

• Target field (required)—drop-down list for selecting the KUMA event field that should receive the data.

  This setting is not available for the enrichment source of the Table type.

Page top