Administrator guides

This chapter provides information about using the Kaspersky CyberTrace web interface, command-line tools, and applications for SIEM solutions.

Managing Kaspersky CyberTrace Web

This section describes how Administrator users can manage Kaspersky CyberTrace from the web user interface (web interface).

All web interface pages described in this section are available only for user accounts that have the Administrator role. Users with Analyst role cannot access these pages.

Working with default credentials

This section explains how to work with default credentials and passwords for other users.

Default credentials

After Kaspersky CyberTrace is installed, the following default credentials are set for Kaspersky CyberTrace Web:

• User name: admin
• Password: CyberTrace!1

To avoid possible security risks, change this password as soon as possible. For more information, see Logging in to Kaspersky CyberTrace Web.

Resetting the password for the default user

Use the kl_access_util utility to reset the credentials for the default user (admin).

Changing passwords for other users

To create user accounts and change passwords for other users, use the User Settings page.

Service settings

This section explains how to manage different service settings that are available when you select the General tenant or a particular settings tenant in the drop-down list with all available tenants in the upper-left area of the window.

Service settings (General tenant)

You can manage the general service settings in the CyberTrace web user interface by selecting the Settings tab, and then the Service tab. Make sure that the General item is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

The Service tab allows you to edit settings stored in the kl_feed_util.conf and kl_feed_service_log.conf configuration files. You can perform the following actions by clicking the following links below the tab:

• Restart Feed Service
• Export the configuration file

  You can export the kl_feed_service.conf and kl_feed_util.conf configuration files to a directory that you choose.

• Run self-test

  Verifies that the Kaspersky Threat Data Feeds that you use work correctly.

  Please make sure you run the self-test before editing any filtering rules on the Settings > Feeds tab, in the Filtering rules for feeds section.

  If the verification test (self-test) yields incorrect results (that is, if a feed that is expected to produce detections produces none), see possible solutions for this problem in the general troubleshooting section. If the problem persists, contact your technical account manager (TAM).

• Reset statistics

  Clears the Dashboard of all the detection statistics. When you select the General tenant, Kaspersky CyberTrace clears the detection statistics for all tenants.

  It is recommended to perform this operation after successfully integrating CyberTrace with a SIEM solution: this way, the dashboard will not display any detection events generated during the verification test and will only contain real detection events, if there are any.

The Settings tab displays the Feed Service status, which can be one of the following:

• Feed Service is running
• Feed Service is starting
• Feed Service has stopped
• Feed Service is updating feeds

  This status specifies that indicators are loading into the database and indexing. Until all indicators processed, the Indicators tab may contain partially outdated information, and a search for data that is being updated may not be performed correctly. However, the process of matching incoming events is performed based on the actual data and the Kaspersky CyberTrace Web page with detailed information about indicators displays up-to-date data.

Connection settings

In the Connection settings section of the Service tab, you can specify the following settings:

• IP address and port (on Linux, it can be also a UNIX socket) that Feed Service listens on for incoming events

  These settings are stored in the InputSettings > ConnectionString element of the kl_feed_service.conf file.

• IP address and port (on Linux, it can also be a UNIX socket) to which Feed Service sends detection events and alert events

  These settings are stored in the OutputSettings > ConnectionString element of the kl_feed_service.conf file.

• IP address or hostname, and port (on Linux, it can also be a UNIX socket) to which Feed Service sends alert events that inform the event target software of the state of the service

  These settings are stored in the OutputSettings > AlertConnectionString element of the kl_feed_service.conf file.

  You can enable or disable this setting by using Kaspersky CyberTrace Web. When this setting is enabled, Kaspersky CyberTrace does not send alert events to the IP address and port that are stored in the OutputSettings > ConnectionString element of the kl_feed_service.conf file.

• IP address or hostname of the proxy server for updating feeds

  This setting is stored in the Host element of the kl_feed_util.conf file.

• Port of the proxy server for updating feeds

  This setting is stored in the Port element of the kl_feed_util.conf file.

  The preset value is 0. If you do not want to use a proxy server, leave this value unchanged.

• Proxy user name

  This setting is stored encrypted in the User element of the kl_feed_util.conf file.

• Proxy password

  This setting is stored encrypted in the Password element of the kl_feed_util.conf file.

External address of the web interface

In the Web interface section of the Service tab, you can specify the IP address or hostname to be used in Kaspersky CyberTrace events.

This setting is stored in the ResourcesIP element of the kl_feed_service.conf file.

The preset value is 127.0.0.1.

Using a proxy server

To configure CyberTrace to use a proxy server:

Specify proxy settings in the IP address or hostname, Port, User name, and Password fields.

To configure CyberTrace not to use a proxy server:

1. Enter 0 in the Port text field.
2. Clear the IP address or hostname, User name, and Password text fields.

About the disk space notification

When Kaspersky CyberTrace updates feeds, it checks the amount of remaining disk space. If the remaining disk space is low, Kaspersky CyberTrace displays a notification. The notification states how many MB of disk space is still available for the indicator database.

In addition, Kaspersky CyberTrace sends a KL_ALERT_FreeSpaceEnds event.

Service settings (custom tenant)

You can manage the service settings for a particular settings tenant in the CyberTrace web user interface by selecting the Settings tab, and then the Service tab. Make sure that a tenant for which you want to display service settings is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

The Service tab allows you to do the following:

• Edit settings stored in the kl_feed_util.conf and kl_feed_service_log.conf configuration files.
• Reset statistics by clicking the Reset statistics link below the tab.

  This action clears the Dashboard of all the detection statistics related to this tenant.

  We recommend performing this operation after successfully integrating CyberTrace with a SIEM solution. This means the dashboard will not display any detection events generated during the verification test and will only contain real detection events, if there are any.

Connection settings

In the Connection settings section of the Service tab, you can specify the following settings:

• IP address and port (on Linux, it can be also a UNIX socket) that Feed Service listens on for incoming events

  These settings are stored in the InputSettings > ConnectionString element of the kl_feed_service.conf file.

• IP address and port (on Linux, it can also be a UNIX socket) to which Feed Service sends detection events and alert events

  These settings are stored in the OutputSettings > ConnectionString element of the kl_feed_service.conf file.

Feeds settings

You can manage the settings of feeds in the Kaspersky CyberTrace web user interface by selecting the Settings tab, and then the Feeds tab. When you change the settings, you will be asked whether to update the feeds. Depending on the item selected in the drop-down list with all available tenants in the upper-left area of the window, the changes will affect either the general feeds settings (if General is selected) or the feeds settings for a specific tenant (if a specific tenant is selected).

For all tenants, Kaspersky CyberTrace displays only the feeds that are enabled in the General tenant.

For the General tenant, the form allows you to:

• Import a certificate for Kaspersky Threat Data Feeds
• Specify the feeds update period
• Enable and disable feeds
• Select available fields for a feed
• Add actionable fields to a feed
• Specify filtering rules for a feed
• Truncate a feed
• Launch a feeds update manually
• Add custom or third-party feeds
• Configure a custom or third-party feed
• Manage false positives

For a specific tenant, the form allows you to:

• Enable and disable feeds
• Add actionable fields to a feed

Feed tabs

Feeds are listed on the following tabs:

• Kaspersky

  This tab contains Kaspersky Threat Data Feeds.

• <Vendor name>

  Custom and third-party threat data feeds are grouped into tabs by vendor.

  If no vendor name is provided for a feed, it is listed on the Custom feeds tab.

  If there are no custom and third-party feeds, these tabs are not displayed.

• OSINT

  This tab contains OSINT feeds.

• Internal TI

  This tab contains the Internal TI supplier, with indicators added to the database by users.

About the feed update notifications

If any feeds have become available or unavailable with your certificate during an update, a window opens and lists all newly available and unavailable feeds.

In each list of feeds:

• If a currently used feed becomes unavailable, the toggle button remains On but a warning appears next to the feed. The warning states that this feed is no longer supported by your certificate and will not be updated anymore. You can either continue using the last available copy of the unsupported feed, or set the toggle button to Off and stop using the feed.
• For all feeds that become available, the toggle button becomes enabled.

  By default, the toggle button next to all newly available feeds is set to Off. You have to manually set it to On for the feeds that you want to start using.

Importing a certificate for Kaspersky Threat Data Feeds

This section explains how to import a certificate for Kaspersky Threat Data Feeds. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

About the feeds certificate

Kaspersky CyberTrace downloads and updates Kaspersky Threat Data Feeds if they are available with the current certificate.

Importing a feeds certificate for Kaspersky Threat Data Feeds

To import a new feeds certificate for Kaspersky Threat Data Feeds:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Feeds update period section, select the Import certificate link.

   The Import certificate window opens.

4. In the Import certificate window, select the Browse link.
5. In the opened browser window, select the certificate file in the .pem format and open it.
6. In the Import certificate window, click the Save button.

After you import a new certificate, Feed Service checks for and displays all the feeds that are available with it.

Specifying the feeds update period

This section explains what is feeds update period and how to change it. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

About the feeds update period

The feeds update period specifies how often Kaspersky CyberTrace updates all enabled feeds. The default update frequency is 30 minutes. The date and time of the last feeds update is displayed.

Feeds can be rolled back after an update if Kaspersky CyberTrace fails to upload new indicators into the Matching engine (for more information, see the description of the FeedsRollbackEnabled parameter in ServiceSettings).

Feeds update period section

Specifying the feeds update period

To specify the feeds update period:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Feeds update period section, select a value in the Update frequency drop-down list.
4. Scroll to the bottom of the Feeds tab and click the Save button.

Enabling and disabling feeds

This section describes how to enable and disable feeds.

When you enable a feed, it is automatically added and enabled in all tenants. When you disable a feed, it is automatically removed from all tenants.

Enabling and disabling feeds

You can enable or disable feeds, except for the InternalTI supplier on the Internal TI tab, by using the Feed is off or Feed is on toggle button. For the InternalTI supplier on the Internal TI tab, you can only specify actionable fileds.

If you disable a feed, its indicators will not be removed from the indicator database. These indicators will not be displayed on the Indicators tab, but will be taken into consideration regarding the limit on the number of indicators. To avoid that, you must remove the indicators manually before disabling the feed.

When only demo feeds are used, there is a notification at the top of the Feeds tab. Use the Request access to all feeds link in that notification to send your request by email. Alternatively, you can use the Request Kaspersky Threat Intelligence form to subscribe to Kaspersky Threat Intelligence Portal and get commercial feeds, which have a higher level of protection.

Enabling or disabling a feed

Selecting available fields for a feed

This section explains how to specify which fields must be included in a feed. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

About the available fields

Available fields for a feed define which fields from an original feed Kaspersky CyberTrace must include in the resulting feed. For example, if you specify that all but one of the fields in a feed must be ignored, the resulting feed will have only one field for each record.

A feed must have at least one available field that is used for matching. This field must contains IPs, hashes, or URLs.

In the original feed files, some records can have extra fields or can lack some fields; one record can have less or more fields than another record.

If a record has an extra field and this field is selected for a feed, the record will have this field in the resulting feed. If a record has an extra field and this field is not selected for a feed, the record will not have this field in the resulting feed.

If a record lacks a field and this field is selected for a feed, the record will not have this field. If a record lacks a field and this field is not selected for a feed, the record will not have this field.

If you want to exclude records with missing fields from the output, you must create filtering rules for all required fields. You can specify criteria for field values. For more information about filtering rules, see Specifying filtering rules for a feed.

Selecting available fields for a feed

Selecting available feed fields

To select available fields for a feed:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Filtering rules for feeds section select the tab that contains the feed you need to configure.

   You cannot include or exclude available fields for the Internal TI supplier.

4. Locate the feed that you want to configure, and then expand its section.
5. In the settings section for the individual feed, locate the Available fields section.
6. Select the fields that you want to include and remove the selection for fields that you want to exclude.
7. Scroll to the bottom of the Feeds tab, and then click the Save button.

Adding actionable fields to a feed

This section explains how to add extra fields to the outgoing events sent by Kaspersky CyberTrace. These settings are applied to the tenant that is selected in the drop-down list with all available tenants, in the upper-left area of the window.

About the actionable fields

Actionable fields are extra fields that you can insert into outgoing events apart from the context of feed records. You can use actionable fields to extract specific information from the context and pass it in separate fields.

Adding actionable fields

Managing actionable fields

To add actionable fields for a feed:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Filtering rules for feeds section select the tab that contains the feed you need to configure.
4. Locate a feed that you want to configure, and then expand its section.
5. In the settings section for the individual feed, locate the Actionable fields section.
6. Click the Add new field button to add a new actionable field.
7. In the Field name text box, specify the name of a field in the original feed.

   If a feed record contains several equally named fields, and their name is mentioned in the actionable fields list, the outgoing event will contain all of the values delimited by a semicolon in one field.

8. In the Output text box, specify the name of the field, as it will be inserted into outgoing events.

   If the Output text box is empty, the field name in the outgoing event will be the same as the field name specified in the feed.

9. Scroll to the bottom of the Feeds tab, and then click the Save button.

Specifying filtering rules for a feed

This section explains what filtering rules are and how to configure them. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

About the filtering rules

Filtering rules are criteria for a feed and they let you exclude specific records from a feed.

Kaspersky CyberTrace uses filtering rules to filter the downloaded feed files during the update.The filtering rules that you specify are applied to feeds after they are updated, not to the current feeds.

Only those records that match all the specified criteria are included in the output file. If a filtering criterion is specified for a field and the field is missing from a record, this record will not be included in the output file.

Specifying filtering rules for a feed

Filtering rules section

To specify a filtering rule for a feed:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Filtering rules for feeds section, select the tab that contains the feed you need to configure.

   You cannot specify filtering rules for the InternalTI supplier.

4. Locate the feed that you want to configure, and then expand its section.
5. In the settings section for the individual feed, locate the Filtering rules section.
6. Click the Add new rule button.
7. In the Field name drop-down list, select the name of one of the fields available for the feed.

   Each field in a feed can have only one filtering rule associated with it. You cannot have two filtering rules specified for one field.

8. In the Condition drop-down list, select the filtering condition.

   For the list of possible filtering conditions, see section "Possible filtering conditions" below.

9. In the Value text box, specify a filtering criterion for the field.

   For more information on how to define specific filtering criteria, see subsections "Defining filtering criteria for numeric values", "Defining filtering criteria for strings", and "Defining filtering criteria for dates" below.

10. Scroll to the bottom of the Feeds tab, and then click the Save button.

Possible filtering conditions

The table below lists filtering conditions that can be applied to feeds:

Possible filtering conditions

Truncating a feed

This section explains how to limit a feed to a certain maximum number of records. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

About the maximum number of records in a feed

By default, Kaspersky CyberTrace imports all records that match the filtering criteria for a feed. This behavior can be changed. You can set the maximum number of records that must be imported. Kaspersky CyberTrace will import the records in the order they are arranged in the original feed up to the specified maximum of records.

Truncating a feed

Managing the number of records in a feed

To specify the maximum number of records in a feed:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Filtering rules for feeds section, select the tab that contains the feed you need to configure.

   You cannot specify the maximum number of records for the Internal TI supplier.

4. Locate a feed that you want to configure, and then expand its section.
5. In the settings section for the individual feed, select the Truncate feed check box.
6. In the Maximum records text box, specify the maximum number of records for a feed.
7. Scroll to the bottom of the Feeds tab, and then click the Save button.

Launching a feeds update manually

This section explains how to launch a feed update manually. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

About the manual feed updates

Kaspersky CyberTrace updates all enabled feeds automatically with a configured feeds update period. In addition, you can perform a manual feeds update at any time.

If another user has started an update, you will not be able to launch an update until the feeds are updated.

Launching a manual feeds update

To specify the feeds update period:

1. Navigate to the Settings page.
2. Open the Feeds tab.
3. In the Feeds update period section, click the Launch update now button.

About custom, third-party, and Kaspersky feeds

This section describes the feeds available in Kaspersky CyberTrace.

About Kaspersky feeds

For more information about Kaspersky Threat Data Feeds, see About Kaspersky Threat Data Feeds.

About custom and third-party feeds

Custom and third-party feeds are feeds that you can add to Kaspersky CyberTrace.

The addition or deletion of third-party feeds, or turning on the use of OSINT feeds, can be disabled due to restrictions imposed by the licensing level. In this case, the form for adding a custom or third-party feed can be disabled.

Encoding of custom and third-party feeds

All custom or third-party feeds that you add to Kaspersky CyberTrace must be in UTF-8 encoding. If your custom or third-party feeds have a different encoding, make sure to convert them to UTF-8. You can add custom feeds that contain subnet masks of Class C networks. These feeds can be used in the matching process, by marking the feed field as IP.

Certificates and demo feeds

Kaspersky CyberTrace downloads and updates Kaspersky Threat Data Feeds that are available with the current certificate. With the default certificate, only demo feeds are available.

If you use demo feeds, a notification pops up when you select the Settings > Feeds tab. This notification states that you use Kaspersky demo feeds and provides information on how to purchase commercial feeds that have a higher level of protection. The Request access to all feeds link in the notification redirects you to the Request Kaspersky Threat Intelligence form, where you can subscribe to Kaspersky Threat Intelligence Portal and get commercial feeds.

Adding a custom or third-party feed

This section explains how to add a custom or third-party feed and change its settings. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

You can add feeds through only one field of the URL or DOMAIN type. That is, if you mark one field in a feed as URL or DOMAIN, do not mark another field in the feed as URL or DOMAIN. The URL and DOMAIN types are counted as the same field type.

When you add a feed, it is automatically added and enabled in all settings tenants.

Adding a custom feed

To add a feed:

1. In the Filtering rules for feeds section, click Add custom feed.

   The Custom feed window opens:

   

   Adding a custom or third-party feed

2. For any custom or third-party feed, specify the following information:
   • Feed name

     In the feed name, you can use Latin letters, numbers, underscores, and hyphens. The name must differ from other feed names that are already used.

     Do not use FalsePositive or InternalTI as the feed name, since they are reserved for the built-in supplier names of Kaspersky CyberTrace.

     Do not use the @ character in the feed name if basic authentication is used, and the username or password contains @.

   • Vendor name

     From the drop-down list, select the name of the feed vendor or add a new one.

   • Path to the feed

     You can specify the path in one of the following forms:

     • Full path on the computer where Kaspersky CyberTrace is installed
     • Network path

       The specified network path is available for the active user account, while Feed Service and Feed Utility run under the LocalService account. Therefore, if you need to download custom and third-party feeds from a network directory, give the LocalService user account access to this network directory.

       The network directory must be mapped.

       You can only specify the network path in Windows.

     • HTTP(S) address

       Starting from Kaspersky CyberTrace version 4.0, you can download Kaspersky Threat Data Feeds from https://wlinfo.kaspersky.com which were not added at the moment of the product release.

     • FTP address
   • Certificate

     Path to the certificate that gives access to the feed. The full path must be specified.

     You can only specify the certificate path if the feed will be downloaded over an HTTPS connection.

     If you download Kaspersky Threat Data Feeds from https://wlinfo.kaspersky.com, the field contains the preset value Kaspersky Lab certificate. You cannot change this value.

   • Feed type

     This type can be one of the following:

     • json

       If a feed in JSON format contains a field with a subnet mask value, Kaspersky CyberTrace discloses data only if it is a first-level field. If this field is nested, Kaspersky CyberTrace cannot disclose data.

       If you download Kaspersky Threat Data Feeds from https://wlinfo.kaspersky.com, JSON format is used. You cannot change this value.

     • stix

     Kaspersky CyberTrace currently supports STIX versions 1.0 and 1.1.

     • csv
     • xml
     • misp
   • Confidence

     The level of confidence of the feed. This field cannot be empty. The range of possible values is from 1 to 100.

     The preset values are 100 for feeds from Kaspersky, 50 for OSINT feeds, 50 for third-party feeds. You can change these values.

     Level of confidence is provided in the Feeds > Feed > confidence attribute of the Feed Service configuration file.

   • Authentication type

     The authentication type can be Basic or None.

     The basic authentication scheme is available if the path to the feed is an HTTP(S) or FTP address. For this type of authentication, enter the following settings:

     • User name

       This field cannot be empty.

     • Password

     Authentication type is provided in the Settings > Feeds > Feed parameter of the Feed Utility configuration file.

3. For a STIX feed, also specify the following information:
   • Get from a TAXII server

     If this check box is selected, the STIX feed must be downloaded from the TAXII server.

     When a STIX feed is downloaded from the TAXII server, Kaspersky CyberTrace parses this feed and counts the number of indicators.

   • Collection name

     The name of the collection that must be downloaded from the TAXII server. Note that you can specify only one collection name at a time.

     Kaspersky CyberTrace does not support TAXII feeds that have information about the reputation of one object. IBM feeds like xfe.ipr and xfe.url are not supported.

4. For CSV and XML feeds, specify the following information:
   • For a CSV feed, specify a delimiter. After that, the rule will be appllied immediately, and the columns will be split. By default, a semicolon (;) is used as a delimiter.
   • For an XML feed, specify the root element. This allows you to use the names of feed elements relative to the root element. Which element to specify as the root depends on the level of nesting in a given feed.

     In the following example, the root element is root:

     <root> <url>http</url> <ip>1</ip> </root> <root> <url>https</url> <ip>2</ip> </root>

     In the following example, the root element is root/element*:

     <root> <element1> <url>http</url> <ip>1</ip> </element1> </root> <root> <element2> <url>https</url> <ip>2</ip> </element2> </root>

     You cannot use wildcard characters (the asterisk (*) or question mark (?)) to specify the path, only the root element.

After you specify a custom or third-party feed and the settings for it, the feed is fully loaded and a part of it is displayed so that you can choose the fields of the feed to be used in the matching process (see section "Configuring feed fields to be used for matching (CSV, JSON, XML feeds)" below).

Selecting feed fields for matching

This is relevant for feeds in the following formats: CSV, JSON, XML. After a STIX or MISP feed is added, Kaspersky CyberTrace fully loads it for use.

In some cases (when a STIX feed is too large or the TAXII server used for downloading the feed is too slow, or both), it may take Kaspersky CyberTrace up to an hour to load a STIX feed.

Configuring feed fields to be used for matching (CSV, JSON, XML feeds)

To choose feed fields to be used for matching, specify the following information for each field:

• Field type

  One of the following values can be used as the field type:

  • URL
  • MD5
  • SHA1
  • SHA256
  • IP
  • DOMAIN
  • CONTEXT

    Note that there must be at least one field with a type other than CONTEXT. Such fields are used for matching. When such a field is involved in the detection process, a detection event is generated with the %FEEDNAME%_%FIELDTYPE% category, where %FEEDNAME% is the feed name and %FIELDTYPE% is the field type.

    A feed can have one field of the CONTEXT type, at most one field of the URL or DOMAIN type, and several fields of other types. The URL and DOMAIN types are considered the same field type.

• Field name

  This name will be referred to in the matching process.

  In the field name, you can use Latin letters, numbers, underscores, and hyphens. The name must contain at least one Latin letter.

• Reference to the data in the feed:
  • For a CSV feed, specify the column number.
  • For a JSON feed, specify the property name.

    For JSON feeds, the name of the property is case-sensitive. Specify property names in the same case as they are in a JSON feed.

    To specify a nested field, use a slash (/): for example, mainField/subField.

  • For an XML feed, specify the element.

    Specify the full path to the element relative to the root element. You cannot use wildcard characters (the asterisk (*) or question mark (?)) to specify the path, only the root element. The path is case sensitive.

    In the following example, if you specified root/element* as the root element, then the full path to the elements relative to the root element is url and ip, not root/element1/url or root/element2/ip:

  <root> <element1> <url>http</url> <ip>1</ip> </element1> </root> <root> <element2> <url>https</url> <ip>2</ip> </element2> </root>

When adding a custom or third-party feed, feeds updating can be performed. In this case, you will see a notification about it, and a new feed will not be added. We recommend that you wait awhile and then try again to add a feed.

Configuring a custom or third-party feed

This section explains how to configure a custom or a third-party feed. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

Changing the settings of a custom or third-party feed

To change the settings of a custom or third-party feed:

1. In the Filtering rules for feeds section, select the tab that contains the feed you need to configure.

   You cannot change the settings for the Internal TI supplier.

2. Click the name of the feed that you want to modify, and then click Edit.

   

   Editing a custom or third-party feed

3. In the Edit custom feed window that opens, make any necessary changes.
4. Click Save.

You can change all the settings of a custom or third-party feed, except the feed type. For example, you cannot change a CSV feed to JSON.

Adding new fields to a custom or third-party feed

If a new field or fields have been added to your custom or third-party feed being used, and you want Kaspersky CyberTrace to start using these new fields, do the following:

• For a CSV, JSON, or XML feed, follow the procedure outlined in subsection "Changing the settings of a custom or third-party feed" above, to open the Edit custom feed window. Then add each new field by clicking the Add new rule button and specifying the values described above in "Step 2. Choosing feed fields to be used for matching".

  If you do not specify any new fields for a feed, the feed will contain them, but they will not be displayed in the Available fields subsection and will not be used in the matching process.

• For a STIX feed, in the Filtering rules for feeds section, click the feed name and, in the Available fields subsection, select any new field or fields that must be used by CyberTrace, and then click Save.

  For more information, see section Specifying filtering rules for a feed.

Deleting a custom or third-party feed

To delete a custom or third-party feed:

1. In the Filtering rules for feeds section, select the tab that contains the feed you need to delete.

   You cannot delete the Internal TI supplier.

2. Click the name of the feed that you want to delete, and then click Delete.

Managing false positives

This section explains how to manage the False Positives supplier on the Feeds tab. Make sure that the General tenant is selected from the drop-down list that has all available tenants, in the upper-left area of the window.

You can access the false positives list by clicking the Manage False Positives button in the Filtering rules for feeds section.

Managing the false positives list

To access the false positives list, click the Manage False Positives button.

The False Positives window opens:

False Positives list

You can edit the false positives list of indicators as follows:

• Select the URL, Hash, or IP address tab to manage the group you want.

  On the URL tab, you can specify a URL containing a wildcard symbol * (for example, example.com/testpage/*, which will match URLs such as example.com/testpage/test1 and example.com/testpage/test/long_url).

  Starting from Kaspersky CyberTrace version 4.0, the * symbol in the URL is not used as a wildcard. The * just means the "asterisk."

  Kaspersky CyberTrace will apply normalization rules to any URL that you add on the URL tab and which is not yet contained in the indicator database. Thus, the representation of these URLs may change. For example, if you add a URL that contains a port, this port value will be removed. For instructions on how Kaspersky CyberTrace normalizes a URL, see subsection "URL normalization rules" below.

• Every indicator must be on a separate line in the text box.

The false positives list is checked only after all events from a thread have been matched against all the suppliers. The main purpose of the false positives list is to enable Kaspersky CyberTrace to ignore detections for trusted indicators. If any feed produces a detection, but a given indicator is found in the false positives list, Kaspersky CyberTrace does not generate a detection event. In this case, on the Dashboard tab, in the Supplier statistics table, the value in the False positives column corresponding to the supplier that produced the detection is incremented by one. The values in the False positives column show how many false detections were produced by each supplier. For more information about the Dashboard, see section "Kaspersky CyberTrace Dashboard".

URL normalization rules

Any URLs added to the false positives list on the URL tab will be normalized according to the following URL normalization rules:

1. Remove dot segments ("." and "..") according to the algorithm described in RFC 3986, section 5.2.4 Remove Dot Segments (https://www.ietf.org/rfc/rfc3986.txt):

   http://www.example.com/../a/b/../c/./d.html => http://www.example.com/a/c/d.html

2. Remove the protocol:

   http://example.com => example.com

3. Convert internationalized domain names according to the Punycode algorithm described in RFC 3492 (https://www.ietf.org/rfc/rfc3492.txt):

   тест.рф => xn--e1aybc.xn--p1ai

4. Remove the www prefix:

   www.example.com => example.com

5. Remove repeated slashes:

   example.com//dir/test.html => example.com/dir/test.html

6. Remove the trailing slash at the end of the URL:

   example.com/ => example.com

7. Remove the authorization information:

   login:password@example.com => example.com

8. Remove the port number:

   example.com:80/index => example.com/index

9. Remove the #fragment reference:

   example.com#fragment => example.com

10. Remove dots at the end of the host name:

    example.com./index.html => example.com/index.html

11. Convert percent-encoded symbols to UTF-8 according to RFC 3986 (https://www.ietf.org/rfc/rfc3986.txt) and RFC 2279 (https://www.ietf.org/rfc/rfc2279.txt).
12. Convert all characters to lower case:

    EXAMPLE.COM => example.com

13. Convert the IP address (if any) leading to the requested host to dot-decimal notation:

    0112.0175.0117.0150 => 74.125.79.104

Matching process settings

You can manage the matching process settings by selecting the Settings tab and then the Matching tab.

Event sources

In Kaspersky CyberTrace, regular expressions and event normalizing rules are grouped by event source. Regular expressions and event normalizing rules that are not related to a specific event source are grouped under the default event source. Each event source must have at least one regular expression. You can add or remove event sources other than default or edit their properties.

The Event sources page displays all event sources defined in the Feed Service configuration file, except the default event source. You can edit the following properties of the displayed event sources directly in the Event sources page:

• Source ID

  The name of the event source. It must be unique among the event source names used. In the name the following symbols are allowed: Latin letters, numbers, a hyphen (-), and a period (.).

• IP address or Host name or Regular expression

  The IP address of a device that issues events for which you want to add parsing rules. The host name of a device that issues events. The regular expression that will match the source in the events received by Kaspersky CyberTrace.

  The value in the Host name box must be the same as the value in the HOSTNAME field of the incoming syslog messages from this event source.

  The value in the Regular expression box must be optimized. For more information on how to optimize regular expressions, see the "Optimization of regular expressions" subsection of the "About regular expressions" section.

To edit the event normalizing rules and regular expressions defined for a displayed event source, select the corresponding Properties link. For the default event source, you must select the Edit default rules link. In both cases, the form for editing event source properties opens (see subsection "Form for editing event source properties" below).

For more information on event sources and regular expressions used for parsing events that are issued by various devices, see section "About regular expressions".

Adding event sources

To add an event source:

1. Select the Add new event source link.

   The Add New Event Source Wizard starts.

   

   Adding a new event source (step 1)

2. Specify the name and the IP address or host name or the regular expression of a new event source.
3. Click Next.

   If the data entered at the previous step is correct, the form for specifying regular expressions and event normalizing rules opens (see subsection "Form for editing event source properties" below).

   Kaspersky CyberTrace will attempt to obtain data from received events by using the default regular expressions. We recommend that you keep collecting events for at least five seconds.

   

   Specifying the event source properties

4. Specify the event source properties and click OK.

   If the entered event source properties are correct, a new event source is created.

Form for editing event source properties

The form for editing event source properties has an upper area and the lower area. The upper area displays events and highlights substrings that are extracted by a selected regular expression. The lower area has two tabs: the Normalizing rules tab and the Regular expressions tab.

When the form for editing event source properties opens, it starts collecting events that are issued by the event source. These events are processed according to the normalizing rules and the result is displayed in the upper area of the form.

If you have specified the host name of the event source, but the HOSTNAME field of incoming events cannot be extracted, no event is displayed. To fix this problem, either specify the IP address or regular expression of the event source, or change the format of events.

You can pause (or resume), or restart collecting the arriving events in real time. If you restart collecting the incoming events, the text box for displaying events is cleared. This text box can contain up to 50 lines. If more data arrives, older data is removed.

Specifying event normalizing rules

In the lower area of the form for editing event source properties, select the Normalizing rules tab to add, remove, or edit normalizing rules that will be applied to incoming events that meet the conditions of the event source. You can specify which character sequences must be replaced with others (replacing rules) and which character sequences must be used for identifying events to ignore (ignoring rules). If you clear the Apply normalizing rules check box, all the controls for specifying normalizing rules will be disabled, and no normalizing rule will be saved for the event source being edited.

Do not specify the newline character (\n) in replacing rules. Use the InputSettings > EventDelimiter element of the Feed Service configuration file to separate compound incoming events into individual events.

For an event source that is being created, initially the form under the Normalizing rules tab is filled with the normalizing rules specified for the default event source.

Specifying regular expressions

In the lower area of the form for editing event source properties, select the Regular expressions tab to add, remove, or edit regular expressions that will be applied to incoming events that meet the conditions of the event source. For an event source that is being created, initially the form under the Regular expressions tab is filled with the regular expressions that are specified for the default event source and that extract at least some data from the events that are displayed.

Regular expressions have the following properties:

• Indicator type

  The type of data to be extracted from an event. You can set one of the following indicator types:

  • URL
  • MD5
  • SHA1
  • SHA256
  • HASH
  • IP
  • DOMAIN
  • CONTEXT
• Regular expression name

  The regular expression name must be unique among the names of the regular expressions that relate to the same event source.

• Regular expression itself

  The regular expression used for extracting the required value from an event. For more information on regular expressions, see section "About regular expressions".

• Extract all check box

  If this check box is cleared, the regular expression will extract only the first matched value. If this check box is selected, the regular expression will extract all of the matched values.

• Concatenation rules

  You can specify how to concatenate different parts of extracted data to a single value. For information about concatenation of extracted data, see section "About regular expressions", subsection "Compound values".

  

  Setting regular expressions and their properties

You can highlight values that match the regular expressions that you specified for the event source. Click inside the text box that contains the regular expression that you want to highlight.

Specifying event filters

You can specify filtering rules for detection events sent to a SIEM solution from Kaspersky CyberTrace. Kaspersky CyberTrace will send detection events only if the flag for sending detection events to the SIEM solution (the ioc_supplier_send_match event attribute from the indicator database) is set to true and all fields of a feed record that matched the indicator meet the filtering criteria. Note that if the detected indicator does not have the attribute specified in the filtering rule, this indicator is considered to meet the filter. However, all detection events will be included in statistics and displayed on the Dashboard and Detections tabs.

To specify filters for detection events:

1. In the Field name drop-down list, select the value corresponds to the indicator attribute name from the indicator database to which filtering rules are applied.

   To learn more about possible values for %FIELD_NAME%, see section "Working with indicators".

2. In the Condition drop-down list, select a filtering condition.

   For the list of possible filtering conditions, see section "About filtering criteria for sending events to SIEM".

3. In the Value text box, specify a filtering value.
4. Click the Add new filter button if you want to add new event filter.

If necessary, you can edit or delete any filtering rules.

Adding normalizing rules

This section explains how to add normalizing rules to an event source.

About normalizing rules

Normalizing rules are used for transforming events. After Kaspersky CyberTrace applies normalizing rules to an incoming event, the event is processed using regular expressions.

There are two types of normalizing rules:

• Replacing rules

  Rules for replacing one character sequence with another.

• Ignoring rules

  Rules for ignoring events that contain a character sequence.

If the replacing rules and ignoring rules are set, replacing rules are applied first and ignoring rules are applied second.

In the specified regular expressions, the asterisk (*) and question mark (?) are not treated as wildcard characters.

Adding normalizing rules

Adding normalizing rules

To add a normalizing rule:

1. Navigate to the Settings page.
2. Open the Matching tab.
3. Locate an event source that must use the new normalizing rule. Click to open source properties.

   The window with the properties of the selected event source opens.

4. Locate the Normalizing rules tab.
5. Select the Apply normalizing rules check box.
6. If normalizing rules are already specified for the event source, add a new entry. Click Add new rule to add extra text boxes for new rule parameters.
7. Specify rule parameters:
   • For a replacing rule, specify a regular expression in the Regexp to replace text box and a replacement in the Replace with text box.
   • For an ignoring rule, specify a regular expression in the Ignore events that contain this expression text box.
8. Click the OK button.

About regular expressions

This section describes regular expressions and provides information about using them.

About regular expressions

Regular expressions are used to parse incoming events processed by normalizing rules. They extract information to be checked in feeds and to be used in outgoing events.

The preset regular expressions correspond to the format of the events used in the verification test.

After the verification test is performed, you may have to add some new regular expressions or change existing ones for use with specific event source software. For examples of regular expressions to be used for parsing events issued by popular devices, see section "Regular expressions for popular devices".

We recommend that you set regular expressions for extracting data such as the IP address and port of the event source, and of the event target, user name, and date. Use these regular expressions to define the format of the outgoing events.

About regular expression names

You can use any name for a regular expression except the following ones:

• SourceId
• MatchedIndicator
• RecordContext
• Category
• ActionableFields
• Confidence
• IndicatorInfo

Compound values

The concatenate attribute is used to set a rule for creating a compound value from data extracted from an event. A rule refers to groups of extracted data by using #N symbols, where N is the number of a group (starting from 1). If a backslash (\) precedes the hash symbol (#), the latter is not used in the number of a group; instead, the # is treated merely as a number sign.

The following example event is parsed:

url_1=http://domain test_event url_2=/page/mypage test

The regular expressions used and the results of parsing of the example event are provided in the table below.

Examples of applying regular expressions

If no concatenation rule is set or the value of the concatenate attribute is empty, and the regular expression contains more than one group, the values of the groups are concatenated in the order in which they appear in the regular expression.

If the concatenate attribute contains more groups than the regular expression contains, the extra groups will be ignored and will be substituted with the corresponding #N text.

Multiple matching

When parsing an event by using a regular expression, it is possible to extract all values that match the regular expression. For this purpose, set the value of the extract attribute to "all". If this value is set to "first" or the attribute is not specified, only the first value that matches the regular expression will be extracted.

For every matched value a separate detection event is generated. If the detection process does not affect a certain event field, the value of this field in the output event is set to a hyphen (-).

Specifying characters by their hexadecimal code

Feed Service uses regular expressions that conform to PCRE syntax. This syntax allows specifying a character by its code in several ways.

Feed Service does not support specifying a character in \x{hhh..} format. Instead, specify a character by its code in the following way: \uhhhh, where hhhh is the hexadecimal code of the character. For example, you cannot use a ([\x{00a1}-\x{ffff}]) expression, but you can use a ([\u00a1-\uffff]) expression.

Optimization of regular expressions

You can optimize regular expressions to prevent backtracking that interfere with matching a string.

To optimize regular expressions, use the following rules:

• Use possessive quantifiers (++, *+).
• If possible, use non-matching group (?:) with outer brackets.
• Try to use alternation as little as possible and find matches at the end of the string. The alternation operator has the lowest precedence of all regular expression operators.
• Use the anchors (^, $) that match the starting and the ending position within the string.
• Use an atomic group. Atomic groups automatically discard all backtracking positions remembered by any tokens inside the group. The syntax is (?> ...).
• In long regular expressions, try to avoid exponentially increasing the amount of backtracking. An example such as (qwerty.*)* is not recommended.

About filtering criteria for sending events to SIEM

You can specify filtering rules for detection events by using Kaspersky CyberTrace. Each filtering rule is set in the Matching > Event source filters section. The indicator attribute name is set from the indicator database, and filtering conditions and filtering values are specified in the Field name drop-down list, the Condition drop-down list, and the Value text box, respectively. Note that if the detected indicator does not have the attribute specified in the filtering rule, this indicator is considered as meeting the filtering criteria.

Kaspersky CyberTrace will send detection events only if the flag for sending detection events to the SIEM solution (the ioc_supplier_send_match_event attribute from the indicator database) is set to true and all fields of a feed record that matched the indicator meet the filtering criteria.

The table below lists filtering conditions that can be applied to detection events:

Possible filtering conditions

Configuring event sources with custom regular expressions

This section explains how to use the web interface to add and configure event sources with custom regular expressions.

Kaspersky CyberTrace sends information about a detection to a SIEM as a single event. To reduce dwell time, all context information that is required for triage and investigation should be included into this event. This can be accomplished by configuring event sources and custom regular expressions.

Repeat the steps below for every unique source of events that you have.

To configure an event source with custom regular expressions:

1. Create an event source as described in section "Matching process settings".
2. In addition to regular expressions for indicators (URL, IP, MD5 and other indicator types), add regular expressions of CONTEXT type. Regular expressions of CONTEXT type must match any information that will be relevant for the response, such as event identifiers, request identifiers, and time stamps. This context information will also help to search for raw events in the SIEM software, if required.
3. We recommend replacing universal regular expressions for indicators with the ones that match the event source event format. Regular expressions for many popular devices are described in section "Regular expressions for popular devices".
4. Add all CONTEXT regular expressions to the Detection events format field. To do so, specify the names of all regular expressions in the detection events format by using the %RegexpName% pattern.

Example

The event source for this example is a McAfee Web Gateway source.

The event source with custom regular expressions is configured as follows:

1. Create a new event source.

   This event source sends logs through the SIEM software, so adding this event source by IP is not possible because all such event sources will have the IP address of the SIEM software. Instead, specify a regular expression that will identify this event source based on the data contained in the events. For example, the expression can match a device name or a device version that are contained in the events. Note that If your event source sends the events directly to Kaspersky CyberTrace, specify such source by its IP instead.

   

   Creating a new event source

2. Start collecting events and receive several sample events.

   Only those events that matched the regular expression specified in the previous step will be displayed.

   For example, events with the following data were received:

   McAfeeWG|time_stamp=[01/Jan/2015:02:12:31 +0800]|auth_user=jsmith|src_ip=10.10.69.1|server_ip=192.0.2.1|host=www.example.com|url_port=80|status_code=301|bytes_from_client=279|bytes_to_client=1149|categories=Business, Software/Hardware|rep_level=Minimal Risk|method=GET|url=http://www.example.com/|media_type=text/html|application_name=|user_agent=Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)|block_res=0|block_reason=|virus_name=|hash=|filename=|filesize=753|

   

   Collecting events

3. Stop collecting events. Regular expressions for URL and IP will be specified automatically. Replace these expressions with custom ones. Then add regular expressions of CONTEXT type.

   To match the user name contained in the events, add an expression named RE_USERNAME. Specify the following value for the expression: auth_user\=(.*?)(?:\|)

   To match the source IP address, add an expression named RE_SRCIP. Specify the following value for the expression: src_ip\=(.*?)(?:\|)

   To match the URL, add an expression named RE_URL. Specify the following value for the expression: url\=(.*?)(?:\|)

   To match the HTTP status code, add an expression named RE_HTTPCODE. Specify the following value for the expression: status_code=(\d+)

   

   Specifying custom regular expressions

4. Specify an event output format that contains these regular expressions:

   eventName=%Category% matchedIndicator=%MatchedIndicator% url=%RE_URL% src=%RE_SRCIP% ip=%RE_IP% http_code=%RE_HTTPCODE% usrName=%RE_USERNAME% %RecordContext%

   

   Specifying the output format of events

After the steps above are done, the detected events will contain the context fields. For example, an event from Kaspersky CyberTrace can have the following information:

device=McAfee eventName=KL_IP_Reputation matchedIndicator=192.0.2.1 url=- src=10.10.69.1 ip=192.0.2.1 http_code=301 category=test usrName=jsmith first_seen=01.01.2017 00:00 ip=192.0.2.1 ip_geo=ru last_seen=20.11.2019 10:02 popularity=1 threat_score=75

Regular expressions for popular devices

This section provides regular expressions that are to be used for parsing events issued by popular devices.

Devices of different versions can issue events of different format, so it may be that you must use other regular expressions than those provided in this section.

FireEye devices

The events from FireEye devices require the following regular expressions:

• Events in CEF format

  Field	Regular expression
  URL1	filePath=([^\s]*?)\s
  URL2	cs5=([^\s]*?)\s
  MD5	fileHash=([^\s]*?)\s
  SrcIp	src=([^\s]*?)\s
  DstIp	dst=([^\s]*?)\s

• Events in CSV format

  Field	Regular expression
  URL1	cnchost=([^,]*?),
  URL2	objurl=([^,]*?),
  MD5	fileHash=([^,]*?),
  SrcIp	src=([^,]*?),
  DstIp	dst=([^,]*?),

Blue Coat SG devices

The events from Blue Coat SG devices require the following regular expressions:

• SYSLOG events

  Field	Regular expression
  URL	OBSERVED\s"(?:.*?)"\s(.*?)\s
  URL2	http\s(.*?)\s\d+\s(.*?)\s

Websense devices

The events from Websense devices require the following regular expressions:

• CEF events

  Field	Regular expression
  URL	request\=(.*?)(?:\s|$)
  IP address	dst\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\s|$)

• LEEF events

  Field	Regular expression
  URL	url\=(.*?)(?:\s|$)
  IP address	dst\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\s|$)

• key-value pairs

  Field	Regular expression
  URL	url\=(.*?)(?:\s|$)
  IP address	dst_ip\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\s|$)

  Squid devices

The events from Squid devices require the following regular expressions:

McAfee Web Gateway devices

The events from McAfee Web Gateway devices require the following regular expressions:

• Standard events

  Field	Regular expression
  URL	url\=(.*?)(?:\|)
  IP address	server_ip\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\|)

• CEF events

  Field	Regular expression
  URL	request\=(.*?)(?:\s|$)
  IP address	dst\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\s|$)

• SYSLOG events

  Field	Regular expression
  URL	(?:GET|POST)\s(.*?)(?:\s)

Check Point URL Filtering devices

The events from Check Point URL Filtering devices require the following regular expressions:

• SYSLOG events

  Field	Regular expression
  IP address	(?:dst)\s(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Juniper Networks SRX devices

The events from Juniper Networks SRX devices require the following regular expressions:

• SYSLOG events

  Field	Regular expression
  IP address	(?:\sdestination-address)\="(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})"\s

Check Point Firewall devices

The events from Check Point Firewall devices require the following regular expressions:

• SYSLOG events

  Field	Regular expression
  IP address	dst\:(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Palo Alto Networks devices

The events from Palo Alto Networks devices require the following regular expressions:

• LEEF events

  Field	Regular expression
  IP address	dst\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\s|$)

• SYSLOG events

  Field	Regular expression
  IP address	(?:dst.*?)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

• CEF events

  Field	Regular expression
  IP address	dst\=(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})(?:\s|$)

Fortinet FortiGate devices

The events from Fortinet FortiGate devices require the following regular expressions:

• SYSLOG events

  Field	Regular expression
  IP address	(?:dst.*?)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Cisco IPS devices

The events from Cisco IPS devices require the following regular expressions:

Snort devices

The events from Snort devices require the following regular expressions:

• UNIFIED2 events

  Field	Regular expression
  IP address	(?:destination.*?)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

• CSV events

  Field	Regular expression
  IP address	(?:.*?,.*?,.*?,.*?,)(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Alternatively, you can use the following regular expressions for parsing events of all types:

Cisco IronPort devices

The events from Cisco IronPort devices require the following regular expressions:

• SYSLOG events

  Field	Regular expression
  URL	(?:GET|POST)\s(.*?)\s
  IP address	(?:NONE|DIRECT|DEFAULT_PARENT)\/(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})

Event format settings

You can manage the settings for formats of events in the CyberTrace web user interface by selecting the Settings tab and then the Events format tab. Depending on the item selected in the drop-down list with all available tenants in the upper-left area of the window, you edit either the general event format settings (if General is selected) or the event format settings for a particular settings tenant (if a particular settings tenant is selected).

Kaspersky CyberTrace events formats

On the Events format tab, you can specify the formats of detection events, alert events, record context, and actionable fields context.

We do not recommend changing the format of events format manually. Select the check boxes with the patterns that you want to use in outgoing events and Kaspersky CyberTrace will update the format automatically.

Some event sources may require that you change the event format, depending on your integration. For more information, see the section "Setting event formats for specific event sources" below.

For more information about formats and patterns that you can specify, see About event formats and patterns.

This tab has the following text fields:

• Alert events format—Specify the format for outgoing events that inform the event target software of the state of Feed Service.
• Detection events format—Specify the format for outgoing detection events.

  This section consists of two subsections:

  • Service fields

    Values of these fields are patterns generated by Kaspersky CyberTrace.

    Select the check boxes with the patterns that you want to use in outgoing detection events. Kaspersky CyberTrace will update the format automatically.

  • Value from the event

    Values of these fields are extracted from the incoming events with regular expressions defined for the event source.

    Select the check boxes with the patterns that you want to use in outgoing detection events. Kaspersky CyberTrace will update the format automatically.

• Records context format—Specify the format in which the names and values of the feed fields are inserted into outgoing events.
• Actionable fields context format—Specify the format in which the names and values of the actionable feed fields are inserted into outgoing events.

Setting event formats for specific SIEM solutions

The correct format of alert and detection events depends on your SIEM solution. If you change the format of events in CyberTrace, you may also need to update your integration with the SIEM solution.

For ArcSight:

• You may need to update the event formats and actionable fields.

  For more information, see Step 3. Configuring CyberTrace for interaction with ArcSight.

For Qradar:

• You may need to add parsing for new fields.

  For more information, see Step 5. Retrieving custom event properties.

For RSA NetWitness:

• You may need to specify how new fields are parsed.

  For more information, see Step 2. Sending events from Feed Service to RSA NetWitness and RSA NetWitness troubleshooting.

For LogRhythm:

• You may need to specify how new fields are parsed.

  For more information, see Adding Kaspersky CyberTrace events.

Specifying the format of detection and alert events

Kaspersky CyberTrace allows you to specify the format of detection and alert events by selecting the needed patterns.

To specify the format of detection events:

1. Select the patterns that must appear in detection events.
2. Click the Save button at the bottom of the page.

To specify the format of alert events:

1. Select the patterns that must appear in alert events.
2. Click the Save button at the bottom of the page.

You can specify the format of alert events if the General tenant is selected in the drop-down list with all available tenants, in the upper-left area of the window.

About event format patterns

You can use formats and patterns to include specific information into the events generated by Kaspersky CyberTrace.

Formats are strings that determine the format of an event or pattern. Patterns are special wildcards that you can use when specifying formats. A pattern is replaced by actual data when an event is generated.

About alert and detection events

You can specify formats for two types of events generated by Kaspersky CyberTrace:

• Detection events

  These are outgoing events that hold information about detected matches with indicators.

  For more information about the format of detection events, see "Detection events format" below.

• Alert events

  These are outgoing events that inform the event target software about the state of Feed Service.

  For more information about the format of alert events, see "Alert events format" below.

Record context format

The %RecordContext% format specifies how context fields must be added to an event. You can specify a format for this pattern in the Records context format field.

You can use the following patterns in the %RecordContext% format:

• %ParamName%

  The name of the field in the feed.

• %ParamValue%

  The value of the field.

The %RecordContext% format is used in the formats of detection and alert events:

• Detection events

  The %RecordContext% pattern determines the format of the context fields passed in a detection event.

  For example, if %RecordContext% is %ParamName%=%ParamValue%, then for a feed with the "Ip" and "Geo" fields, the following string can be produced (note the space symbol between the data of the two fields): "Ip=192.0.2.100 Geo=ru,br,ua,cz,us".

• Alert events

  The %RecordContext% pattern determines the format of the context fields passed in an alert event.

  For more information about information contained in alert fields, see Alert events sent by Kaspersky CyberTrace.

  The fields are specific for each type of alert event. For example, if %RecordContext% is %ParamName%=%ParamValue%, and a feed is updated, the following string can be produced: "feed=Phishing_URL_Data_Feed.json records=200473".

Actionable field context format

The %ActionableFields% format specifies how actionable fields must be added to an event. You can set a separate format for this pattern in the Actionable fields context format field.

You can use the following patterns in the %ActionableFields% format:

• %ParamName%

  The name of the actionable field.

• %ParamValue%

  The value of the actionable field.

The %ActionableFields% format is used in the format of detection events:

The %ActionableFields% pattern determines the format of the actionable fields passed in a detection event.

For example, if %ActionableFields% is %ParamName%:%ParamValue%, and the cn1 and cn2 fields are specified for the feed, then the following string can be produced: "cn1:Example Device cn2:Example Environment".

Alert events format

You can specify this format in the Alert events format field.

You can use the following patterns in this format:

• %Alert%

  The type of the event.

• %Date%

  Current date and time in the Mon DD HH:MM:SS format.

• %RecordContext%

  Context of the event, as described in the "Record context format" section above.

The following is an example of the alert events format:

If a feed update event is generated, the example above produces the following event:

Detection events format

You can specify this format in the Detection events format field.

You can use the following patterns in this format:

• %Category%

  Category of the detected object.

• Regular expression name

  This pattern is a name of the regular expression. It is substituted by a value extracted from the event field that matches a regular expression. For example, if a regular expression has the name RE_URL, the pattern for it is %RE_URL%, and the generated event holds the value that matched this regular expression.

• %MatchedIndicator%

  Detected indicator (a URL, hash, or IP address) that caused the event.

• %SourceId%

  Event source identifier. This is the name that you specified for the event source on the Matching tab.

  The identifier of the preconfigured event source is Default.

• %Confidence%

  The level of confidence. This value is taken from the indicator confidence value of a feed that contains matched indicators from the detection event.

• %IndicatorInfo%

  The link to the Kaspersky CyberTrace Web page that contains information about the detected indicator.

• %ActionableFields%

  Actionable fields, as described in the "Actionable field context format" section above.

• %RecordContext%

  Context of the event, as described in the "Record context format section" above.

The following is an example of the OutputSettings > EventFormat element:

The example above produces the following events:

Patterns for ArcSight

Feed Service sends service events in the CEF format. The event formats for ArcSight must comply with the requirements of the CEF format.

For detection events, specify the following format:

In addition to the general patterns, the detection events format for ArcSight uses the following regular expression patterns referenced by regular expression names:

• %DST_IP%—Destination IP address.
• %DeviceIp%—IP address of the endpoint device where the event occurred.
• %RE_HASH%—Hash contained in the event.
• %RE_URL%—URL contained in the event.
• %Device%—Device vendor.
• %Product%—Device name.
• %UserName%—Name of the user that was active on the endpoint device.
• %Id%—Event identifier.

For alert events, specify the following format:

In the format above, 4 (or another value from 1 to 10) is the level (severity) of the alert events from Kaspersky CyberTrace.

Patterns for RSA NetWitness

The values of the detection and alert event formats must correspond to the event formats set in the v20_cybertracemsg.xml file. If you change the formats, edit the v20_cybertracemsg.xml file accordingly.

The following is an example of the detection events format:

In addition to the general patterns, the detection events format for RSA Net Witness uses the following regular expression patterns referenced by regular expression names:

• %RE_URL%—URL contained in the event.
• %RE_HASH%—Hash contained in the event.
• %DST_IP%—Destination IP address.
• %SRC_IP%—Source IP address.
• %DeviceIp%—IP address of the endpoint device where the event occurred.
• %Device%—Device vendor.
• %DeviceAction%—Action taken by the device.
• %UserName%—Name of the user that was active on the endpoint device.

Alert events sent by Kaspersky CyberTrace

This section describes alert events that can be generated by Kaspersky CyberTrace.

KL_ALERT_ConfigurationUpdated

This event is generated if Feed Service has reloaded the configuration file.

This event has no context fields.

KL_ALERT_FeedBecameAvailable

This event is generated if a feed that can be used with the current certificate has become available.

This event has the following field:

• feed

  Feed name.

KL_ALERT_FeedBecameUnavailable

This event is generated if a feed that is being used with the current certificate has become unavailable.

This event has the following context field:

• feed

  Feed name.

KL_ALERT_OutdatedFeed

This event is generated if a feed has not been updated during the specified period.

This event has the following context field:

• feed

  Feed name.

KL_ALERT_ServiceUnavailable

This event is generated when the watchdog module has detected that Feed Service has crashed or frozen.

This event has no context fields.

KL_ALERT_ServiceStopped

This event is generated when Feed Service is stopped successfully.

This event has no context fields.

KL_ALERT_ServiceStarted

This event is generated when Feed Service is started successfully.

This event has no context fields.

KL_ALERT_UpdatedFeed

This event is generated when a feed is updated and loaded by Feed Service. This means that new indicators from the feed can be used in the matching process. Please note that these indicators may be added to the database later, as they are loaded asynchronously.

This event has the following context fields:

• feed

  Feed name.

• records

  The number of records loaded from the feed.

KL_ALERT_FailedToUpdateFeed

This event is generated when Feed Service fails to load a new feed (for example, due to the limitation on the number of indicators that are imposed by the license key) and continues using an old feed.

This event has the following context fields:

• feed

  Feed name.

• error

  Error message from Feed Utility or the text "Error while applying feed <FeedName>".

KL_ALERT_LicenseExpires

This event is generated to inform you that the license key that is being used will expire in less than 30 days.

This event has the following context fields:

• license_name

  Name of the license key.

• expiration_date

  Expiration date of the license key.

KL_ALERT_LicenseExpired

This event is generated when a current license key has expired.

This event has the following context fields:

• license_name

  Name of the license key.

• expiration_date

  Expiration date of the license key.

KL_ALERT_EPSLimitExceeded

This event is generated when the limit on the number of processed events per second (EPS) imposed by the licensed key or licensing level has been exceeded.

This event has the following context fields:

• current_eps

  Actual number of EPS that arrive in Feed Service.

• license_limit_eps

  Limit on the number of EPS that is imposed by the license key or licensing level.

KL_ALERT_EPSHardLimit

This event is generated when Feed Service limits the number of events processed per second to the maximum number of events for the current license key or licensing level. The limit applies regardless of the number of incoming events.

• license_limit_eps

  Limit on the number of EPS that is imposed by the license key or licensing level.

KL_ALERT_LicenseChanged

This event is generated when Kaspersky CyberTrace starts to use another license key or licensing level.

This event has the following context fields:

• license_name

  Name of the license key.

  If no license key is used, this context field is not included.

• expiration_date

  Expiration date of the license key.

  If no license key is used, this context field is not included.

• licensing_level

  Licensing level of the key, if a license key is used.

  Licensing level, if a license key is not used.

KL_ALERT_RetroScanCompleted

This event is generated when the retrospective scan task succeeded.

This event has the following context fields:

• iocs_rescanned

  Number of scanned indicators.

• iocs_detected

  Number of detected indicators.

• retroscan_report

  Link to the result of the retrospective scan.

  This field is absent if the value of the iocs_detected field is 0.

KL_ALERT_RetroScanError

This event is generated when the retrospective scan task failed.

This event has the following context field:

• error

  Short text error description.

KL_ALERT_RetroScanStorageExceeded

This event is generated when the limit on the size of the saved events has been exceeded.

This event has the following context field:

• storage_size_limit

  Limit on the size of the saved events.

KL_ALERT_FreeSpaceEnds

This event is generated when the available disk space becomes low.

This event has the following context field:

• msg

  Amount of disk space that is still available for the indicator database.

  The message has the following format: "Free space left: %FreeSpace% Mb", where %FreeSpace% is the remaining number of MB available for the indicator database.

KL_ALERT_IndicatorsStoreLimitExceeded

This event is generated when the limit on the size of the saved indicators has been exceeded.

This event has the following context fields:

• current_indicators_count

  Current number of indicators.

• license_limit_indicators

  Limit on the number of indicators that is imposed by the license key.

KL_ALERT_IndicatorsStoreHardLimit

This event is generated when Kaspersky CyberTrace limits adding and updating of indicators.

This event has the following context fields:

• license_limit_indicators

  Limit on the number of indicators that are imposed by the license key.

• msg

  Message that new indicators cannot be added to the database due to the limitation on the number of indicators that is imposed by the license key.

User settings

You can manage the user accounts settings by selecting the Settings tab, and then the Users tab.

The Users tab contains the following tabs:

• Local accounts

  On this tab, you can add a new user account, and change or delete existing user accounts.

• LDAP

  On this tab, you can provide the option to use domain user accounts for authentication in Kaspersky CyberTrace Web.

Working with local accounts

Kaspersky CyberTrace supports multiple users mode. On the Local accounts tab, you can add a new user account, and change or delete existing user accounts.

The form for managing user accounts settings can be disabled due to restrictions imposed by the licensing level. In this case, only the admin account will be available.

To add a new account:

1. Click the Add new user button.

   The New user window opens.

2. Enter the account login in the Login field.

   If you are using domain authentication along with local accounts, do not create local accounts with the names defined for existing domain accounts.

3. Enter the full name in the Full name field.

   You can leave this field blank.

4. Set your password in the Password field and repeat it in the Confirm password field.

   Your password must be from six to 16 characters long and must contain at least one lowercase letter, one uppercase letter, and one special character.

5. Choose your role in the drop-down list:
   • Analyst. Analysts have access to all features of Kaspersky CyberTrace, except those reserved for Administrators.
   • Administrator. Administrators can manage user accounts and configure Kaspersky CyberTrace.

   The default role is Analyst.

   Administrators can download logs that may contain data considered private, security-related, or sensitive. In addition, administrators can browse the search results for all users.

6. Click OK to create an account.

To edit a user:

1. Click the button next to the user that you want to edit.

   The Change user window opens.

2. Edit one of the fields.
3. Click OK.

To delete a user:

1. Click the button next to the user that you want to delete.

   The Delete user window opens.

2. Click Delete.
3. Click Yes to confirm that you want to delete the selected user.

Configuring LDAP authentication

Kaspersky CyberTrace supports LDAP user authentication to allow logging on as a domain user. This section explains how to configure this type of authentication by using Kaspersky CyberTrace Web.

Kaspersky CyberTrace supports the use of Active Directory only if the domain controller is running Windows. The use of Active Directory with Linux-based domain controllers is possible, but not guaranteed.

The LDAP section allows you to perform the following actions:

• Enable LDAP authentication
• Test the connection to the LDAP server
• Specify connection settings
• Configure accounts filtering

Enabling LDAP authentication

To enable LDAP authentication,

Click the LDAP auth enabled toggle button.

The LDAP server will now be used for user authentication.

When LDAP authentication is enabled, you still can interact with Kaspersky CyberTrace under a local user account.

Testing the connection to the LDAP server

Go through the procedure below to make sure that a connection to the LDAP server is established.

To test the connection to the LDAP server:

1. Click the Test connection with LDAP link.

   The Test connection with LDAP window opens.

2. Specify the following settings:
   • User name for test connection
   • User password for test connection
3. Click Test.

A connection test can be performed only if you specified all the necessary settings for connecting to the server.

Connection settings

In the Connection settings section of the LDAP tab, you can specify the following settings:

• IP address or FQDN (fully qualified domain name) and port of the LDAP server

  This setting is stored in the AuthenticationServer > ConnectionString element of the kl_feed_service.conf file.

• SSL-secured connection

  You can enable the use of a secure connection to the LDAP server by using Kaspersky CyberTrace Web.

  This setting is stored in the useEncryption attribute of the AuthenticationServer > ConnectionString element of the kl_feed_service.conf file

• Path to the LDAP database

  The path to the database containing user accounts that can access Kaspersky CyberTrace.

Accounts filtering

The Accounts filtering section contains filtering rules for administrator and analyst accounts.

You can configure the following properties:

• Account format

  You can select one of two formats:

  • User Principal Name

    If this option is selected, users must provide a user name that is not an email address when performing authentication (for example user, but not user@domain.com).

  • SAM Account Name

    If this option is selected, users must provide a user name in the following format when performing authentication: Domain\User.

• Administrator accounts filter

  The filter for LDAP user accounts that defines which users must be assigned the Administrator role depending on their common name in Active Directory.

  If this value is not specified, all users who login using LDAP authentication and pass the analyst account filter will be assigned the Analyst role.

  This setting is stored in the AuthenticationServers > AdministratorAccountsFilter element of the kl_feed_service.conf file.

• Analyst account filter

  The filter for LDAP user accounts that defines which users must be assigned the Analyst role depending on their common name in Active Directory.

  If this value is not specified, all users who login using LDAP authentication and do not pass the administrator account filter will be assigned the Analyst role.

  This setting is stored in the AuthenticationServers > AnalystAccountsFilter element of the kl_feed_service.conf file.

  As an example, in the figure below the filters are configured so that the users who are members of the Admins group will be assigned the Administrator role, and the users who are members of either the Operators or the Analysts group will be assigned the Analyst role.

  

  Example of accounts filters

If the AdministratorAccountsFilter and AnalystAccountsFilter elements of the kl_feed_service.conf file contain values, and the user that is trying to log in is not included in any of the specified groups, Kaspersky CyberTrace will return an error and deny access to the web user interface for this user.

Logging settings

You can manage the logging settings in the CyberTrace web user interface by selecting the Settings tab, and then the Logging tab.

Under Logging settings, you can set the following settings:

• Log directory—Directory in which log files are to be stored
• Log level

  The following values are possible:

  • None

    Logging is turned off.

    Corresponds to the non log level in the kl_feed_service_log.conf configuration file.

  • Error

    Only errors are logged.

    Corresponds to the err log level in the kl_feed_service_log.conf configuration file.

  • Info

    Errors and information messages are logged.

    Corresponds to the inf log level in the kl_feed_service_log.conf configuration file.

  • Debug

    All messages are logged

    Corresponds to the any log level in the kl_feed_service_log.conf configuration file.

• Maximum log size (MB)—In megabytes
• Delete old logs when Feed Service starts—Option of removing log files created by previous sessions
• Write to syslog—Option of whether logging must be performed by the syslog daemon

  If this option is selected, all the other settings are ignored.

  This option is available only on Linux systems.

For information about the contents of log files, see section "Feed Service logging", subsection "Log file contents".

Licensing settings

You can manage the Kaspersky CyberTrace licensing in the CyberTrace Web by selecting the Settings tab, and then the Licensing tab.

On the Licensing tab, you can view information about license keys and perform the following actions:

• Add a new license key
• Remove a license key

  

  Licensing tab

The Licensing tab contains a list of license keys. The active license key is highlighted, and all inactive keys are unavailable (dimmed).

For each license key, the following information is available:

• Licensing level of the key
• Expiration date of the key
• Limit on the events per second (EPS) number
• The current number of indicators loaded into Kaspersky CyberTrace

  This item displays the number of unique indicators for all enabled suppliers. Also note that if an indicator is present in multiple suppliers, duplications of this indicator are discarded from the total number.

• The maximum number of indicators allowed with this licensing level
• Information whether the search feature in CyberTrace Web is available
• Information whether custom feeds can be used
• Information whether multiuser mode is available
• License key file name

About the licensing

This section describes the licensing levels of Kaspersky CyberTrace.

Commercial licensing levels and Community Edition

The Kaspersky CyberTrace licensing policy provides two licensing level types:

• Community Edition

  This is a limited licensing level that can be used without a license key. This licensing level does not have an expiration date.

  If no license key is installed, the Community Edition licensing level is used.

  The Community Edition licensing level has the following limitations:

  • Multi-user mode is not available. Kaspersky CyberTrace has a single account with the Administrator role that can perform all operations.
  • No more than 250 events per second are processed.
  • No more than 1,000,000 records can be loaded from all feeds.
• Commercial licensing levels

  These licensing levels are available with a license key and are suited for different enterprise scenarios.

  License keys are created by Kaspersky specialists. To obtain a license key, contact your technical account manager (TAM).

Information contained in the license key

A license key is a sequence of bits that you can apply to use a certain licensing level of Kaspersky CyberTrace.

A license key contains the following information:

• Licensing level
• Expiration date
• Limit on events per second (EPS)
• Limit on the total number of indicators that can be loaded from Kaspersky Threat Data Feeds

In addition, a license key lists available features: the Search feature, use of third-party feeds, multi-user mode, multitenancy mode, and IOCs export.

License keys are stored in the %service_dir%/httpsrv/lic directory. You can manage license keys by using the Settings > Licensing tab.

Active and additional license keys

A license key may be active or additional.

An active license key is a license key that is currently used by Kaspersky CyberTrace. Kaspersky CyberTrace cannot have more than one active license key.

An additional license key is a license key that is not currently in use. The additional license key automatically becomes active when the current active license key expires. An additional license key can be added only if an active license key has already been added.

Kaspersky CyberTrace chooses an active license key to use, based on the expiration date of the license key. If there is a single license key that has not expired, this key is used as an active key. If there are several license keys that have not expired, the license key with the nearest expiration date is chosen. If all keys have expired, the Community Edition licensing level is used.

License key expiration notifications

Kaspersky CyberTrace sends alerts and generates events when license keys expire or are about to expire.

If less than 30 days are left until the key expires, Kaspersky CyberTrace displays an alert in CyberTrace Web and generates a KL_ALERT_LicenseExpires event.

After the license key expires, Kaspersky CyberTrace displays an alert in CyberTrace Web and generates a KL_ALERT_LicenseExpired event. Afterward, Kaspersky CyberTrace automatically searches for an additional license key and makes it active. If there are no additional license keys, Kaspersky CyberTrace changes its licensing level to Community Edition, displays an alert in CyberTrace Web, and generates a KL_ALERT_LicenseChanged event.

Licensing and DMZ

Kaspersky CyberTrace does not send any licensing information to Kaspersky. Licensing functionality works in scenarios involving a DMZ.

Limitations imposed by license keys

This section describes limitations that are imposed on the Kaspersky CyberTrace functionality by license keys of different licensing levels.

Limit on events per second (EPS)

For the Community Edition licensing level, Kaspersky CyberTrace processes no more events per second than permitted by the licensing level.

Several commercial licensing levels have EPS limits specified in the licensing key. Kaspersky CyberTrace limits EPS for these licensing levels in two stages.

At the first stage, if the EPS limit is exceeded at any point in time, Kaspersky CyberTrace displays alerts in Kaspersky CyberTrace Web and generates the KL_ALERT_EPSLimitExceeded event. All events above the EPS limit continue to be processed.

At the second stage, if the EPS limit was continuously exceeded for a period of seven days, the limit takes effect. Kaspersky CyberTrace displays alerts in Kaspersky CyberTrace Web and generates the KL_ALERT_EPSHardLimit event. Kaspersky CyberTrace starts to process events within the limit imposed by the license key. After you change the license key or start to process no more events than are specified in the licensing key, this limitation ceases to apply.

Limit on the number of indicators

If the license key or licensing level sets a limit on the total number of indicators, Kaspersky CyberTrace reads, in total, no more records from all the feeds than are permitted by the license key or licensing level.

For the Community Edition licensing level, if the number of indicators exceeds a limit, Kaspersky CyberTrace sends the KL_ALERT_IndicatorsStoreHardLimit service event and then, through its Web UI, displays a notification that there is a limitation on the adding and updating of indicators. When the KL_ALERT_IndicatorsStoreHardLimit event is generated, Kaspersky CyberTrace stops reading new records.

For other licensing levels, if the number of indicators exceeds a limit, Kaspersky CyberTrace sends the KL_ALERT_IndicatorsStoreLimitExceeded service event, and then displays a notification through its Web UI. This notification informs that Kaspersky CyberTrace will set a restriction on the adding and updating of indicators after five days. Then, if the total number of indicators continuously exceed a limit for this period, Kaspersky CyberTrace sends the KL_ALERT_IndicatorsStoreHardLimit service event and displays another notification informing that the limitation has taken effect. When the KL_ALERT_IndicatorsStoreHardLimit event is generated, Kaspersky CyberTrace stops reading new records.

After you change the license key or start to process no more indicators than are specified in the licensing key, this limitation ceases to apply.

Limit on the use of custom feeds

If the license key does not allow the use of custom feeds, only Kaspersky Threat Data Feeds can be used. OSINT feeds cannot be used in this case.

Other limitations

The following features can be disabled by the license key or licensing level:

• Search feature in Kaspersky CyberTrace Web
• Multi-user mode

  If multi-user mode is disabled, you can sign in under the admin account only. This user account has the Administrator role.

Adding a license key

You can add an active key or an additional license key to Kaspersky CyberTrace by using Web UI.

To add a license key:

1. In Kaspersky CyberTrace Web, select the Settings tab, and then the Licensing tab.
2. Click Add new license key.

   The Add a license key window opens.

3. Click Browse.

   Your browser opens a window where you can select a file.

4. Select a license key.

   License keys are created by Kaspersky specialists. To obtain a license key, contact your technical account manager (TAM).

5. Confirm the selection.
6. In the Add a license key window, click Add.

   Kaspersky CyberTrace will import the selected key. If there is no active key, the imported key will be selected as an active key.

Deleting a license key

You can delete license keys by using Kaspersky CyberTrace Web UI.

To delete a license key:

1. In Kaspersky CyberTrace Web, select the Settings tab, and then the Licensing tab.
2. Click the Delete icon () for the license key that you want to delete.

   The Delete the license key window opens.

3. Click Delete to delete the license key.

   Confirmation of the deletion will be displayed.

4. Click Close.

Tenants settings

Kaspersky CyberTrace supports a multi-tenant architecture that allows you to manage tenants. A tenant is a client-specific set of configuration parameters. By default, Kaspersky CyberTrace uses a General tenant that provides the overall settings. You can create or edit the Kaspersky CyberTrace tenants in CyberTrace Web by selecting the Settings tab, and then the Tenants tab.

On the Tenants tab, you can view information about the tenants that are used in Kaspersky CyberTrace and perform the following actions:

• Add a new tenant
• Edit a tenant configuration
• Delete a tenant

Adding tenants

To add a tenant:

1. Click the Add new tenant link.

   The New tenant window opens.

2. Specify a name for the new tenant in the Tenant field.
3. Specify a description for this tenant in the Description field.
4. Select a SIEM.

   You can select a SIEM supported by Kaspersky CyberTrace or a custom one (a non-supported SIEM solution).

   This SIEM will be used in the tenant for sending events to CyberTrace.

   Depending on the selected SIEM, CyberTrace will specify regular expressions, detection events, and service events that are used in integration with this solution.

   For the full list of supported SIEMs, see subsection "Supported SIEM solutions" below.

5. Specify connection parameters specific for the tenant that Kaspersky CyberTrace will use for incoming events:
   • Select what type of connection you want to use.
   • In the IP address and Port fields, specify an IP address and port.
   • In the UNIX socket field, specify a UNIX socket.
6. Specify an IP address and port specific for the tenant that Kaspersky CyberTrace will use for outgoing events.
7. Click Save.

Editing a tenant configuration

To edit a tenant configuration:

1. Click the Edit button next to the tenant that you want to edit.
2. Edit the tenant configuration:
   • Tenant name

     You cannot change the tenant name for the General tenant.

   • Description
3. Click Save.

Deleting tenants

To delete a tenant:

1. Click Delete next to the tenant that you want to delete.
2. Confirm that you want to delete the tenant.

Supported SIEM solutions

Kaspersky CyberTrace supports integration with several SIEM solutions. Thus, CyberTrace uses a number of preset settings for each SIEM, such as settings for parsing events and event format settings (for detection and service events).

The following SIEM solutions are supported:

• Splunk
• ArcSight ESM
• RSA NetWitness
• IBM QRadar
• LogRhythm
• Kaspersky Unified Monitoring and Analysis Platform

Indicators export settings

You can export information related to the indicators to a CSV file. This report contains data about indicators exported from the indicator database and filtered by specified rules. This section explains how to create an export task that can be published as a report file in the CSV format and how to configure the data that must be included in this report.

The Indicators export tab displays the Indicators export tasks list with existing export tasks and allows you to do the following:

• Add a new export task.
• Manage existing export tasks.

  You can perform the following actions with existing export tasks:

  • Edit existing export tasks.
  • Delete existing export tasks.
  • Enable the scheduled task that regularly updates the indicators export task.
  • Launch the export task.

Adding a new export task

To create a new export task:

1. Click Add task.

   The Add indicators export task window opens.

2. In the Task properties section, specify the following settings for every field:
   • Task name

     The name of the export task.

   • Maximum records

     You can specify the maximum number of indicators that can be included in the report.

     The maximum possible value is 50000.

   • Export every

     Update frequency (in hours) for generating a report.

   • Delimiter

     The delimiter for splitting of fields in the report file. By default, this value is ';'.

3. In the Limit access to specified credentials section, specify the following information for every field:
   • Use authorization to download indicators export

     Specify this setting if you want to use authentication for downloading the indicators export.

     If this setting is used, specify the credentials:

     • User name

       User name that must be used when requesting a report.

       This user name is intended only for access to a specific report and it is not the same as the Kaspersky CyberTrace user account.

     • Password

       Password that must be used when requesting a report.

4. In the Fields to export section, specify filtering rules for the fields that you want to export:
   • Field name

     Name of the field to which filtering rules are applied and/or that must be exported.

   • Condition

     Filtering condition that is applied to the field.

   • Value

     Filtering criteria for the field. This value must meet the requirements described in the "Working with indicators" section.

   • Include

     Specify this setting if you want to include the field in the report file.

     By default, this field must be included in the report file.

   • Output name

     Name of the output field which must contain the values from the exported field.

   In the CSV report file, output fields have the same order that you specify through Kaspersky CyberTrace Web. If you change the position of the export rule to increase or decrease its priority, the new priority level will be retained in the CSV report file.

5. If necessary, specify the rules for sorting data in the Sort conditions section:
   • Field name

     Specify the field you want to sort.

   • Direction

     You can sort your values in ascending or descending order. This order is retained in the CSV report file.

6. Click Next.

   The Export preview window opens. This window displays a table with an example of an indicators export.

7. Click Add to apply the specified settings and add this task to the Indicators export tasks list.

   If you want to change the setting specified in the previous step, click Back.

   If you want to reset all the settings and close the window, click Cancel.

Managing an existing export task

To edit an existing export task:

1. In the Indicators export tasks list, locate the task that you need, and then click Edit.
2. Change the settings as described in the instructions above.

To delete an existing export task:

In the Indicators export tasks list, locate the task that you need, and then click Delete.

To enable scheduled indicators export:

Click the Enable scheduled export task toggle button.

If this setting is turned off, you cannot access reports that were created earlier.

To launch an export task:

In the Indicators export tasks list, locate the task that you need, and then click Launch export.

After that, the report with the exported indicators becomes available for download at the following address:

https://%CyberTrace_WebAddress%/reports/%iocexport_name%

where %iocexport_name% is the name of the specified export task.

Retrospective scan settings

Kaspersky CyberTrace allows you to save events containing indicators that are not detected, and then perform a retrospective scan of these events. This section explains how to configure Kaspersky CyberTrace for using the retrospective scan.

The Retrospective scanning tab allows you to do the following:

• Enable or disable the retrospective scan.
• View the current size of saved events.
• Remove saved events.

  Saved events cannot be removed when the retrospective scan is in progress. If you want to disable the retrospective scan and removed the saved events, you must wait until the current retroscan task is finished.

  

  Retrospective scanning tab

• On the General settings tab, manage the following settings:
  • Set the frequency of the scheduled retrospective scan task or disable the scan on schedule. If 1 month is selected, retrospective scan starts every 30 days.
  • Enable or disable the size limit for events that must be saved for the retrospective scan.
  • Set the maximum size of events (in gigabytes) that must be saved for the retrospective scan.
  • Set how long (in days) the events used for the retrospective scan must be stored.
  • Set how long (in days) the results of the retrospective scan must be stored.

  

  General settings tab

• On the Feeds used in retroscan tab, enable or disable feeds that must be used in the retrospective scan.

  

  Feeds used in retroscan tab

• On the Fields saved for retroscan tab, configure the following settings:
  • Enable or disable saving events related to a specific tenant for use in the retrospective scan.

    If you exclude a tenant from the retrospective scan, the regular expressions contained in this tenant become unavailable for selection.

  • Select regular expressions contained in a tenant for use in the retrospective scan.

    You must select at least one regular expression.

  

  Fields saved for retroscan tab

Feed Service Guide

This section explains how Feed Service works, how to configure it, and how to manage it from the command line or by using a script.

About Feed Service

Feed Service is one of the components of Kaspersky CyberTrace. It runs as a service and works as follows:

1. Feed Service listens on a specified port for incoming events.
2. Feed Service matches indicators in received events against those in the feeds that are used.
3. Feed Service generates events in the specified format that contains detected indicators on the basis of the incoming events and the feed records involved in the detection process.
4. Feed Service sends the generated events to the specified address and port.

Managing Feed Service

This section describes how to manage Feed Service.

Managing Feed Service by using systemd (Linux)

You can manage Feed Service by using systemd.

You can use all standard systemd commands to manage Feed Service.

Checking the service status

To check the status of Feed Service, use the following command:

systemctl status cybertrace.service

Starting, stopping, restarting the service

To start Feed Service, use the following command:

systemctl start cybertrace.service

Feed Service begins to receive events only after it has loaded the feeds into memory and has sent the KL_ALERT_ServiceStarted event.

To stop Feed Service, use the following command:

systemctl stop cybertrace.service

To restart Feed Service, use the following command:

systemctl restart cybertrace.service

Adding and removing the service from the list of services loaded on boot

By default, Feed Service is loaded on boot. You can change this behavior with the following commands.

To add Feed Service to the list of services loaded on boot, use the following command:

systemctl enable cybertrace.service

To remove Feed Service from the list of services loaded on boot, use the following command:

systemctl disable cybertrace.service

Managing Feed Service using the script (Windows)

The %service_dir%\bin\kl_control.bat file is a script for managing Feed Service.

Command-line options

The script supports the following command-line options:

• start—Starts Feed Service and the watchdog Windows service.

  The script prints the information about whether this operation succeeded or failed.

  Feed Service begins to receive events only after it has loaded the feeds into memory and has sent the KL_ALERT_ServiceStarted event.

• stop—Stops Feed Service and the watchdog Windows service.

  The script prints the information about whether this operation succeeded or failed.

• restart—Restarts the services of Kaspersky CyberTrace: stops them and starts them again.

  The script prints the information about whether this operation succeeded or failed.

• status—Prints the information about the status of the services of Kaspersky CyberTrace.
• reloaddb—Instructs Feed Service to reload the feeds it is using.
• help—Prints the Help information: how to use the kl_control.bat script.

You can use only one option at a time.

The script can also be launched without parameters. In this case, it prints the list of the available options and their descriptions.

Example

The following command starts Feed Service:

Managing Feed Service from the command line (Linux)

You can run the Feed Service binary file from the command line.

We recommend that you use systemd to manage Feed Service. Running Feed Service from the command line is intended for troubleshooting purposes.

The binary file uses the flags described in the following table.

Flags for kl_feed_service

Example

The following command runs Feed Service in watchdog mode and specifies serv.conf as the configuration file.

Managing Feed Service from the command line (Windows)

You can run the Feed Service binary file from the command line.

We recommend managing Feed Service by using the script. Running Feed Service from the command line is intended for troubleshooting purposes.

The binary file uses the flags described in the following table.

Flags for kl_feed_service.exe

If no flag is specified, the kl_feed_service.exe program prints the list of available flags to the screen.

Feed Service uses the configuration file that resides in the same folder as the kl_feed_service.exe binary file.

Feed Service configuration reference

This section describes the configuration file of Feed Service.

We recommend that you configure Feed Service by using the Settings > Service tab. Use this section only for configuration that cannot be done from the web interface.

About the configuration file (Feed Service)

Feed Service reads settings from a configuration file.

Editing the configuration file

If the configuration file is absent or does not follow the rules specified in this section, Feed Service does not start and prints an error message.

To avoid potential issues, make a backup copy of the Feed Service configuration file before you make any changes in the file. If Kaspersky CyberTrace does not work properly after you have reconfigured Feed Service, replace the configuration file with its backup copy.

Applying changes made to the configuration file

You have to reload the configuration file or restart Feed Service after you edit the configuration file for the changes to take effect.

You can apply changes made to certain elements of the configuration file by reloading the configuration file but without restarting Feed Service.

These elements include:

• InputSettings > RegExps
• Feeds
• OutputSettings > RecordFieldContextFormat
• OutputSettings > AlertFormat
• OutputSettings > EventFormat

To reload the configuration file without restarting Feed Service,

Run the script for managing Feed Service with the reloadconfig option.

To apply changes made to other elements of the configuration file, you have to restart Feed Service.

To restart Feed Service,

Run the script for managing Feed Service with the restart option.

Configuration file location (Linux)

In Linux, the configuration file used by Feed Service can be specified in the following ways:

• When Feed Service is started from the command line, the path to a configuration file can be specified as an argument. By default, this path is ../etc/kl_feed_service.conf.
• When Feed Service is managed using the script, the path to the configuration file is specified in the script parameters. By default, this path is %service_dir%/etc/kl_feed_service.conf.

Configuration file location (Windows)

The configuration file used by Feed Service is named kl_feed_service.conf and resides in the same directory as the Feed Service binary file.

Case sensitivity

All the names used in the configuration file (for example, pattern names or feed fields) are case-sensitive.

Using special characters

To use special characters (for example, an ampersand or angle brackets) in regular expressions and other parameters, enclose the text of the elements in a CDATA section.

The following example uses braces around parameters:

EULA

Specifies whether the terms of the End User License Agreement (EULA) were accepted by a user.

Attributes

This element has no attributes.

Possible values

This element has the following possible values.

EULA element possible values

Example

The following is an example of a EULA element.

IndicatorDatabaseSettings

Defines the indicator database settings.

Path

IndicatorDatabaseSettings

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested elements:

• ConnectionString

  Specifies the IP address and port that will be used for accessing the indicator database.

  For more information about this element, see the "IndicatorDatabaseSettings > ConnectionString" section below.

IndicatorDatabaseSettings > ConnectionString

This element must contain a string with an IP address and a port. See the example below.

This element is mandatory.

Example

The following is an example of this element.

Domains

Contains configuration for tenants.

Path

Domains

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested element:

• Domain

Example

The following is an example of this element.

Domain

Defines configuration for a single feed in a tenant.

Path

Domains > Domain

Attributes

This element has the following attributes.

Domain element attributes

Nested elements

This element is a container for the following nested elements:

• Description

  Tenant description

  This element is optional.

• InputSettings

  Input connection settings specific for this tenant. See InputSettings.

• OutputSettings

  Output connection settings specific for this tenant. See OutputSettings.

  This section is for a specific tenant. It cannot have AlertConnectionString and AlertFormat as nested elements.

• Feeds

  Feed settings for the feeds used in this tenant.

  For more information about this element, see the Domain > Feeds section below.

Domain > Feeds

Feed settings for the feeds used in the tenant.

This element is a container for the following nested element:

• Feed

  Settings for a specific feed in a tenant.

Example

The following is an example of this element.

Domain > Feeds > Feed

Defines configuration for a single tenant.

Path

Domains > Domain > Feeds > Feed

Attributes

This element has the following attributes.

Feed element attributes

Nested elements

This element is a container for the following nested element:

• ActionableFields

  Actionable fields for this feed in this tenant.

  The specified actionable fields replace actionable fields that are defined for this feed in the Feeds > Feed > ActionableFields element (if such fields are defined).

  This element is optional.

  For more information, see Feeds > Feed > ActionableFields.

Example

The following is an example of this element.

InputSettings

Contains input settings for the General tenant.

This element defines the IP address and port that Feed Service will listen on for incoming events, normalizing rules for processing the events, and regular expressions for parsing the events.

Path

InputSettings

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested elements:

• RegExps

  Contains Boost regular expressions that are used to parse incoming events originating from different sources.

• ConnectionString

  Specifies the IP address and port (or the Windows-named pipe) that the service will listen on for incoming events.

• EventDelimiter

  Specifies the rule for the splitting of incoming events,

Example

The following is an example of this element.

RegExps

Contains Boost regular expressions that are used to parse incoming events originating from different sources.

Path

InputSettings > RegExps

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested element:

• Source

  Contains parameters for a specific event source.

Example

The following is an example of this element.

Source

Contains parameters for a specific event source.

The regular expressions and event normalizing rules specified in the configuration file are grouped by event sources that are represented by Source elements. Usually these event sources are devices that issue events, which afterward are checked by Feed Service. Every Source element contains a set of rules. There can be one or more Source elements in the InputSettings > RegExps element.

Rules workflow

The way Feed Service chooses rules from different Source elements is described in the following flow chart.

Choosing a rule

Note that event normalizing rules are applied first and regular expressions are applied afterwards.

The regular expressions of the default event source for finding URLs, IP addresses, and hashes are universal; that is, they can be used for parsing events issued by most devices. They can be used for parsing events that contain multiple URLs, but cannot be used, for example, for parsing events that contain URLs with no protocol specified. The use of universal regular expressions lowers the performance of Feed Service, compared to using device-specific regular expressions. Also, the universal regular expressions do not handle the dispersal, in an event, of different parts of a URL (for example, the host and the path). The universal regular expressions for finding hashes can extract symbol sequences that actually are not hashes.

Special event sources

There must be an event source with the default identifier (<Source id="default">). Rules of the default event source have lower priority than rules specific to the event source. Rules specific to the event source are applied first. Rules of the default event source are applied next. If a rule specific to the event source and a rule of the default event source have the same name, the rule of the default event source is applied only if the rule specific to the event source had no matches.

There are two special event sources that you can use: http_single_lookup (<Source id="http_single_lookup">) and http_file_lookup (<Source id="http_file_lookup">).

The rules of the http_single_lookup event source are used when single values are searched for by using CyberTrace Web.

The rules of the http_file_lookup event source are used when hashes of specified files or indicators in log files are searched for by using CyberTrace Web. Therefore, if you have to search for values contained in a log file that has a special format, we recommend specifying rules for the http_file_lookup event source.

If the configuration file contains the http_single_lookup event source or the http_file_lookup event source, we strongly recommend that you do not remove the regular expressions specified in these special event sources by default and, instead, edit them as needed.

Path

InputSettings > RegExps > Source

Attributes

This element has the following attributes.

Source element attributes

Nested elements

This element is a container for the following nested elements:

• Regular expressions

  Regular expressions that are used to parse incoming events originating from this source.

  Each regular expression is a separate element with the name of the regular expression.

• NormalizingRules

  Rules for modifying incoming events.

Example

The following is an example of this element.

Source > Regular expression

Defines a regular expression for an event source.

Path

InputSettings > RegExps > Source > %RegexpName%

This element has the name of the regular expression.

Attributes

This element has the following attributes.

%RegexpName% element attributes

Value

This element contains a Boost regular expression.

For more information about specifying values for this parameter, see section "About regular expressions".

Example

The following is an example of this element.

Source > NormalizingRules

Defines rules for modifying incoming events.

For more information about normalizing rules, see Adding normalizing rules.

Path

InputSettings > RegExps > Source > NormalizingRules

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested elements:

• Replace

  These elements specify replacing rules.

• Ignore

  These elements specify ignoring rules.

NormalizingRules > Replace

Defines a replacing rule.

This element has the following attributes.

Replace element attributes

NormalizingRules > Ignore

Defines an ignoring rule

This element has the following attributes.

Ignore element attributes

Example

The following is an example of this element.

ConnectionString

Specifies the IP address and port (or the Windows-named pipe) that the service will listen on for incoming events.

This element is mandatory.

Path

InputSettings > ConnectionString

Attributes

This element has no attributes.

Value

The value is formatted as <ip_address>:<port> (if an IP address and port are used) or as \\.\pipe\<pipe_name> (if a Windows-named pipe is used).

The IP address must consist of four decimal octets, each separated by a dot. The value in each octet must be less than 256.

Example

The following is an example of this element.

EventDelimiter

Specifies the rule for the splitting of incoming events.

Path

InputSettings > EventDelimiter

Attributes

This element has no attributes.

Value

The value rule must have the following format:

<EventDelimiter>%START_EVENT_SYMBOLS%</EventDelimiter>

The %START_EVENT_SYMBOLS% value contains the regular expression that corresponds to the beginning of the substring of the incoming event.

The rule processes all occurrences of the %START_EVENT_SYMBOLS% value. If the %START_EVENT_SYMBOLS% value is found in the string of the incoming event, this string will be split into multiple events by the addition of the newline character (\n) before every matched substring.

If the %START_EVENT_SYMBOLS% value does not match any substring of the incoming string, the incoming string is considered a single event.

If an event contains newline characters (\n), such an event will be split by using both the %START_EVENT_SYMBOLS% value and newline characters as event delimiters.

Example

The following is an example of this element.

Feeds

Defines how events must be checked against feeds.

Path

Feeds

Attributes

This element has the following attributes:

Feeds element attributes

Nested elements

This element is a container for the following nested element:

• Feed

  Every Feed element describes a feed.

  A configuration file must contain at least one Feed element.

Example

The following is an example of this element.

Feed

Describes a feed or supplier.

Path

Feeds > Feed

Attributes

This element has the following attributes:

Feed element attributes

Nested elements

This element is a container for the following nested elements:

• Field

  This element is obsolete starting from Kaspersky CyberTrace version 4.0.

  A Field element specifies the rules for checking an event against the records of the feed.

  For more information about this element, see section "About feed matching rules".

• ActionableFields

  Defines actionable fields.

Example

The following is an example of this element.

Feed > ActionableFields

Defines the fields that must be inserted into the outgoing events apart from the context. An outgoing event contains context and actionable fields.

Path

Feeds > Feed > ActionableFields

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested element:

• ActionableField

  Defines a single actionable field.

ActionableFields > ActionableField

Defines a single actionable field.

This element has the following attributes:

ActionableField element attributes

Example

The following is an example of this element.

About feed matching rules

Starting from Kaspersky CyberTrace version 4.0, the Feeds > Feed > Field element is obsolete. Information in this section applies only to Kaspersky CyberTrace 3.1.0 and below.

Feed Service checks incoming events against the feeds specified in the Feeds > Feed elements. For each specified feed, matching rules are set with one or more Field elements. Each Field element describes how a particular field in the feed must match the data from incoming events.

The following is an example of a Field element:

The Field element must have the following attributes:

• name

  The name of the field in the feed must be specified in the name attribute. Note that field names are case-sensitive. For example, fields "md5" and "MD5" are different fields.

  To specify nested fields, use the '/' delimiter. For example, name="detail/info" specifies the info field in a feed that has the following content:[ { "hash":"234D123...", "detail": [ { "info" : "some value" } ] } ].

• matching_type

  The type of matching must be specified in the matching_type attribute.

  The following values are possible:

  • "Url"

    The event field will be checked for conformance to the URL masks stored in the feed. For more information about the masks stored in feeds, contact your technical account manager (TAM).

  • "Exact"

    Comparison of two strings will be performed: the event field and the field stored in the feed.

  • "Hash"

    The event field will be checked to determine whether it is equal to the one stored in the feed. This matching type is used only for hashes.

• input_regexp_to_match

  Name of a regular expression that is used for matching. The value of this attribute must be one of the regular expression names from the InputSettings > RegExps element.

• category

  The category that will be used in the outgoing event.

All the attributes of the Field element are required.

The following is an example of the feed matching rules for a specific feed:

The following is an example of the feed matching rules for a specific feed for ArcSight:

If you have events that contain the event source IP address, we recommend that you check them against IP Reputation Data Feed. This must be done because the event source may be a bot or a malicious device of some other kind that takes part in a DoS attack on the user resources. To check such events against IP Reputation Data Feed, add an SRC_IP regular expression to find the event source IP addresses. Also, add a rule for IP Reputation Data Feed to use the SRC_IP regular expression, so that the configuration file will contain the following records:

Also, add the reference to the value found by the SRC_IP regular expression (%SRC_IP%) to the OutputSettings > EventFormat element.

IoCExports

Contains indicators export settings.

Path

IoCExports

Attributes

This element has the following attributes.

IoCExports element attributes

Nested elements

The IoCExports element is a container for the following nested element:

• IoCExport

Example

The following is an example of this element.

IoCExport

Defines settings for a single report with exported data.

Path

IoCExports > IoCExport

Attributes

This element has the following attributes.

IoCExport element attributes

Nested elements

The IoCExports > IoCExport element is a container for the following nested element:

• Filter

  A configuration file must contain at least one Filter element.

Example

The following is an example of this element.

IoCExport > Filter

Defines the filtering rules for data to be exported.

Path

IoCExports > IoCExport > Filter

Attributes

This element has the following attributes.

Filter element attributes

Example

The following is an example of this element.

RetroScan

Contains retrospective scan settings.

Path

RetroScan

Attributes

This element has the following attributes.

RetroScan element attributes

SendEventFilters

Contains filtering rules for detection events from Kaspersky CyberTrace. You can specify several filtering rules at once.

Path

SendEventFilters

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested element:

• Filter

  A filtering rule.

SendEventFilters > Filter

This element defines a filtering rule.

For more information about this element and possible values of its attributes, see section "Working with indicators".

This element has the following attributes:

ActionableField element attributes

Example

The following is an example of this element.

OutputSettings

Contains output settings for the General tenant.

Defines the address and port of the event target software to send the outgoing events to, and the format of the outgoing events.

Path

OutputSettings

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested elements:

• EventFormat

  Specifies the format of outgoing events.

  For more information about the values of this element, see About event formats and patterns.

  The EventFormat element is mandatory.

• RecordFieldContextFormat

  Specifies how context fields must be added to an event.

  For more information about the values of this element, see About event formats and patterns.

  The RecordFieldContextFormat element is mandatory.

• ActionableFieldContextFormat

  Specifies how actionable fields must be added to an event.

  For more information about the values of this element, see About event formats and patterns.

  The ActionableFieldContextFormat element is mandatory.

• AlertFormat

  Specifies the format for outgoing events that inform the event target software of the Feed Service state.

  For more information about the values of this element, see About event formats and patterns.

  The AlertFormat element is optional. If it is absent from the configuration file, no notification is made.

• ConnectionString

  Specifies the IP address and port (or the Windows-named pipe) to which the service will send outgoing events.

  The ConnectionString element is mandatory.

  For more information about this element, see the "OutputSettings > ConnectionString" section below.

• AlertConnectionString

  Specifies the IP address (or host) and port to which the service will send service alerts.

  The AlertConnectionString element is optional.

  For more information about this element, see the "OutputSettings > AlertConnectionString" section below.

• FinishedEventFormat

  Specifies the format of the informational event that is generated for each processed event.

  The FinishedEventFormat element is mandatory.

  For more information about this element, see the "OutputSettings > FinishedEventFormat" section below.

OutputSettings > ConnectionString

Specifies the IP address (or host) and port to which the service will send service alerts.

The string is formatted as <ip_address>:<port> (if an IP address and port are used) or as \\.\pipe\<pipe_name> (if a Windows-named pipe is used). The IP address must consist of four decimal octets, each separated by a dot. The value in each octet must be less than 256.

OutputSettings > AlertConnectionString

Specifies the IP address (or host) and port to which the service will send service alerts.

The value of this element is formatted as <ip_address>:<port> (if an IP address and port are used) or as \\.\pipe\<pipe_name> (if a Windows-named pipe is used). The IP address must consist of four decimal octets, each separated by a dot. The value in each octet must be less than 256.

The AlertConnectionString element is optional. If the element is omitted, the enabled attribute with the false value is used for this element.

This element has the following attributes:

AlertConnectionString element attributes

OutputSettings > FinishedEventFormat

Specifies the format of the informational event that is generated after an event is processed.

If this parameter is enabled, Kaspersky CyberTrace will generate an informational event for each event that it processes. An informational event is generated even if there were no detections.

The FinishedEventFormat element is mandatory.

The value of this element specifies the event format. You can use the %RecordContext% pattern and regular expression names in the format. For more information about patterns, see About event formats and patterns.

The %RecordContext% pattern will provide the following fields, if used:

• category

  It is "LookupFinished" for events of this type.

• sent_events

  The number of events sent to a SIEM solution.

• total

  Concatenation of the following substrings formed for every category assigned to detection events:

  <category>:<number_of_detections>;

  If there were no detections, the sent_events parameter is set to 0, and the total string is empty.

This element has the following attributes:

FinishedEventFormat element attributes

Example

The following is an example of this element.

ServiceSettings

Defines settings for the Feed Service process.

Path

ServiceSettings

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested elements:

• Bases

  Specifies the path to the directory that contains feeds from Kaspersky. If a relative path is set, it is calculated relative to the directory that contains the service binary file.

  The Bases element is mandatory.

• BasesBackup

  Specifies the path to the directory that contains backup version of feeds from Kaspersky. If a relative path is set, it is calculated relative to the directory that contains the service binary file.

  The BasesBackup element is mandatory.

• BasesDownload

  Specifies the path to the directory that contains downloaded feeds from Kaspersky. If a relative path is set, it is calculated relative to the directory that contains the service binary file.

  The BasesDownload element is mandatory.

• TemporaryDir

  The directory for temporary files.

  The TemporaryDir element is optional. If it is omitted, the default value is used.

  In Linux, the default value is /tmp.

  In Windows, the default value is %TEMP% (the current Windows user's temporary folder).

• OutdatedBasesAlertPeriod

  The time interval in hours following the last feed update, after which a notification about an outdated feed is sent to the event target. To turn off notifications, set this parameter to 0. This setting is taken into account for every feed that has no outdated_alert_period attribute.

  The OutdatedBasesAlertPeriod element is optional. If it is omitted, the default value 0 is used.

• ScannersCount

  The number of scanners. Every scanner handles a single TCP connection.

  If you want to run Feed Service in watchdog mode, specify one scanner in addition to the number of scanners needed for Feed Service itself. This must be done because the watchdog module uses an additional scanner.

  The ScannersCount element is optional. If it is omitted, the default value 9 is used.

• ScanningThreadsPerScanner

  The number of threads per scanner.

  The ScanningThreadsPerScanner element is optional. If it is omitted, the default value 8 is used.

• EventSendingRetriesCount

  Number of times Feed Service tries to resend a detection event to a SIEM solution if the first attempt at sending fails. If the value of EventSendingRetriesCount is 0, Feed Service sends each detection event one time and does not attempt to resend it.

  Maximum possible value is 10. The preset value is 3.

  The EventSendingRetriesCount element is mandatory.

• EventSendingRetriesTimеout

  Time interval between attempts made by Feed Service to resend a detection event to a SIEM solution, in seconds. Maximum possible value is 60.

  The EventSendingRetriesTimеout element is mandatory.

  The preset value is 10.

• FeedsRollbackEnabled

  Specifies if feeds rollback is enabled or disabled.

  If feeds rollback is enabled, feeds are rolled back when Kaspersky CyberTrace fails to upload new indicators into the Matching engine after feeds are updated. Kaspersky CyberTrace removes new indicators from the database and uses the previous feeds.

  Possible values:

  • true — feeds rollback is enabled.
  • false — feeds rollback is disabled.

  Kaspersky CyberTrace reads FeedsRollbackEnabled only during initialization and does not reread it after.

  By default, there is no FeedsRollbackEnabled element in the configuration file. If this element is missing, feeds rollback is enabled.

Example

The following is an example of this element.

GUISettings

Defines settings for the CyberTrace Web.

Path

GUISettings

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested elements:

• HTTPServer

  Contains the CyberTrace HTTP service parameters.

• FeedUtil

  Contains the Feed Utility parameters.

• AuthenticationServers

  Defines settings for connection to the LDAP servers.

Example

The following is an example of this element.

HTTPServer

Contains the CyberTrace HTTP service parameters.

Path

GUISettings > HTTPServer

Attributes

This element has no attributes.

Optional

The HTTPServer element is optional. If it is omitted, the default values are used.

Nested elements

This element is a container for the following nested elements:

• ConnectionString

  Specifies the IP address and port where the CyberTrace HTTP service is available.

  The ConnectionString element is optional. If it is omitted, the 127.0.0.1:443 IP address and port are used. After the installation process is complete, the default value of the HTTPServer > ConnectionString element is 0.0.0.0:443.

• SSLCertificatePath

  Path to the PEM-formatted certificate on a local computer for HTTPS connections. If a relative path is specified, it is calculated relative to the executable file.

  For security reasons, do not store your certificate in a shared folder accessible over a network and do not specify the path to a network shared folder containing your certificate.

  The SSLCertificatePath element is optional. If it is omitted, the ../httpsrv/kl_feed_service_cert.pem file is used.

• SSLPrivateKeyPath

  Path to the PEM-formatted private key on a local computer for HTTPS connections. If a relative path is specified, it is calculated relative to the executable file.

  For security reasons, do not store your private key in a shared folder accessible over a network and do not specify the path to a network shared folder containing your private key.

  The SSLPrivateKeyPath element is optional. If it is omitted, the ../httpsrv/kl_feed_service_private.pem file is used.

• TemplatesPath

  Path to the directory that contains layout pages for CyberTrace HTTP Service. If a relative path is specified, it is calculated relative to the executable file.

  The TemplatesPath element is optional. If it is omitted, the ../httpsrv/templates/ directory is used.

• ResourcesIP

  Contains the IP address or hostname that is used in the %IndicatorInfo% field of detection events, in the service events to notify that a retrospective scan completes, and in the result of exporting an indicator.

  The ResourcesIP element is mandatory. The default value is 127.0.0.1.

Example

The following is an example of this element.

FeedUtil

Contains the Feed Utility parameters.

Path

GUISettings > FeedUtil

Attributes

This element has no attributes.

Nested elements

This element is a container for the following nested element:

• ConfigurationPath

  Path to the Feed Utility configuration file.

  The ConfigurationPath element is mandatory.

Example

The following is an example of this element.

AuthenticationServers

Defines settings for connection to the LDAP servers.

Path

GUISettings > AuthenticationServers

Attributes

This element has no attributes.

Mandatory

This element is mandatory.

Nested elements

This element is a container for the following nested element:

• AuthenticationServer

  Contains the LDAP connection settings parameters.

Example

The following is an example of this element.

AuthenticationServers > AuthenticationServer

Contains the LDAP connection settings parameters.

Path

GUISettings > AuthenticationServers > AuthenticationServer

Optional

The AuthenticationServer element is optional.

Attributes

This element has the following attributes.

AuthenticationServer element attributes

Nested elements

This element is a container for the following nested elements:

• ConnectionString

  Contains the connection parameters for the LDAP server.

  For more information about this element, see subsection "AuthenticationServer > ConnectionString" below.

• DomainName

  The path to the database that contains the user accounts that can access Kaspersky CyberTrace.

  For more information about this element, see subsection "AuthenticationServer > DomainName" below.

• AdministratorAccountsFilter

  Contains filtering rules for user accounts that must be assigned the Administrator role.

  The AdministratorAccountsFilter element must not contain the value that is specified in the AnalystAccountsFilter element.

  The AdministratorAccountsFilter element can be empty.

• AnalystAccountsFilter

  Contains filtering rules for user accounts that must be assigned the Analyst role.

  The AnalystAccountsFilter element must not contain the value that is specified in the AdministratorAccountsFilter element.

  The AnalystAccountsFilter element can be empty. If the value is not specified, all user accounts that access Kaspersky CyberTrace will have the Analyst role.

  This element is mandatory.

AuthenticationServer > ConnectionString

IP address or FQDN (fully qualified domain name), and port of the LDAP server.

This element is mandatory and cannot be empty.

This element has the following attributes.

ConnectionString element attributes

AuthenticationServer > DomainName

The path to the database that contains the user accounts that can access Kaspersky CyberTrace.

This element is mandatory and cannot be empty.

This element has the following attributes.

DomainName element attributes

Example

The following is an example of this element.

Feed Service logging

This section describes how Feed Service logs its activity.

Enabling logging

By default, logging is disabled. To enable logging, use the kl_feed_service_log.conf file in the bin directory where the binary file of the service is located. Fill in the kl_feed_service_log.conf file as described in this section. This file is used by Feed Service and the watchdog module. A change in the contents of the kl_feed_service_log.conf file results in the new settings being applied; this process takes several seconds.

Enabling logging decreases the performance of Feed Service. Use logging only if you encounter problems and errors.

Logging and data security

If you enable logging, Feed Service can write to the log files any of the following information that can be considered private, security-related, or sensitive:

• Unmodified data (URLs, IP addresses, hashes, and other data) as it is received by Feed Service.
• The results of checking events against a feed.

Log files are regular text files. No information written to the log files is encrypted. The log files have standard inherited access rights. We recommend that you assign the directory for storing log files the appropriate rights so that only the administrator can read the log files.

Kaspersky CyberTrace does not send log files or any data contained in them to Kaspersky. For technical support purposes, your Technical Account Manager can ask you to provide log files.

Log files are stored until they are explicitly deleted by a user. If the Append parameter in the logging configuration file is 0, the previous log files are deleted when Feed Service is started. If the Append parameter in the logging configuration file is 1, the information is retained during the full cycle of Feed Service usage

If you uninstall Kaspersky CyberTrace, these log files will not be deleted if the directory with log files is located outside of the Feed Service installation directory (as specified by the LogsFolder parameter).

For more information about data written to the log files, see subsection "Log file contents" below.

Logging configuration file

The kl_feed_service_log.conf file is an XML file. Its fields are described in the table below.

Configuration file example

The following kl_feed_service_log.conf file example enables logging at the dbg logging level. Logs will be stored in the logs subdirectory of the directory where the Feed Service binary file resides.

Log files name format

Feed Service writes messages to files named "kl_feed_service-<pid>-<date_time>.log" or "kl_feed_service-<pid>-<date_time>_<index>.log".

The watchdog module writes messages to files named "kl_feed_servicewd-<pid>-<date_time>.log" or "kl_feed_servicewd-<pid>-<date_time>_<index>.log".

Log file contents

If the err logging level is used, Feed Service writes the following information to the log:

• Feed Service version and PID of the service process.
• The Feed Service configuration file parameters.
• Path to the directory with feeds.
• Errors occurred when normalizing a URL.
• Errors occurred while establishing a TCP connection.
• Watchdog messages:
  • Feed Service freezing or crashing.
  • Restarting Feed Service.
• Errors that occurred while changing the state of CyberTrace Web interface elements. Identifiers of user sessions where these errors occurred.
• Information about errors that occurred while handling CyberTrace Web HTTP requests:
  • Request method, GET or POST.
  • Request path for GET and POST requests.
  • GET request parameters. For POST requests, the request body is not logged.
  • Total elapsed time for handling the request.

If the inf logging level is used, Feed Service writes the following information to the log:

• Information to be written at the err logging level.
• Establishing or closing a TCP connection.
• Receiving and sending TCP requests and responses.
• Switching Feed Service to same-socket mode.
• Putting new databases into effect.
• Detecting an event that matches some record in a feed.
• Incoming event.

  Note that incoming events can contain private data, and so we recommend that you protect the log files from unauthorized access.

• Messages about navigating CyberTrace Web pages. Identifiers of user sessions for these messages.

If the dbg logging level is used, Feed Service writes the following information to the log:

• Information to be written at the inf logging level.
• Substrings in events information that regular expressions match.
• The result of checking an event against a feed.
• Sending test messages to and from the watchdog module.
• Messages about actions performed on CyberTrace Web pages. Identifiers of user sessions for these messages.
• Information about successful CyberTrace Web HTTP requests:
  • Request method, GET or POST.
  • Request path for GET and POST requests.
  • GET request parameters. For POST requests, the request body is not logged.
  • Total elapsed time for handling the request.

About resending detection events

By default, if an error occurs when Feed Service is sending a detection event to a SIEM solution, this event is sent again after a period of time specified in the configuration file.

You can configure how many times Feed Service attempts to send each detection event and the interval between attempts by using the EventSendingRetriesCount and EventSendingRetriesTimеout elements of the configuration file.

Feed Service in ReplyBack mode

In some cases, it is necessary for Feed Service to send responsive events back to the event source to the same socket from which the original events are received (ReplyBack mode).

To shift Feed Service to ReplyBack mode:

Send X-KF-ReplyBack as the first message after a TCP connection with Feed Service is established.

During the current TCP session every responsive event will be sent back to the same socket from which the original event was received. The setting specified in the OutputSettings > ConnectionString element of the configuration file will be ignored.

You can configure Feed Service to send an informational event indicating that event checking has finished.

To enable sending finishing events, perform one of the following actions:

• Set the enable attribute of the OutputSettings > FinishedEventFormat element of the Feed Service configuration file to true.
• Send X-KF-SendFinishedEventX-KF-ReplyBack as the first message after a TCP connection with Feed Service is established.

  In this case, the enable attribute of the OutputSettings > FinishedEventFormat element in the Feed Service configuration file will be ignored.

To instruct Feed Service that it must shift to ReplyBack mode and send finishing events:

Send X-KF-SendFinishedEventX-KF-ReplyBack as the first message after a TCP connection with Feed Service is established.

By default, in ReplyBack mode, Kaspersky CyberTrace does not save the statistics of the detection events received during the current connection.

To save the statistics of the detection events in ReplyBack mode:

• If you want Feed Service to send finishing events, send X-KF-SendFinishedEventX-KF-ReplyBackX-KF-SaveStatistic as the first message after a TCP connection with Feed Service is established.
• If you do not want Feed Service to send finishing events, send X-KF-ReplyBackX-KF-SaveStatistic as the first message after a TCP connection with Feed Service is established.

Features of event processing by Feed Service

This section outlines the way in which Feed Service processes incoming events.

Feed Service receives and sends events using TCP. Events are encoded in UTF-8.

Feed Service processes special characters in incoming events as follows:

• Feed Service treats the line feed character (0x0A) as the delimiter between events.

  Thus, text before a line feed character and text after it belong to different events.

• Feed Service ignores the carriage return character (0x0D).

Limitations on Feed Service incoming events

Feed Service processes events that are no larger than 64 kilobytes (KB). If an incoming event exceeds 64 KB, Feed Service splits it into several events and processes them separately. Thus, an indicator can be split in two and may not be extracted and checked.

We recommend that you configure the event source (the SIEM software), or the normalization rules, or both, so that the incoming events will not exceed 64 KB in size.

Feed Utility guide

This section explains how to use Feed Utility.

Working with feeds

Feed Utility is a tool that can download, filter, and compile Kaspersky Threat Data Feeds according to a specified set of rules defined in its configuration file. These rules can also be set by using Kaspersky CyberTrace Web.

Downloading

Feed Utility downloads archives containing feeds from the update servers. Each downloaded archive contains one feed. Before downloading Kaspersky Threat Data Feeds, Feed Utility checks whether they are newer than those being used. Before downloading OSINT and third-party feeds, Feed Utility does not perform such checking.

Feed Utility uses a certificate for authentication. The certificate also defines which Kaspersky Threat Data Feeds can be downloaded by Feed Utility. For example, if you have a demo certificate, Feed Utility can download only demo feeds.

Processing and filtering

After the archives containing feeds are downloaded, Feed Utility unpacks the archives and processes the original feed files. The feed files are modified according to a combination of feed rules, filtering rules, and other parameters specified in the Feed Utility configuration file. These parameters define the data that will be included in the resulting feeds, the output format of the resulting feeds, and the maximum number of records in the resulting feed.

Filtering is the process of modifying the original feed files according to specified filtering criteria. Filtering criteria are defined in the filtering rules for each feed. Depending on the intended Feed Utility usage scenario, you may want to create a feed that uses only a subset of information contained in the original feed. This can be achieved by using a combination of feed rules and filtering rules.

Default filtering rules

The default Feed Utility configuration file that is shipped in the Kaspersky CyberTrace distribution kit contains the following filtering rule for IP Reputation Data Feed and Demo IP Reputation Data Feed:

Only feed records whose threat_score parameter is not less than 75 are detected.

Kaspersky considers malicious those IP addresses whose threat_score is not less than 75. IP addresses whose threat_score is less than 75 are considered related to spam and posing no threat.

If you reduce the boundary value or remove the filter, you will have many detections by Demo IP Reputation Data Feed and IP Reputation Data Feed that will be mere notifications about detecting spam IP addresses.

To remove the filter:

1. Open the Settings > Feeds page of Kaspersky CyberTrace Web.
2. For Demo IP Reputation Data Feed (or IP Reputation Data Feed) remove the filtering rule for the threat_score field.

Compiling

If you use Feed Utility with Feed Service, feeds that contain URL masks must be converted to binary format. Feed Utility compiles the URL masks extracted from these feeds and creates binary files which are then used by Feed Service to quickly match URLs from received events to URL masks. Compiling is performed automatically by Feed Utility, if the UrlMatcherField option is specified in the feed rules.

Reloading

When notified, Feed Service reloads the feeds for use, that is, it unloads the old feeds from memory and loads the new ones.

Updating feeds

Feed Utility command-line options

Feed Utility is a console application. You can invoke it from the command line.

Syntax

Feed Utility uses the following syntax in Linux:

./kl_feed_util [options]

Feed Utility uses the following syntax in Windows:

kl_feed_util.exe [options]

Options

The following options are available:

• -h [ --help ]

  Prints the help message.

• -v [ --verbose ]

  Enables verbose mode.

  If verbose mode is enabled, Feed Utility prints detailed information about its activity to the screen. If verbose mode is disabled, brief information is printed.

• -s [ --silent ]

  Enables silent mode.

  If silent mode is enabled, Feed Utility does not print information about its activity to the screen.

• -c [ --config ] arg

  Specifies the path to the configuration file. The path must be specified in the arg argument.

  You can use absolute or relative paths. If a relative path is specified, it is calculated relative to the Feed Utilty binary file.

  The default value for this parameter is kl_feed_util.conf. Feed Utility searches for this file in the directory where its binary file is located.

• -d [ --download ]

  Enables downloading mode.

  If this option is specified, Feed Utility downloads feeds, but does not process them.

  Downloaded files will be located in the directory specified in the WorkDir parameter of the Feed Utility configuration file.

• -u [ --unpack ]

  Unpack downloaded feeds.

  If this option is specified, Feed Utility unpacks the feeds after downloading.

  This option can be used only in combination with -d or -p option.

• -p [ --processing ]

  Enables processing mode.

  If this option is specified, Feed Utility processes feeds, but does not download or unpack them. Feed Utility does not delete the original feed files.

  Feed Utility looks for feeds in the directory specified in the WorkDir parameter of the Feed Utility configuration file.

  In processing mode, Feed Utility does not delete the original feed files, located in the WorkDir directory. This may lead to a situation where this directory contains several versions of one feed file. In this case, Feed Utility will print an error message. To avoid this situation, you must manually delete the original feed files from the WorkDir directory after they are processed by Feed Utility.

• -f [--feed] arg

  Download or process the specified feed.

  The name of the feed must be specified in the arg argument. This name must correspond to the value of the Name parameter specified in feed rules (Feeds > Feed > Name).

  You can specify more than one feed. In this case, separate feed names with a semicolon (;).

  This parameter can be used with -d and -p parameters.

• -i [--input]

  Parses an external feed and converts it to JSON format according to parsing rules defined for this feed.

  The name of the feed must be specified with -f format.

• --set-proxy username:password@host:port

  Writes specified proxy connection settings to the Feed Utility configuration file. The username and password parameters are written in encrypted form.

  Specify the user name in the username parameter, password in the password parameter, and proxy server address and port in the host and port parameters.

  If a proxy server does not require authentication, use the --set-proxy host:port format.

• --set-taxii username:password@feedname@taxii-address@collectionname

  Writes specified TAXII server connection settings to the Feed Utility configuration file. The username and password parameters are written in encrypted form.

  If a TAXII server does not require authentication, use the feedname@taxii-address@collectionname format.

• --set-basic-auth username:password@feedname

  Writes the specified basic authentication settings to the Feed Utility configuration file. The username and password parameters are written in encrypted form.

  If a password is not required, use the username:@feedname format.

• --speedtest

  Measures the average speed with which Feed Utility downloads feeds from Kaspersky servers.

  You can combine this parameter with the -с parameter to specify the path to the configuration file that will be used.

Syntax examples

The following command runs Feed Utility with default parameters. Feed Utility will download, unpack, and process feeds.

• In Linux:

  ./kl_feed_util

• In Windows:

  kl_feed_util.exe

The following command runs Feed Utility in verbose mode with a configuration file named custom_configuration.conf, which is located in the same directory as the utility binary file.

• In Linux:

  ./kl_feed_util -v -c custom_configuration.conf

• In Windows:

  kl_feed_util.exe -v -c custom_configuration.conf

The following command makes Feed Utility download and unpack feeds.

• In Linux:

  ./kl_feed_util -d -u

• In Windows:

  kl_feed_util.exe -d -u

With the following command, Feed Utility processes the unpacked feeds. In this case, Feed Utility does not download the feeds; it only looks for the unpacked feed files and processes them.

• In Linux:

  ./kl_feed_util -p

• In Windows:

  kl_feed_util.exe -p

The following command makes Feed Utility unpack and process feeds.

• In Linux:

  ./kl_feed_util -u -p

• In Windows:

  kl_feed_util.exe -u -p

The following command makes Feed Utility download, unpack, and process the specified feed.

• In Linux:

  ./kl_feed_util -f Demo_Botnet_CnC_URL_Data_Feed

• In Windows:

  kl_feed_util.exe -f Demo_Botnet_CnC_URL_Data_Feed

The following command specifies proxy connection parameters. These parameters are written to the configuration file.

• In Linux:

  ./kl_feed_util --set-proxy 'user:pass@proxy.example.com:3128'

• In Windows:

  kl_feed_util.exe --set-proxy 'user:pass@proxy.example.com:3128'

The following command specifies proxy connection parameters for a proxy that does not require authentication. These parameters are written to the configuration file.

• In Linux:

  ./kl_feed_util --set-proxy 'proxy.example.com:3128'

• In Windows:

  kl_feed_util.exe --set-proxy 'proxy.example.com:3128'

The following command specifies TAXII server connection parameters. These parameters are written to the configuration file.

• In Linux:

  ./kl_feed_util --set-taxii 'user:pass@Example_Feed_Name@http://example.com@Example_Collection'

• In Windows:

  kl_feed_util.exe --set-taxii 'user:pass@Example_Feed_Name@http://example.com@Example_Collection'

The following command displays an average speed with which Feed Utility downloads the feeds from Kaspersky servers.

• In Linux:

  ./kl_feed_util --speedtest

• In Windows:

  kl_feed_util.exe --speedtest

Output example

The following example demonstrates a typical Feed Utility output. Feed Utility downloads demo feeds, and then unpacks and processes them.

Сonfiguring Feed Utility

This section explains how to configure Feed Utility.

About the configuration file (Feed Utility)

Feed Utility reads settings from a configuration file.

Editing the configuration file

If the configuration file is absent or does not follow the rules specified in this section, Feed Utility does not start and prints an error message.

We recommend that you create a backup copy of the Feed Utility configuration file before you make any changes in it. If Kaspersky CyberTrace does not work properly after you have reconfigured Feed Utility, replace the configuration file with its backup copy.

Configuration file location (Linux)

In Linux, the configuration file used by Feed Utility is named kl_feed_util.conf and is located in the %service_dir%/etc directory.

Configuration file location (Windows)

The configuration file used by Feed Utility is named kl_feed_util.conf and resides in the same directory as the Feed Utility binary file, %service_dir%/bin.

Encoding requirements

The Feed Utility configuration file, including all paths that it specifies, must be in UTF-8 encoding. If you use non-ASCII symbols in the configuration file, and the file is not in UTF-8 encoding, Feed Utility will not start.

Absolute and relative paths

When defining directories and files used by Feed Utility, you can use absolute and relative paths. If a relative path is specified, it is calculated from the directory that contains the Feed Utility binary file.

Configuration file parameters (Feed Utility)

Feed Utility reads the configuration parameters, feed rules, filtering rules, and parsing rules for feeds from the configuration file. This file is in XML format and has several groups of parameters.

The paths in the configuration file must contain only the characters used in the operating system locale, otherwise Feed Utility will not work.

Feed (feed rules, filtering rules, and parsing rules)

The Feed parameter contains rules for a particular feed. This element has several types of nested parameters:

• Feed rules specify how this particular feed must be processed by Feed Utility.
• Filtering rules are criteria that Feed Utility uses to filter the original feed files. Filtering rules are a part of feed rules for each feed.
• Parsing rules are rules for custom feeds (OSINT feeds, and other feeds that are not Kaspersky Threat Data Feeds). These parameters specify how each feed must be parsed by Feed Utility.

This parameter has the following attributes:

• enabled

  Specifies whether Feed Utility must download and process this feed.

  If enabled is true, Feed Utility downloads and processes the feed. If enabled is false, Feed Utility skips this feed.

The following example demonstrates how feed rules, filtering rules, and parsing rules are nested in the configuration file.

FeedsDir

The FeedsDir parameter specifies the directory where Feed Utility puts processed feed files.

WorkDir

The WorkDir parameter specifies the directory where Feed Utility puts the downloaded and unpacked feed files.

If this parameter is not specified, Feed Utility uses the default temporary directory of the operating system.

WorkDir cannot be equal to FeedsDir.

CertFile

The CertFile parameter specifies the path to the certificate file. This certificate is used by Feed Utility to download feeds.

The certificate file must be in PEM format.

SourceIPs

The SourceIPs parameter specifies the IP addresses that are used by Feed Utility to download feeds.

This parameter is optional. If it is omitted or has an empty value, Feed Utility resolves Kaspersky server addresses by their domain names.

You can specify one or more IPv4 addresses in this parameter. To specify several IP addresses, use the semicolon (";") as a delimiter.

The following example demonstrates specifying IP addresses in the SourceIPs parameter.

SourceDomains

The SourceDomains parameter specifies the domain names that are used by Feed Utility to download feeds.

You can specify one or more domain names in this parameter. To specify several domain names, use the semicolon (";") as a delimiter. Feed Utility will attempt to download feeds from the specified domain names in the order they appear in the configuration file.

When SourceDomains and SourceIP parameters are used together, domains specified in the SourceDomains parameter are used before IP addresses specified in the SourceIP parameter. If all attempts to download feeds fail, Feed Utility will generate an error message.

You can use Unicode symbols in this parameter.

The following example demonstrates specifying IP addresses in the SourceDomains parameter.

CreateExternalFeedInfoList path="PATH"

This parameter is obsolete. It is ignored in the current version of Kaspersky CyberTrace.

The CreateExternalFeedInfoList parameter specifies whether a list of supported OSINT feeds must be generated. This parameter is mandatory.

If this parameter is 1, Feed Utility creates a list of supported OSINT feeds, osint_feed_list.conf, in a directory specified in the path attribute. If you added any custom or third-party feeds to Kaspersky CyberTrace, Feed Utility also creates a list of these feeds, custom_feed_list.conf, in the same directory as osint_feed_list.conf.

If this parameter is 0, Feed Utility does not create a list of supported OSINT feeds.

The following example demonstrates specifying a path where the list must be created. In this example, the list will be created in a directory where Feed Utility binary is located.

NotifyKTFS path="PATH"

The NotifyKTFS parameter specifies whether Feed Service must be notified about the feed updates.

This parameter can be used only with json output format.

If this parameter is 1, Feed Utility notifies Feed Service that the feeds must be reloaded. A path to the Feed Service binary file must be specified in the path attribute of this parameter.

If this parameter is 0, Feed Utility does not notify Feed Service.

EULA

The EULA parameter specifies whether the terms of the End User License Agreement (EULA) were accepted by a user.

If this value is accepted, the terms of the EULA were accepted.

If this value is rejected, the terms of the EULA were not accepted. In this case, Feed Utility cannot be used.

RetryCount

The RetryCount parameter specifies the number of attempts to download a Kaspersky Threat Data Feed. Feed Utility tries to re-download a feed when a connection timeout, partial downloading, and other errors occur.

If the specified number of attempts were unsuccessful, Feed Utility displays an error message and continues its operation.

This parameter is used only for Kaspersky Threat Data Feeds. OSINT feeds and other custom feeds will not be re-downloaded by Feed Utility.

This parameter is optional. If this parameter is not specified, Feed Utility uses the default value of 10.

If this parameter is 0, the number of attempts is not limited.

SequentialDownload

The SequentialDownload parameter specifies whether Feed Utility must download feeds in sequential or parallel mode.

If this value is 1 or true, Feed Utility downloads feeds in sequential mode, one by one.

If this value is 0 or false, Feed Utility downloads feeds in parallel mode, all feeds at the same time.

By default, this parameter has the value of 0.

OutputFormat

The OutputFormat parameter defines the output format for all feeds. This parameter can have the following values:

• json

  The feeds are in JSON format. The feed files have a .json extension.

  This is the default value. If the OutputFormat parameter is omitted, this value is used to define the output format.

• txt

  The feeds are in plain text format (UTF-8 with BOM). The feed files have a .txt extension.

  • delimiter attribute

    In this format, record fields are separated with a delimiter. The default delimiter is ";". To specify a custom delimiter, use the delimiter attribute as follows:

    <OutputFormat delimiter="%delimiter%">txt</OutputFormat>

    Here, substitute %delimiter% with a symbol that must be used as a delimiter.

  • indicatorPerLine attribute

    To output one record field per line, set the indicatorPerLine attribute to 1 as follows:

    <OutputFormat indicatorPerLine="1">txt</OutputFormat>

    If you use this attribute, subfields specified in the RequiredFields feed rule must have the same parent field. For example, "files/MD5;files/SHA1" is valid, while "files/MD5;whois/domain" is invalid and will result in an error.

  If this output format is specified, all feed rules in the configuration file must include a RequiredFields parameter. The RequiredFields parameter specifies the order in which the fields are written to the output feed.

• csv

  Same as txt. The feed files have a .csv extension.

  You can use delimiter and indicatorPerLine attributes.

• openioc

  The feeds are in OpenIOC format. The feed files have an .ioc extension.

  You can specify the version of the OpenIOC format in the version attribute: it can be either 1.0, or 1.1. If the attribute is omitted, version 1.1 is used.

  Converting feeds to OpenIOC 1.0 format has some restrictions. Phishing URL Data Feed and Malicious URL Data Feed cannot be converted to OpenIOC 1.0 format; an error message is printed instead. For other feeds, only hash and IP address fields are converted. Converting feeds to OpenIOC 1.1 format has no such restrictions.

  Feeds in OpenIOC format take significantly more hard disk space than the original feed files.

• stix

  The feeds will be in STIX format. The files will have an .xml extension.

  For STIX format, feeds with URL masks must have the type field.

  Feeds in STIX format take significantly more hard disk space than the original feed files.