Source

Contains parameters for a specific event source.

The regular expressions and event normalizing rules specified in the configuration file are grouped by event sources that are represented by Source elements. Usually these event sources are devices that issue events, which afterward are checked by Feed Service. Every Source element contains a set of rules. There can be one or more Source elements in the InputSettings > RegExps element.

Rules workflow

The way Feed Service chooses rules from different Source elements is described in the following flow chart.

sources_flowchart

Choosing a rule

Note that event normalizing rules are applied first and regular expressions are applied afterwards.

The regular expressions of the default event source for finding URLs, IP addresses, and hashes are universal; that is, they can be used for parsing events issued by most devices. They can be used for parsing events that contain multiple URLs, but cannot be used, for example, for parsing events that contain URLs with no protocol specified. The use of universal regular expressions lowers the performance of Feed Service, compared to using device-specific regular expressions. Also, the universal regular expressions do not handle the dispersal, in an event, of different parts of a URL (for example, the host and the path). The universal regular expressions for finding hashes can extract symbol sequences that actually are not hashes.

Special event sources

There must be an event source with the default identifier (<Source id="default">). Rules of the default event source have lower priority than rules specific to the event source. Rules specific to the event source are applied first. Rules of the default event source are applied next. If a rule specific to the event source and a rule of the default event source have the same name, the rule of the default event source is applied only if the rule specific to the event source had no matches.

There are two special event sources that you can use: http_single_lookup (<Source id="http_single_lookup">) and http_file_lookup (<Source id="http_file_lookup">).

The rules of the http_single_lookup event source are used when single values are searched for by using CyberTrace Web.

The rules of the http_file_lookup event source are used when hashes of specified files or indicators in log files are searched for by using CyberTrace Web. Therefore, if you have to search for values contained in a log file that has a special format, we recommend specifying rules for the http_file_lookup event source.

If the configuration file contains the http_single_lookup event source or the http_file_lookup event source, we strongly recommend that you do not remove the regular expressions specified in these special event sources by default and, instead, edit them as needed.

Path

InputSettings > RegExps > Source

Attributes

This element has the following attributes.

Source element attributes

Attribute

Description

id

A unique identifier of the event source.

In detection events, the identifier of the event source can be referred to by the %SourceId% pattern.

ip

The IP address of the event source.

If an event has arrived from an event source that has the specified IP address, the event is processed by using the rules contained in this Source element. If the IP address of the event source is not among those specified in the ip attribute of the Source elements, the host name of the event source is determined and a Source element that has this host name in the hostname attribute is searched for. The rules from that Source element are used for processing the event.

The ip attribute cannot be set for the default, http_single_lookup, and http_file_lookup event sources.

hostname

The host name of the event source. The value of the host name is extracted from the event. In syslog events, the host name follows the timestamp (https://tools.ietf.org/html/rfc5424). For example, in the event Feb 2 11:57:59 sample-hostname alert: sample event text, the host name is sample-hostname.

If an event has arrived from an event source that has the specified host name and the IP address of the event source is not among those specified in the ip attribute of the Source elements, the event is processed by using the rules contained in this Source element.

The hostname attribute cannot be set for the default, http_single_lookup, and http_file_lookup event sources.

regex

The regular expression that is used to determine if an event comes from the source.

The specified regular expression is applied to an event. If the regular expression matches the event one or more times, the event is considered to be from the source. In this case, the event is processed by using the rules contained in this Source element.

The hostname attribute cannot be set for the default, http_single_lookup, and http_file_lookup event sources.

Nested elements

This element is a container for the following nested elements:

Example

The following is an example of this element.

<Source id="CustomSource" ip="192.0.2.15">

<RE_MD5 type="MD5" extract="all">([\da-fA-F]{32})</RE_MD5>

<RE_SHA1 type="SHA1" extract="all">([\da-fA-F]{40})</RE_SHA1>

<RE_SHA256 type="SHA256" extract="all">([\da-fA-F]{64})</RE_SHA256>

<NormalizingRules>

...

</NormalizingRules>

</Source>

Page top