Connector, file type

Connectors of the file type are used to retrieve data from any text file. One string in a file is considered to be one event. Strings delimiter: \n. This type of connector is available for Linux agents and for Windows agents.

If while creating the collector at the Transport step of the installation wizard, you specified a connector of the file type, at the Event parsing in the Mapping table, you can pass the name of the file being processed by the collector or the path to the file in the KUMA event field. To do this, in the Source column, specify one of the following values:

$kuma_fileSourceName to pass the name of the file being processed by the collector in the KUMA event field.
$kuma_fileSourcePath to pass the path to the file being processed by the collector in the KUMA event field.

When you use a file connector, the new variables in the normalizer will only work with destinations of the internal type.

To read Windows files, you need to create a connector of the file type and manually install the agent on Windows. In one Windows Agent, you can configure multiple connections of different types, but there must be only one file type. The Windows agent must not read its files in the folder where the agent is installed.

We do not recommend running the agent under an administrator account; read permissions for folders/files must be configured for the user account of the agent. We do not recommend installing the agent on important systems; it is preferable to send the logs and read them on dedicated hosts with the agent.

Settings for a connector of the file type are described in the following tables.

Basic settings tab

Setting	Description
Name	Unique name of the resource. The maximum length of the name is 128 Unicode characters. Required setting.
Tenant	The name of the tenant that owns the resource. Required setting.
Type	Connector type: file. Required setting.
Tags
Path to the file.	The full path to the file that you want to interact with. For example, `/var/log/som?[1-9].log` or `с:\folder\logs.`. The following paths are not allowed: `(?i)^[a-zA-Z]:\\Program Files`. `(?i)^[a-zA-Z]:\\Program Files $x86$`. `(?i)^[a-zA-Z]:\\Windows`. `(?i)^[a-zA-Z]:\\ProgramData\\Kaspersky Lab\\KUMA`. File and folder mask templates Masks: `''`—matches any sequence of characters. `'[' [ '^' ] { <range of characters> } ']'`—class of characters (may not be left blank). `'?'`—matches any single character. Ranges of characters: `[0-9]` for numerals `[a-zA-Z]` for Latin alphabet characters Examples: `/var/log/som?[1-9].log` `/mnt/dns_logs//dns.log` `/mnt/proxy/access.log` Limitations when using prefixes in file paths Prefixes that cannot be used when specifying paths to files: `/` `/bin` `/boot` `/dev` `/etc` `/home` `/lib` `/lib64` `/proc` `/root` `/run` `/sys` `/tmp` `/usr/` `/usr/bin/` `/usr/local/` `/usr/local/sbin/` `/usr/local/bin/` `/usr/sbin/` `/usr/lib/` `/usr/lib64/` `/var/` `/var/lib/` `/var/run/` `/opt/kaspersky/kuma/` Files are available at the following paths: `/opt/kaspersky/kuma/clickhouse/logs/` `/opt/kaspersky/kuma/mongodb/log/` `/opt/kaspersky/kuma/victoria-metrics/log/` Limiting the number of files for watching by mask The number of files simultaneously watched by mask can be limited by the `max_user_watches` setting of the Core. To view the value of this setting, run the command: `cat /proc/sys/fs/inotify/max_user_watches` If the number of files for watching exceeds the value of the `max_user_watches` setting, the collector cannot read any more events from the files and the following error is written to the collector log: `Failed to add files for watching {"error": "no space left on device"}` To make sure that the collector continues to work correctly, you can configure the appropriate rotation of files so that the number of files does not exceed the value of the `max_user_watches` setting, or increase the `max_user_watches` value. To increase the value of this setting, run the command: `sysctl fs.inotify.max_user_watches=<number of files>` `sysctl -p` You can also add the value of the `max_user_watches` setting to sysctl.conf so make sure it is kept indefinitely. After you increase the value of the `max_user_watches` setting, the collector resumes correct operation. Required setting.
Modification timeout, sec	The time in seconds for which the file must not be updated for KUMA to apply the action specified in the Action after timeout drop-down list to the file. Default value: `0`, meaning that if the file is not updated, KUMA does not apply any action to it. The entered value must not be less than the value that you entered on the Advanced settings in the Poll interval, sec field.
Action after timeout	The action that KUMA applies to the file after the time specified in the Modification timeout, sec: Do nothing. The default value. Add a suffix adds the .kuma_processed extension to the file name and does not process the file even when it is updated. Delete deletes the file.
Auditd	This toggle switch enables the auditd mechanism to group auditd event lines received from the connector into an auditd event. If you enable this toggle switch, you cannot select a value in the Delimiter drop-down list because \n is automatically selected for the auditd mechanism. If you enable this toggle switch in the connector settings of the agent, you need to select \n in the Delimiter drop-down list in the connector settings of the collector to which the agent sends events. The maximum size of a grouped auditd event is approximately 4,174,304 characters.
Description	Description of the resource. The maximum length of the description is 4000 Unicode characters.

Advanced settings tab

Setting	Description
Debug	Ths switch enables resource logging. This toggle switch is turned off by default.
Buffer size	Buffer size in bytes for accumulating events in the RAM of the server before sending sending them for further processing or storage. The value must be a positive integer. Default buffer size: 1,048,576 bytes (1 MB). Maximum buffer size: 67,108,864 bytes (64 MB).
Number of handlers	Number of handlers that the service can run simultaneously to process response rules in parallel. To determine the number of handlers, you can use the following formula: (<number of CPUs> / 2) + 2.
File/folder polling mode	Specifies how the connector rereads files in the directory: Monitor changes means the connector rereads files in the directory at an interval in milliseconds specified in the Poll interval, ms field if the files are not being updated. The default value. For example, if the files are constantly being updated, and the value of Request interval, ms is `5000`, the connector rereads the files continuously instead of every 5000 milliseconds. If the files are not being updated, the connector rereads them every 5000 milliseconds. Track periodically means the connector rereads files in the directory at an interval in milliseconds specified in the Polling interval, ms field, regardless of whether the files are being updated or not.
Poll interval, ms	The interval in milliseconds at which the connector rereads files in the directory. Default value: `0` means the connector rereads files in the directory every 700 milliseconds. In the File/folder polling mode drop-down list, select the mode the connector must use to reread files in the directory. The entered value must not be less than the value that you entered on the Basic settings in the Modification timeout, sec field. We recommend entering a value less than the value that you entered in the Event buffer TTL field because this may adversely affect the performance of Auditd.
Character encoding	Character encoding. The default is UTF-8.
Event buffer TTL	Buffer lifetime for auditd event lines, in milliseconds. Auditd event lines enter the KUMA collector and accumulate in the buffer. This allows multiple auditd event lines to be grouped into a single auditd event. The buffer lifetime countdown begins when the first auditd event line is received or when the previous buffer lifetime expires. Possible values: 700 to 30,000. The default value is `2000`. This field is available if you have enabled the Auditd toggle switch on the Basic settings tab. The auditd event lines accumulated in the buffer are kept in the RAM of the server. We recommend caution when increasing the buffer size because memory usage by the KUMA collector may become excessive. You can see how much server RAM the KUMA collector is using in KUMA metrics. If you want a buffer lifetime to exceed 30,000 milliseconds, we recommend using a different auditd event transport. For example, you can use an agent or pre-accumulate auditd events in a file, and then process this file with the KUMA collector.
Transport header	Regular expression for auditd events, which is used to identify auditd event lines. You can use the default value or edit it. The regular expression must contain the `record_type_name`, `record_type_value`, and `event_sequence_number` groups. If a multi-line auditd event contains a prefix, the prefix is retained for the first line of the auditd event and discarded for the following lines. You can revert to the default regular expression for auditd events by clicking Reset to default value.

Page top