Complicated SQL queries

You can use the search string to manually create SQL queries of any complexity for filtering events.

To manually generate an SQL query:

Go to the Events section of the KUMA web interface.
An input form opens.
Enter your SQL query into the input field.
Click the button.

You will see a table of events that satisfy the criteria of your query. If necessary, you can filter events by period.

Supported functions and operators

SELECT—event fields that should be returned.
For SELECT fields, the program supports the following functions and operators:
- Aggregation functions: count, avg, max, min, sum.
- Arithmetic operators: +, -, *, /, <, >, =, !=, >=, <=.
  You can combine these functions and operators.
  
  If you are using aggregation functions in a query, you cannot customize the events table display, sort events in ascending or descending order, receive statistics, or perform a retroscan.
FROM—data source.
When creating a query, you need to specify the events value as the data source.
WHERE—conditions for filtering events.
- AND, OR, NOT, =, !=, >, >=, <, <=
- IN
- BETWEEN
- LIKE
- ILIKE
- inSubnet
- match (the re2 syntax of regular expressions is used in queries)
GROUP BY—event fields or aliases to be used for grouping the returned data.
If you are using data grouping in a query, you cannot customize the events table display, sort events in ascending or descending order, receive statistics, or perform a retroscan.
ORDER BY—columns used as the basis for sorting the returned data.
Possible values:
- DESC—descending order.
- ASC—ascending order.
OFFSET—skip the indicated number of lines before printing the query results output.

LIMIT—number of strings displayed in the table.

The default value is 250.

If you are filtering events by user-defined period and the number of strings in the search results exceeds the defined value, you can click the Show next records button to display additional strings in the table. This button is not displayed when filtering events by the standard period.

Example queries:

SELECT * FROM `events` WHERE Type IN ('Base', 'Audit') ORDER BY Timestamp DESC LIMIT 250
In the events table, all events with the Base and Audit type are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.
SELECT * FROM `events` WHERE BytesIn BETWEEN 1000 AND 2000 ORDER BY Timestamp ASC LIMIT 250
All events of the events table for which the BytesIn field contains a value of received traffic in the range from 1000 to 2000 bytes are sorted by the Timestamp column in ascending order. The number of strings that can be displayed in the table is 250.
SELECT * FROM `events` WHERE Message LIKE '%ssh:%' ORDER BY Timestamp DESC LIMIT 250
In the events table, all events whose Message field contains data corresponding to the defined %ssh:% template in lowercase are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.
SELECT * FROM `events` WHERE inSubnet(DeviceAddress, '10.0.0.1/24') ORDER BY Timestamp DESC LIMIT 250
In the events table, all events for hosts that are in the 10.0.0.1/24 subnet are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.
SELECT * FROM `events` WHERE match(Message, 'ssh.*') ORDER BY Timestamp DESC LIMIT 250
In the events table, all events whose Message field contains text corresponding to the ssh.* template are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.
SELECT max(BytesOut) / 1024 FROM `events`SELECT
Maximum amount of outbound traffic (KB) for the selected time period.
SELECT count(ID) AS "Count", SourcePort AS "Port" FROM `events` GROUP BY SourcePort ORDER BY Port ASC LIMIT 250
Number of events and port number. Events are grouped by port number and sorted by the Port column in ascending order. The number of strings that can be displayed in the table is 250.

The ID column in the events table is named Count, and the SourcePort column is named Port.

If you want to use a special character in a query, you need to escape this character by placing a backslash (\) character in front of it.

Example:

SELECT * FROM `events` WHERE match(Message, 'ssh:\'connection.*') ORDER BY Timestamp DESC LIMIT 250

In the events table, all events whose Message field contains text corresponding to the ssh: 'connection' template are sorted by the Timestamp column in descending order. The number of strings that can be displayed in the table is 250.

When switching to the query builder, the query parameters that were manually entered into the search string are not transferred to the builder so you will need to create your query again. Also, the query created in the builder does not overwrite the query that was entered into the search string until you click the Apply button in the builder window.

After updating KUMA to version 1.6, event filtering that uses an SQL query containing the inSubnet condition may result in error Code: 441. DB::Exception: Invalid IPv4 value. If this is the case, you must add the directive <cast_ipv4_ipv6_default_on_conversion_error>true</cast_ipv4_ipv6_default_on_conversion_error> in the profiles → default section of the file /opt/kaspersky/kuma/clickhouse/cfg/config.d/users.xml on the storage servers (on each machine of the ClickHouse cluster).

For more details on SQL, refer to the ClickHouse documentation.