Event stream control using nginx

To control the event stream using nginx, you need to create and configure an nginx server to receive events from the event source and then forward these to collectors.

To enable nginx event stream control on the event source server:

1. Create two or more identical collectors that you want to use to ensure uninterrupted reception of events.
2. Install nginx on the server intended for event stream control.
   • Installation command in Oracle Linux 8.6:

     $sudo dnf install nginx

   • Installation command in Ubuntu 20.4:

     $sudo apt-get install nginx

     When installing from sources, you must compile with the parameter -with-stream option:
     $ sudo ./configure -with-stream -without-http_rewrite_module -without-http_gzip_module

3. On the nginx server, add the stream module to the nginx.conf configuration file that contains the rules for forwarding the stream of events between collectors.

   Example stream module

   Example module in which event stream is distributed between the collectors kuma-collector-01.example.com and kuma-collector-02.example.com, which receive events via TCP on port 5140 and via UDP on port 5141. Balancing uses the nginx.example.com ngnix server.

   stream {  upstream syslog_tcp { server kuma-collector-1.example.com:5140; server kuma-collector-2.example.com:5140; } upstream syslog_udp { server kuma-collector-1.example.com:5141; server kuma-collector-2.example.com:5141; }  server { listen nginx.example.com:5140; proxy_pass syslog_tcp; } server { listen nginx.example.com:5141 udp; proxy_pass syslog_udp; proxy_responses 0; } }  worker_rlimit_nofile 1000000; events { worker_connections 20000; } # worker_rlimit_nofile is the limit on the number of open files (RLIMIT_NOFILE) for workers. This is used to raise the limit without restarting the main process. # worker_connections is the maximum number of connections that a worker can open simultaneously.

   With a large number of active services and users, you may need to increase the limit of open files in the nginx.conf settings. For example:

   worker_rlimit_nofile 1000000;

   events {

   worker_connections 20000;

   }

   # worker_rlimit_nofile is the limit on the number of open files (RLIMIT_NOFILE) for workers. This is used to raise the limit without restarting the main process.

   # worker_connections is the maximum number of connections that a worker can open simultaneously.

4. Restart nginx by running the following command:

   systemctl restart nginx

5. On the event source server, forward events to the nginx server.

Event stream control is now enabled on the event source server.

Nginx Plus may be required to fine-tune balancing, but certain balancing methods, such as Round Robin and Least Connections, are available in the base version of ngnix.

For more details on configuring nginx, please refer to the nginx documentation.