Scheduling feeds conversion

After you make the first conversion and import feeds into a MISP instance, you can make Kaspersky Threat Feed App for MISP run periodically at a specific interval using the cron utility.

Update interval

The specific update interval of a feed depends on a feed type and on its limit of records.

To schedule the conversion to be done every 30 minutes, run the following commands from the command line:

crontab -l > /tmp/misp_feeds_conv_crontab
echo "*/30 * * * * python %service_dir%/main.py" >> /tmp/misp_feeds_conv_crontab
crontab /tmp/misp_feeds_conv_crontab && echo "Success" || echo "Failed"

The cron utility will now execute the main.py script every 30 minutes.

If the converter script does not finish the feed conversion process by the time the cron utility runs the converter again, then an attempt to run it again will produce an error. After the converter script finishes the feed conversion process, it can be run again.

Conversion performance

The initial conversion process converts all records (within RECORDS_COUNT) from all the enabled feeds to MISP format. Subsequent converter calls make a diff with existing feeds but make no changes in records that were changed. The full update will launch only if the interval between the time of the last full update and the current moment exceeds the value defined in the full_update_interval_h parameter (by default, 12 hours).

Loading all the feeds into MISP can take up to several days, but you can create a pool of MISP instances with one instance for each feed. In this case, set up the converter on each node to convert only one feed.

If done regularly, subsequent feed updates should not take more than an hour. The less frequently the updates are done, the bigger the diff and, consequently, the longer the update process.

Page top