ANN2SNN package

The ANN2SNN package is intended for creating neural networks that can be converted into spiking neural networks and placed for execution on the AltAI-1 neuromorphic processor. The ANN2SNN package is implemented in Python.

To create neural networks, it is necessary to use special ternary layers, which take into account the limitations typical of the AltAI-1 neuromorphic processor. The ternary layers used to create the neural network correspond to the layers in the Keras library, with the following main differences:

• The input and output data of each of the layers of the neural network are binary tensors of the corresponding dimension.
• All weights of the neural network layers are in the range {-1,0,1}.
• The bias value in the neural network layer is in the range from –32,768 to 32,767.

When training a neural network created using ternary layers, the following features should be taken into account:

• If non-binarized data exists for training and/or validation, it must first be converted to a binary form. For this, you can use encoding and decoding algorithms or create neural network training blocks that encode and decode data.
• When training binary classification models using the gradient descent method, for the differentiability of the loss function, you must specify the sigmoid function as the output layer activation function. After the training, you will need to replace the sigmoid activation of the output layer with activation using the Heaviside function (Heaviside step function).
• When training multi-class classification models with the gradient descent method, for the differentiability of the loss function, you must specify the multivariate logistic function (softmax function) as the output layer activation function. When such a neural network is placed, the output vector is automatically converted to the binary form that implements the indicator of the maximum output vector component.

Serialization and deserialization of neural networks is performed using standard methods provided by the Keras library, with the custom_objects parameter specified in the form of a dictionary of ternary layers and the Heaviside activation function.

During placement, the neural network is validated against the processing unit's constraints. The neural network is validated against the neuromorphic processing unit configuration settings specified in the YAML file. As a result of placement, information about the number of cores required to run a neural network on the AltAI-1 neuromorphic processor is displayed. The neural network is converted into a format supported by the processing unit, to be then executed on the hardware processor or its software emulator later. If the neural network cannot be placed on the neuromorphic processing unit because the maximum allowed number of cores is exceeded, an appropriate error is displayed.

The AltAI-1 neuromorphic processor provides the ability to scale by increasing the number of VLSI chips. For questions regarding scaling, please contact Kaspersky specialists.

You can make a rough estimate of the number of cores required to execute a neural network using the formula below, where:

• x corresponds to the number of cores required. After calculating the number of cores using the formula, round the result up.
• i corresponds to the ordinal number of the neural network layer.
• n corresponds to the number of layers of the neural network.
• w corresponds to the first dimension of the tensor of the neural network layer.
• h corresponds to the second dimension of the tensor of the neural network layer.
• c corresponds to the third dimension of the tensor of the neural network layer.

  

  Formula for estimating the number of cores

The number of cores required for placing a fully connected TernaryDense layer depends on the number of input neurons in the layer. If the number of input neurons does not exceed 512, you can calculate the required number of cores by using the formula below, where:

• x corresponds to the number of cores required. After calculating the number of cores using the formula, round the result up.
• i corresponds to the ordinal number of the neural network layer.
• n corresponds to the number of fully connected layers of the neural network.
• U is the number of target neurons.

  

  Formula for estimating the number of cores for the TernaryDense layer

If the number of input neurons exceeds 512, you can calculate the required number of cores by using the formula below, where:

• x corresponds to the number of cores required. After calculating the number of cores using the formula, round the result up.
• i corresponds to the ordinal number of the neural network layer.
• n corresponds to the number of fully connected layers of the neural network.
• U is the number of target neurons.
• N is the number of input neurons for the layer.

  

  Formula for estimating the number of cores for the TernaryDense layer

For neural networks created on the basis of the ANN2SNN package, the following inference options are possible:

• Inference for the number of ticks equal to the number of layers in the neural network. With this inference option, only one layer of the neural network is calculated per tick of neural network execution, and the processed data is transmitted to the next layer. The inference time is directly proportional to the number of layers in the neural network.
  

  The time slice during which the function of all neurons in the core is calculated. During a tick, all spikes generated by the cores or received on the input lines of neurons are delivered to the target neurons.

• Inference per tick. With this inference option, each layer of the neural network receives data from the previous layer and processes it. The inference time is about 1.5 ms.

Page top