Here is a representation of the most commonly used activation functions in Neural Networks (with formula) .

Activation functions are not only important in the final output layer (the one that then gives us the result), but also and especially in the internal propagation layers.

These functions are sensitive to the “z” value, which is given by the sum of the multiplication of the features by each of their own weights connected to a specific node plus the final bias.