When does Vanishing Gradients occur?
Vanishing Gradient occurs when the derivative or slope becomes smaller as we go backward layer by layer during backpropagation. When weights update is small the training time takes long; this may even bring to a complete halt the neural network training. Vanishing Gradient: with sigmoid and tanh activation fn, as the derivatives of sigmoid and…
Book: Automated Machine Learning

The books collect papers written in the context of successful competitions in machine learning. They also include analyses of the challenges, tutorial material, dataset descriptions, and pointers to data and software. Together with the websites of the challenge competitions, they offer a complete teaching toolkit and a valuable resource for engineers and scientists
Book: Introduction to Scientific Programming with Python

This book introduces programming for scientific and computational applications using the Python programming language. The presentation style is compact and example-based, making it suitable for students and researchers with little or no prior experience in programming. The book uses relevant mathematics and natural science examples to present programming as a practical toolbox that can quickly…
Top Machine Learning Algorithms
Top machine learning algorithms and tasks.
Most used activation functions in Neural Networks
Here is a representation of the most commonly used activation functions in Neural Networks (with formula) . Activation functions are not only important in the final output layer (the one that then gives us the result), but also and especially in the internal propagation layers. These functions are sensitive to the “z” value, which is…
Principal Component Analysis – PCA
𝐏𝐫𝐢𝐧𝐜𝐢𝐩𝐚𝐥 𝐂𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭 𝐀𝐧𝐚𝐥𝐲𝐬𝐢𝐬 (𝐏𝐂𝐀), was one of the earliest methods used to 𝐝𝐞𝐭𝐞𝐫𝐦𝐢𝐧𝐞 𝐭𝐡𝐞 𝐜𝐨𝐫𝐫𝐞𝐥𝐚𝐭𝐢𝐨𝐧𝐬 among data and to reduce the dimensionality of the space, 𝑿, by projecting it into a lower dimensional space 𝒁. 𝐏𝐂𝐀 attempts to find a mapping 𝒇 that preserves, to the 𝐠𝐫𝐞𝐚𝐭𝐞𝐬𝐭 𝐞𝐱𝐭𝐞𝐧𝐭 𝐩𝐨𝐬𝐬𝐢𝐛𝐥𝐞, 𝐭𝐡𝐞 𝐯𝐚𝐫𝐢𝐚𝐧𝐜𝐞 of the data points…



