ML Interview Questions

Can you explain the vanishing gradient problem in deep neural networks and discuss several methods to mitigate it?

Explain batch normalization in deep learning. How does it work, and what are its benefits and limitations?

Describe and compare the ReLU, sigmoid, tanh, and other common activation functions used in neural networks. Discuss their characteristics, advantages, and limitations, and explain in which scenarios each would be most suitable.

Explain the key components of a Convolutional Neural Network (CNN) architecture, detailing the purpose of each component. How have CNN architectures evolved over time to improve performance and efficiency? Provide examples of notable architectures and their contributions.

Describe how backpropagation is utilized to optimize neural networks. What are the mathematical foundations of this process, and how does it impact the learning of the model?

Explain attention mechanisms in deep learning. Compare different types of attention (additive, multiplicative, self-attention, multi-head attention). How do they work mathematically? What problems do they solve? How are they implemented in modern architectures like transformers?