โ† Nvidia questions

Gradient descent, vanishing gradient, feature selection vs reduction, PCA, overfitting

Filed under Nvidia

Free read 3 of 3

Gradient descent updates parameters opposite the gradient of the loss: theta = theta - lr * grad (batch, stochastic or mini-batch). Vanishing gradients occur when many small derivatives multiply in deep networks (sigmoid or tanh saturation), so early layers barely learn. Fixes: ReLU, batch norm, residual connections, good initialisation, LSTM or GRU.

The question

Explain gradient descent, the vanishing gradient problem, feature selection versus feature reduction, the steps of PCA, and ways to reduce overfitting.

Approach

  1. Gradient descent updates parameters opposite the gradient of the loss: theta = theta - lr * grad (batch, stochastic or mini-batch). Vanishing gradients occur when many small derivatives multiply in deep networks (sigmoid or tanh saturation), so early layers barely learn. Fixes: ReLU, batch norm, residual connections, good initialisation, LSTM or GRU.

Tips

  • What they are testing: Breadth across core ML topics, with the right cause-and-fix reasoning.
  • Common mistake: Confusing feature selection with PCA, or forgetting to standardise before PCA.

Related Nvidia questions

Collected from public candidate posts and coding forums. Updated Nov 2026.