Machine Learning Quiz
This section presents worked practice questions in the style of GATE DA to consolidate your supervised learning knowledge. Each question is followed by a d...
Practice Question Set: Supervised Learning and Model Evaluation
This section presents worked practice questions in the style of GATE DA to consolidate your supervised learning knowledge. Each question is followed by a detailed explanation that reinforces both the answer and the underlying concept.
Question 1: A logistic regression model predicts probabilities using the sigmoid function. The decision boundary is the set of points where the predicted probability equals 0.5. What condition on the linear combination z = w^T x does this correspond to? Answer: sigma(z) = 0.5 when z = 0, so the decision boundary is the hyperplane w^T x = 0. This is a plane (or line in 2D) passing through the origin of the feature space (shifted by bias).
Question 2: A decision tree is trained on a dataset where the root node contains 40 positive and 60 negative examples. After splitting on feature A, the left child has 30 positive and 10 negative, and the right child has 10 positive and 50 negative. What is the information gain of feature A? Entropy of root: H = -(0.4 log2 0.4 + 0.6 log2 0.6) ≈ 0.971. Left child: H_L = -(0.75 log2 0.75 + 0.25 log2 0.25) ≈ 0.811. Right child: H_R = -(0.167 log2 0.167 + 0.833 log2 0.833) ≈ 0.650. Weighted average: (40/100)*0.811 + (60/100)*0.650 = 0.324 + 0.390 = 0.714. Information gain = 0.971 - 0.714 = 0.257.
Question 3: A classifier achieves TP=80, TN=70, FP=20, FN=30. Compute precision, recall, and F1. Precision = 80/(80+20) = 0.8. Recall = 80/(80+30) ≈ 0.727. F1 = 2*0.8*0.727/(0.8+0.727) ≈ 0.762.
Question 4: If training accuracy is 98 percent and test accuracy is 65 percent, the model is exhibiting: (A) underfitting, (B) overfitting, (C) optimal fit, (D) high bias. Answer: (B) overfitting. The large train-test gap indicates the model has memorized the training data.
Exam strategy: on GATE MCQ questions requiring numerical computation, work out the answer before looking at the options to avoid being misled by near-correct distractors. On MSQ (multiple select) questions, eliminate clearly wrong options first, then verify each remaining option independently.
Practice Question Set: Unsupervised Learning and Neural Networks
This section continues the GATE DA practice quiz with unsupervised learning and neural network questions at the level of difficulty you will encounter in the actual exam.
Question 1: K-means clustering is run on a 1D dataset with points {1, 2, 8, 9} with k=2. Initial centroids are 1 and 8. After one iteration, what are the new centroids? Assign each point to the nearest centroid: 1->1, 2->1, 8->8, 9->8. New centroid 1 = (1+2)/2 = 1.5. New centroid 2 = (8+9)/2 = 8.5. After one iteration, centroids are 1.5 and 8.5.
Question 2: In PCA, the first principal component of a dataset has eigenvalue 6 and the second has eigenvalue 2. The remaining eigenvalues each equal 0.5 and there are 8 of them. What fraction of variance does the first component explain? Total variance = 6 + 2 + 8*0.5 = 6 + 2 + 4 = 12. Fraction = 6/12 = 0.5 or 50 percent.
Question 3: A neural network has one hidden layer with ReLU activation and a softmax output layer. If all input values to the hidden layer are negative, what is the output of the hidden layer? ReLU(x) = 0 for x <= 0, so all hidden units output 0. The input to the softmax layer is all zeros (after multiplying zero activations by weights, assuming no bias, or bias-only if biases exist). Softmax of a zero vector produces a uniform distribution.
Question 4: Which regularization technique produces sparse weight vectors in neural networks? (A) L2 weight decay, (B) L1 regularization, (C) Dropout, (D) Batch normalization. Answer: (B) L1 regularization, because the subgradient of the L1 penalty drives weights to exactly zero.
Question 5: The SVM dual problem has the property that the solution w = sum over support vectors of [alpha_i * y_i * x_i]. If only 3 of 1000 training points have non-zero alpha_i, how many support vectors are there? Answer: 3. Non-zero alpha_i values correspond exactly to support vectors.
Review tip: after each practice question, identify which formula or concept the question tested and verify you could reproduce the solution from scratch in under two minutes. Slow solutions indicate a gap in automatic recall that needs one more targeted review.
Practice Question Set: ML Algorithms, Complexity, and GATE Patterns
This final quiz section focuses on algorithmic questions about machine learning methods and the meta-skill of recognizing question patterns in GATE DA.
Question 1: Gradient descent for linear regression updates weights as w = w - alpha * (2/n) * X^T (Xw - y). If the learning rate alpha is too large, what behavior do you expect? The updates overshoot the minimum, causing the loss to oscillate or diverge. This is detectable by monitoring training loss: if it increases or oscillates, reduce alpha.
Question 2: The VC dimension of a linear classifier in d dimensions is d+1. If d=2, the VC dimension is 3, meaning a linear classifier can shatter any set of 3 points but not all sets of 4 points. This concept quantifies the model complexity and appears in GATE questions about capacity and generalization bounds.
Question 3: Adaboost assigns weights to training examples. Misclassified examples in one round receive higher weights in the next round. After t rounds, the final classifier is a weighted majority vote of t weak classifiers. The error rate of the ensemble decreases exponentially with t if each weak learner has accuracy better than 50 percent. What is required of each weak learner? It must perform better than random guessing (accuracy > 0.5), not necessarily well.
Question 4: The kernel function K(x, z) = (x^T z + 1)^2 expands to the inner product in a feature space that includes all degree-2 polynomial features plus the original features plus a constant. Using this kernel in an SVM allows learning quadratic decision boundaries while solving an optimization problem in the original input space. This is the computational advantage of kernels.
Question 5: When comparing random forests to gradient boosting, random forests train trees in parallel (independent) and gradient boosting trains them sequentially (dependent). Random forests have lower variance due to bagging; gradient boosting has lower bias due to sequential correction. For large datasets with sufficient computation, gradient boosting often achieves better accuracy but is more prone to overfitting with excessive trees and a high learning rate.
GATE pattern recognition: questions about ensemble methods almost always contrast at least two algorithms. Prepare two to three distinguishing facts about each pair: random forest vs gradient boosting, bagging vs boosting, hard margin SVM vs soft margin SVM.