Back to Study Material GATE Exam

Probability and Statistics

Probability theory provides the language for reasoning about uncertainty, and GATE DA tests it heavily because every learning algorithm from naive Bayes to...

Probability and Statistics

Foundations of Probability for GATE DA

Probability theory provides the language for reasoning about uncertainty, and GATE DA tests it heavily because every learning algorithm from naive Bayes to Gaussian processes rests on probabilistic assumptions. The exam expects candidates to move fluently between definitions, rules, and applications without hesitation.

A probability space consists of a sample space omega, a sigma algebra F of events, and a probability measure P. For GATE purposes, you work mostly with discrete and continuous random variables. A discrete random variable X assigns a real number to each outcome, and its probability mass function satisfies P(X = x) >= 0 for all x and the sum of all probabilities equals 1. A continuous random variable is characterized by a probability density function f(x), where the probability of X falling in an interval [a, b] is the integral of f from a to b.

Key rules include the addition rule: P(A union B) = P(A) + P(B) - P(A intersection B). The multiplication rule: P(A intersection B) = P(A) * P(B|A). Bayes theorem: P(A|B) = P(B|A) * P(A) / P(B). Bayes theorem is arguably the most important single formula in the GATE DA syllabus because it underlies generative classifiers, posterior inference, and model selection.

Expectation and variance are the two most important summary statistics for a random variable. E[X] = sum of x * P(X=x) for discrete variables. Var(X) = E[X^2] - (E[X])^2. The standard deviation is the square root of variance. For a linear transformation Y = aX + b, E[Y] = aE[X] + b and Var(Y) = a^2 Var(X). These linearity properties are tested frequently in GATE questions about transformed variables.

Common distributions you must know: Bernoulli, Binomial, Poisson, Uniform, Gaussian, Exponential, and Gamma. For each, know the PMF or PDF, mean, variance, and at least one application context. The Poisson distribution models count data with mean = variance = lambda, which makes it easy to identify in exam problems.

Statistical Inference and Hypothesis Testing

Statistical inference is the process of drawing conclusions about a population from a sample. GATE DA covers both frequentist and Bayesian perspectives, and questions test whether you understand the philosophical difference as well as the computational mechanics.

Point estimation involves choosing a single value as the best guess for an unknown parameter. Maximum likelihood estimation, or MLE, finds the parameter value that maximizes the probability of the observed data. For a Gaussian with unknown mean mu and known variance sigma^2, the MLE of mu is simply the sample mean. For a Bernoulli distribution with unknown probability p, the MLE is the proportion of successes in the sample. Understanding why these formulas emerge from the calculus of likelihood maximization is more useful than memorizing them.

Confidence intervals quantify the uncertainty in a point estimate. A 95 percent confidence interval for a mean constructed from a large sample is approximately xbar ± 1.96 * (sigma / sqrt(n)), where xbar is the sample mean, sigma is the population standard deviation, and n is the sample size. A crucial and frequently tested conceptual point is that the confidence interval is not a probability statement about the parameter; it is a statement about the procedure. If you repeated the experiment many times, 95 percent of the intervals constructed this way would contain the true parameter.

Hypothesis testing follows a structured logic. You state a null hypothesis H0 and an alternative H1, choose a significance level alpha (commonly 0.05), compute a test statistic, and compare it to a critical value or compute a p-value. If the p-value is less than alpha, you reject H0. Type I error is rejecting a true H0, with probability alpha. Type II error is failing to reject a false H0, with probability beta. Power is 1 - beta.

GATE tip: for a two-sample t-test question, check whether the variances are stated as equal or unequal, as this changes the pooled versus Welch formulation. Read every hypothesis testing problem carefully for these qualifying conditions before writing any formula.

Distributions, Correlation, and Regression Concepts in Statistics

The joint behavior of two random variables is described by their joint distribution. For discrete variables, the joint PMF is P(X=x, Y=y). Marginal distributions are obtained by summing over the other variable. Conditional distributions are obtained by dividing the joint by the marginal. Independence means the joint equals the product of marginals for all values, which is a stronger condition than uncorrelated.

Covariance measures the linear tendency of two variables to move together. Cov(X, Y) = E[XY] - E[X]E[Y]. The correlation coefficient rho = Cov(X,Y) / (sigma_X * sigma_Y) normalizes covariance to the range [-1, 1]. A correlation of 1 means perfect positive linear relationship; -1 means perfect negative; 0 means no linear relationship but not necessarily independence. GATE DA frequently tests the distinction between zero correlation and independence.

Simple linear regression models the relationship Y = beta_0 + beta_1 * X + epsilon, where epsilon is a zero-mean noise term. The OLS estimators minimize the sum of squared residuals. The formula for the slope is beta_1 = Cov(X,Y) / Var(X) and the intercept is beta_0 = Ybar - beta_1 * Xbar. The coefficient of determination R^2 measures the proportion of variance in Y explained by the model, ranging from 0 to 1.

The Central Limit Theorem states that the sample mean of n independent, identically distributed random variables with finite mean mu and variance sigma^2 converges in distribution to a Normal with mean mu and variance sigma^2/n as n grows large. This is what justifies using the normal distribution for inference even when the underlying data is not normal, and it is a cornerstone result for understanding why so many statistical tests assume normality.

Worked example: if X and Y are independent standard normals, then X + Y is Normal with mean 0 and variance 2, so standard deviation sqrt(2). Questions of this form appear regularly, testing the additive property of independent normal random variables.