Probability in AI
Probability theory provides the mathematical framework for reasoning under uncertainty, which is an unavoidable feature of most real-world AI problems. Sen...
Managing Uncertainty
Probability theory provides the mathematical framework for reasoning under uncertainty, which is an unavoidable feature of most real-world AI problems. Sensors are noisy, observations are incomplete, models are approximations of reality, and the future is inherently unpredictable. AI systems that cannot reason probabilistically will make brittle, overconfident decisions whenever the world does not match their internal model exactly.
The foundation is Bayes' theorem, which describes how to update beliefs in light of new evidence. It states that the posterior probability of a hypothesis given evidence equals the prior probability of the hypothesis multiplied by the likelihood of the evidence given the hypothesis, divided by the marginal probability of the evidence. In notation: P(H given E) equals P(E given H) times P(H) divided by P(E). This provides a principled way to combine prior knowledge with observations to form updated beliefs.
Joint probability distributions describe the probability of every possible combination of values of a set of variables. For a small number of discrete variables this can be stored as a table. For continuous or high-dimensional variables, representing the full joint distribution is intractable, motivating the use of compact representations like Bayesian networks that exploit conditional independence among variables to reduce the representation to manageable size.
Probabilistic Inference
Given a joint distribution or a model like a Bayesian network, probabilistic inference answers queries such as: given that I observe certain symptoms, what is the probability of each disease? Exact inference using variable elimination computes answers by summing out unobserved variables. For large networks, approximate inference using Monte Carlo sampling or variational methods is used instead.
Exam Tip
Exam questions on probability in AI almost always involve applying Bayes' theorem to a word problem. Work through the problem by first identifying the hypothesis, the evidence, the prior, and the likelihood from the problem statement, then substitute into the formula. Writing out each term clearly before computing prevents errors. Also know the law of total probability: P(E) equals the sum over all hypotheses of P(E given H) times P(H).