Bayesian network
A Bayesian network is a directed acyclic graph in which each node represents a random variable and each directed edge represents a direct probabilistic dep...
Compact Probabilistic Models
A Bayesian network is a directed acyclic graph in which each node represents a random variable and each directed edge represents a direct probabilistic dependency from parent to child. Every node is associated with a conditional probability table that specifies the probability of each value of that variable given every combination of values of its parent variables. The network's structure encodes conditional independence assumptions: a variable is conditionally independent of all non-descendants given its parents, a property known as the Markov condition.
The power of Bayesian networks lies in their compact representation of joint probability distributions. A naive joint distribution over n binary variables requires two to the power of n minus one parameters. A Bayesian network with sparse dependencies may require far fewer. For example, a network modelling five binary variables with at most two parents each needs at most five times four parameters, dramatically less than thirty-one for the full joint.
A classic Bayesian network example involves a burglar alarm. The variables are Burglary, Earthquake, Alarm, JohnCalls, and MaryCalls. Burglary and Earthquake are root nodes with prior probabilities. Alarm depends on both. JohnCalls and MaryCalls each depend only on Alarm. Given this network, one can compute the probability that a burglary occurred given that John calls but Mary does not, using Bayes' theorem applied through the network structure via algorithms like variable elimination.
Learning Bayesian Networks
When the network structure is known, parameters are learned by counting frequencies in training data. When the structure is unknown, structure learning algorithms use scoring functions such as the Bayesian Information Criterion to search for the directed acyclic graph that best fits the data. This allows automatic discovery of dependencies from observational data.
Interview Tip
Be ready to explain both inference and learning for Bayesian networks. For inference, know that queries can be exact or approximate, and that exact inference is NP-hard in general but tractable for networks with small treewidth. For a concrete application, describe medical diagnosis: the network encodes prior disease probabilities and symptom likelihoods, and inference updates disease probabilities given observed symptoms. This directly mirrors how MYCIN's probabilistic successors operate.