Back to Study Material Artificial Intelligence

Stochastic environment

A stochastic environment is one where the outcome of an action cannot be fully predicted because randomness or unknown factors influence state transitions....

Stochastic environment

Planning Under Uncertainty

A stochastic environment is one where the outcome of an action cannot be fully predicted because randomness or unknown factors influence state transitions. Even if the agent knows the current state perfectly and chooses an action deliberately, the resulting state is uncertain. The agent can only assign probabilities to possible outcomes.

Handling stochasticity requires different tools than deterministic planning. Instead of seeking a fixed sequence of actions, the agent must find a policy — a mapping from states to actions that performs well on average across all possible random outcomes. Techniques from decision theory, Markov decision processes (MDPs), and reinforcement learning are the standard approaches.

A Markov decision process models the environment with states, actions, transition probabilities, and rewards. The agent's goal is to find a policy that maximizes expected cumulative reward. Value iteration and policy iteration are classical algorithms that solve MDPs exactly. When the state space is too large, reinforcement learning algorithms like Q-learning and policy gradient methods learn good policies through repeated interaction with the environment.

A concrete example: robot navigation in a building with people moving around it. Even if the robot chooses to move forward, a passerby might cross its path, causing an unexpected obstacle. The robot must plan for this possibility, either slowing down to reduce collision risk or selecting routes with fewer pedestrians.

Backgammon is a famous stochastic board game. Dice rolls introduce randomness after each player's turn. TD-Gammon, an early reinforcement learning system trained by Gerald Tesauro, learned to play backgammon at expert level specifically by handling this stochasticity through temporal difference learning.

Exam tip: Distinguish stochastic environments from non-deterministic ones. Stochastic means outcomes follow known probability distributions. Non-deterministic means the agent has no probabilistic model of outcomes at all. Examiners sometimes use these terms interchangeably, but in formal AI they differ.