Exercise 6.2.2
Linear function approximation: design two features for grid world
Reinforcement Learning › Exercises › Lecture 6 · Function Approximation › 6.2 Linear function approximation
Consider an agent in a gigant grid world , where the state \(s\) is defined by the coordinates of the agent \((x_a, y_a)\), a food pellet \((x_f, y_f)\), and a ghost \((x_g, y_g)\). The state space is so large that the agent will only ever experience a tiny fraction of all possible configurations.
Your task is to design a feature vector \(\mathbf{x}(s)\) for a linear value function approximator. Propose a set of two features. For each feature, justify how it helps the agent to generalize its learning, allowing it to make sensible decisions in states it has never visited before.