Subtleties and Ongoing Research

• Scalability
• Generalize utilities from visited states to other states (inductive learning)
• Handle case where state only partially observable
• Design optimal exploration strategies
• Extend to continuous actions, states