Reinforcement Learning Embedded in Brains and Robots

In many ways and in various tasks, computers are able to outperform humans. They can store and retrieve much larger amounts of data or even beat humans at chess. However, when looking at robots they are still far behind even a small child in terms of their performance capabilities. Even a sophisticated robot, such as ASIMO, is limited to mostly pre-programmed behaviours (Weigmann, 2006). The reliance on robots that must be carefully programmed and calibrated before use and thereafter whenever the task changes, is quite unacceptable for robots that have to coexist and cooperate with humans, especially those who are not necessarily knowledgeable about robotics. Therefore there is an increasing need to go beyond robots that are pre-programmed explicitly towards those that learn and are adaptive (Wermter, Weber & Elshaw, 2004; Wermter, Weber, Elshaw, Panchev et al., 2004). Natural, dynamic environments require robots to adapt their behaviour and learn using approaches typically used by animals or humans. Hence there is a necessity to develop novel methods to provide such robots with the learning ability to deal with human competence. Robots shall learn useful tasks, i.e. tasks in which a goal is reached, if executed successfully. Reinforcement learning (RL) is a powerful method to develop goal-directed action strategies (Sutton & Barto, 1998). In RL, the agent explores a ‘state space’ which describes his situation within the environment, by taking randomized actions that take him from one state to another. Crucially, a reward is received only at the final goal state, in case of successful completion. Over many trials, the agent learns the value of all states (in terms of reward proximity), and how to get to higher-valued states to reach the goal. In Section 2 we will review RL in the brain, focusing on the basal ganglia, a group of nuclei in the forebrain implicated in RL.

@InCollection{WEWTW08, 
 	 author =  {Weber, Cornelius and Elshaw, Mark I. and Wermter, Stefan and Triesch, Jochen and Willmot, Christopher},  
 	 title = {Reinforcement Learning Embedded in Brains and Robots}, 
 	 booktitle = {Reinforcement Learning: Theory and Applications},
 	 editors = {Weber, C., Elshaw M., and Mayer N. M.},
 	 number = {},
 	 volume = {},
 	 pages = {119--142},
 	 year = {2008},
 	 month = {Jan},
 	 publisher = {I-Tech Education and Publishing},
 	 doi = {10.5772/5278}, 
 }