Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics

This paper focuses on robotic reinforcement learning with sparse rewards for natural language goal representations. An open problem is the sample-inefficiency that stems from the compositionality of natural language, and from the grounding of language in sensory data and actions. We address these issues with three contributions. We first present a mechanism for hindsight instruction replay utilizing expert feedback. Second, we propose a seq2seq model to generate linguistic hindsight instructions. Finally, we present a novel class of language-focused learning tasks. We show that hindsight instructions improve the learning performance, as expected. In addition, we also provide an unexpected result: We show that the learning performance of our agent can be improved by one third if, in a sense, the agent learns to talk to itself in a self-supervised manner. We achieve this by learning to generate linguistic instructions that would have been appropriate as a natural language goal for an originally unintended behavior. Our results indicate that the performance gain increases with the task-complexity.

@InProceedings{REW22, 
 	 author =  {Röder, Frank and Eppe, Manfred and Wermter, Stefan},  
 	 title = {Grounding Hindsight Instructions in Multi-Goal Reinforcement Learning for Robotics}, 
 	 booktitle = {2022 IEEE International Conference on Development and Learning (ICDL)},
 	 editors = {},
 	 number = {},
 	 volume = {},
 	 pages = {170--177},
 	 year = {2022},
 	 month = {Sep},
 	 publisher = {IEEE},
 	 doi = {10.1109/ICDL53763.2022.9962207}, 
 }